Data processing method and related equipment

Abstract

The embodiment of the invention discloses a data processing method and related equipment, the method can apply an artificial intelligence technology to the field of communication, and the method comprises the steps that first state information is acquired, and the first state information indicates the congestion condition of a network; first prediction information of first control information is obtained, the first control information is any one in a control information set, the control information set comprises multiple pieces of control information, and each piece of control information indicates the sending rate of data; the first prediction information of the first control information indicates advantages and disadvantages of adopting the first control information in the network state indicated by the first state information; and determining target control information from the control information set according to the first prediction information of each piece of control information. According to the scheme, the quality condition of each piece of control information is evaluated in combination with the congestion condition of the network, so that the adaptation degree between the target control information and the congestion condition of the network can be improved, and a better data transmission rate can be explored.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence, and in particular to a data processing method and related equipment. Background Art

[0002] With the continuous development of the information industry, the demand for data in various application scenarios is increasing. For example, in application scenarios such as data express and edge cloud computing, the amount of data transmission is variable, which requires the network to be able to perceive the changes in the transmission throughput rate in the shortest possible time, and then adaptively adjust the data transmission rate. In order to cope with the challenges brought by the above changes, congestion control methods are currently used. The goal of congestion control includes adjusting the data transmission rate to alleviate or avoid congestion, so as to meet the requirements for low latency in data transmission to a certain extent.

[0003] Specifically, during data transmission, the initial value of the receive window (receive window, RWND) can be set to infinity, and the value of the congestion window (congestion window, CWND) increases continuously at a first growth rate until the length of the receive queue reaches the first length, and the value of CWND continues to increase at a second growth rate, and the value of RWND is modified to 1. Since the data transmission rate is determined based on the minimum value of CWND and RWND, the data transmission rate is greatly reduced after the value of RWND is modified to 1; when the length of the receive queue is reduced to the second length, the value of RWND is modified to infinity again, and the value of CWND continues to increase at the first growth rate again; the aforementioned steps are repeated at least once until a packet loss event occurs, the value of CWND is halved, and the value of RWND is modified to 1, that is, one cycle is completed to continue the next cycle.

[0004] However, in the above congestion control algorithm, the value of RWND is either 1 or infinite. The congestion control algorithm is relatively rough as a whole, which is not conducive to exploring a better data transmission rate. Summary of the invention

[0005] The embodiments of the present application provide a data processing method and related equipment, which solve the problem in the prior art that the congestion control algorithm is rough and it is difficult to obtain a better data transmission rate.

[0006] The embodiments of the present application provide the following technical solutions:

[0007] In the first aspect, an embodiment of the present application provides a data processing method, which can apply artificial intelligence technology to the field of communications. In this method, a first device obtains first status information, and the first status information indicates the congestion of the network; the first device obtains first prediction information of first control information; and then, according to the first prediction information of each control information in the control information set, the target control information can be determined from the control information set, and the target control information indicates a data transmission rate. Among them, the first control information is any one of the control information set, and the control information set includes multiple control information, which indicates the transmission rate of data, and the first prediction information of the first control information indicates the advantages and disadvantages of using the first control information under the network state indicated by the first status information.

[0008] In this implementation, each control information in the control information set indicates a data transmission rate. In the process of determining the target control information finally selected, the first prediction information of each control information is first obtained. Since the first prediction information of the first control information can indicate the pros and cons of using the first control information under the network state indicated by the first state information, the pros and cons of using each control information under the network state indicated by the first state information is obtained, and then a target control information is selected from the control information set based on the pros and cons of each control information. Through the above scheme, a more refined congestion control method is provided. Since in this scheme, each time the target control information to be used is determined, the pros and cons of each control information are evaluated in combination with the congestion situation of the network, which is conducive to improving the adaptability between the target control information and the congestion situation of the network, and is conducive to making the finally determined data transmission rate more consistent with the control information of the actual congestion situation of the network, that is, it is conducive to exploring a better data transmission rate.

[0009] In a possible implementation, the first state information may indicate the congestion situation of the network at time t, where t is an integer greater than or equal to 1. For example, the first state information may include any one or more of the following information: the actual usage of the congestion queue at the starting time point of time t, the queue entry rate and the queue exit rate of the congestion queue at time t-1, the average value of the round-trip time (RRT) of all data transmission units transmitted at time t-1, the median value of the RRT of all data transmission units transmitted at time t-1, the maximum value of the RRT of all data transmission units transmitted at time t-1, or other information that can reflect the congestion situation of the network.

[0010] Both “time t-1” and “time t” may be expressed as a time period having a first duration. For example, the first duration may be 5 seconds, 2 seconds, 1 second, 1500 milliseconds, 800 milliseconds or other lengths.

[0011] In one possible implementation, a specific implementation method for the first device to obtain the first prediction information of the first control information may include: the first device inputs the first state information and the first control information into the first machine learning model to obtain the first prediction information output by the aforementioned first machine learning model. In this implementation, the input of the machine learning model includes the first state information and the first control information, and the output of the machine learning model is directly the first prediction information of the first control information, which is conducive to quickly obtaining the first prediction information of the first control information after obtaining the first state information and the first control information, thereby facilitating improving the speed of finally obtaining the target control information.

[0012] In a possible implementation, the first state information is used to indicate the congestion of the network at time t, where t is an integer greater than or equal to 1, and when the weight parameter of the machine learning model adopts the first parameter value, it is the first machine learning model. N second parameter values ​​may be deployed in the first device, each second parameter value includes the parameter value adopted by the weight parameter of the machine learning model at one of the N moments before time t, where N is an integer greater than or equal to 1. Exemplarily, the first parameter value adopted in the first machine learning model can be obtained by updating the third parameter value based on the second state information at time t-1, the second control information adopted at time t-1, and the actual performance information brought about by adopting the second control information at time t-1; wherein the third parameter value is one of the N second parameter values, the state information at time t-1 indicates the congestion of the network at time t-1, the second control information indicates the data transmission rate adopted in the network state at time t-1, and the actual performance information brought about by adopting the second control information at time t-1 represents the pros and cons of adopting the second control information in the network at time t-1.

[0013] Exemplarily, the third parameter value may be a set of parameter values ​​adopted by the machine learning model when determining the second control information adopted at time t-1; or, the third parameter value may also be any one of the N second parameter values.

[0014] In this implementation, the first machine learning model adopted at time t is determined based on the historical data at time t-1 and the second machine learning model adopted N times before time t. That is, the first parameter value adopted by the first machine learning model is timely updated using the latest historical data, which is conducive to improving the performance of the first machine learning model and thus conducive to obtaining better target control information.

[0015] In one possible implementation, when the weight parameter of the machine learning model adopts the second parameter value, it is a second machine learning model, that is, the difference between the first machine learning model and the second machine learning model is that different weight parameters are adopted. The method also includes: the first device inputs the second state information and the second control information into N second machine learning models corresponding to the N second parameter values, and can obtain a second prediction information of the second control information generated by each second machine learning model in the N second machine learning models, that is, can obtain N second prediction information corresponding to the second control information generated by the N second machine learning models; then the first device can update the third parameter value according to the N second prediction information and the actual performance information to obtain the first parameter value, wherein the second prediction information represents the pros and cons of adopting the second control information in the network indicated by the second state information.

[0016] Exemplarily, the first device can calculate the function value of the first loss function based on the N second prediction information corresponding to the second control information and the actual performance information brought about by adopting the second control information at time t-1, and use the back propagation algorithm to perform gradient update on the third parameter value to obtain the first parameter value based on the function value of the first loss function.

[0017] In this implementation, the pros and cons of adopting the second control information are predicted respectively based on the N second machine learning models before time t (that is, N second prediction information), and then the third parameter value is updated based on the actual performance information of the N second prediction information and the second control information, so as to obtain the first machine learning model adopted at time t, that is, the process of determining the value of the weight parameter of the first machine learning model combines the second machine learning model adopted at N moments before time t, and the second machine learning model adopted at N moments before time t has seen N kinds of congestion conditions of the network, which is conducive to obtaining a first machine learning model with better performance, and then conducive to obtaining better target control information, so as to improve the utilization rate of network communication resources while minimizing network congestion.

[0018] In one possible implementation, when each second prediction information is expressed as a comprehensive score value, the first device can obtain the distance between each second prediction information in the N second prediction information and the actual performance information, that is, N distances corresponding to the N second prediction information can be obtained; the first device can also obtain the weight value corresponding to each second prediction information in the N second prediction information, and perform weighted summation of the N distances according to the weight value of each second prediction information to obtain the sum result, that is, the function value of the first loss function; wherein the first loss function indicates the similarity between the N second prediction information and the actual performance information, and the goal of training using the first loss function includes improving the similarity between the N second prediction information and the actual performance information.

[0019] In one possible implementation, the first status information indicates the congestion status of the network at time t, where t is an integer greater than or equal to 1. The specific implementation method for the first device to obtain the first prediction information of the first control information may include: the first device inputs the first status information and the first control information into a third machine learning model, obtains third prediction information output by the third machine learning model, and then determines the first prediction information based on the third prediction information; wherein the third prediction information indicates the congestion status of the network at time t+1 when the first control information is used in the network state indicated by the first status information.

[0020] Optionally, in one case, the first prediction information corresponding to the first state information indicates the difference between the predicted state information and the expected state information of the network at time t+1 when the first control information is adopted in the network state indicated by the first state information. Then, the specific implementation method for the first device to determine the first prediction information according to the third prediction information may include: the first device determines the first prediction information according to the third prediction information and the expected state information.

[0021] In this implementation, another scheme for obtaining the first prediction information of the first control information is provided, which improves the implementation flexibility of the scheme, and firstly predicts the congestion situation of the network at time t+1 based on the first state information and the first control information using a machine learning model, and then determines the pros and cons of using the first control information under the network state indicated by the first state information based on the congestion situation of the network at time t+1, so that the generation process of the first prediction information of the first control information is more refined, which is conducive to obtaining more accurate first prediction information of the first control information, and then is conducive to obtaining target control information that is more adapted to the network at time t, and is conducive to more effective use of the network bandwidth for data transmission within time t.

[0022] In one possible implementation, a first distribution space is modeled by a third machine learning model, the first distribution space represents a distribution space where at least one information pair is located, each of the at least one information pair includes a combination information and predicted state information corresponding to the combination information, the combination information includes a third state information and a third control information, the third state information indicates a network congestion condition, the third control information is any one in a control information set, and the state information corresponding to the combination information indicates a network congestion condition at the next moment when the third control information is adopted in the network state indicated by the third state information.

[0023] In this implementation, the third machine learning model is used to model the first distribution space. The first distribution space represents the distribution space of information pairs consisting of (state information, control information) and the state information at the next moment. The first distribution space is used to determine the first prediction information of each control information. That is, a specific implementation scheme for determining the first prediction information of each control information is provided, which helps to reduce the difficulty of implementing this scheme.

[0024] In one possible implementation, in the method, the first device may also obtain the second state information at time t-1 and the second control information used at time t-1, and then update the posterior distribution of the third machine learning model according to the state information at time t-1, the second control information and the first state information, to obtain an updated third machine learning model; wherein the aforementioned state information at time t-1 indicates the congestion of the network at time t-1. The specific implementation method in which the first device obtains the third prediction information based on the third machine learning model according to the first state information and the first control information may include: the first device obtains the third prediction information from the distribution space indicated by the updated third machine learning model according to the first state information and the first control information.

[0025] In this implementation method, after obtaining the historical data at time t-1, the historical data at time t-1 will be used in a timely manner to update the posterior distribution of the third machine learning model, which is beneficial to improving the similarity between the first distribution space indicated by the third machine learning model and the real distribution space where at least one information pair is located. That is, it is beneficial to model the real distribution space where at least one information pair is located through the third machine learning model, which is beneficial to obtain more accurate prediction state information with the help of the updated third machine learning model, and then it is beneficial to obtain more accurate first prediction information, which is beneficial to select better target control information.

[0026] In one possible implementation, the first state information indicates the congestion situation of the network at time t, and the first prediction information of the first state information indicates the difference between the predicted state information and the expected state information of the network at time t+1 when the first control information is adopted in the network state indicated by the first state information; wherein the predicted state information of the network at time t+1 represents the predicted congestion situation of the network at time t+1, and the expected state information represents the expected congestion situation of the network.

[0027] For example, the first prediction information of the first control information may be a first score, and the first score may include a score value corresponding to the parameter value of each first indicator and / or a score value corresponding to the parameter value of each second indicator. Alternatively, the first score may be a first comprehensive score value obtained by weighted summing the score value corresponding to each first indicator and / or the score value corresponding to each second indicator.

[0028] In this implementation, the first prediction information of the first control information indicates the difference between the predicted state information and the expected state information of the network at time t+1 when the first control information is adopted in the network state indicated by the first state information. Since the expected state information indicates the expected congestion situation of the network, the advantages and disadvantages of adopting the first control information are measured with the help of the expected state information, which is conducive to narrowing the similarity between the actual congestion situation of the network and the expected congestion situation, thereby facilitating users to more conveniently regulate the congestion situation of the network.

[0029] In one possible implementation, the first prediction information of the first control information may include only the predicted number of messages successfully sent within time t. Alternatively, the first prediction information of the first control information may include a second score; for example, the second score may include a score value corresponding to the parameter value of each third indicator in at least one third indicator, and / or a score value corresponding to the predicted number of messages successfully sent within time t. Exemplarily, the aforementioned at least one third indicator may be used to indicate the predicted congestion situation of the network at the starting time point of time t+1, for example, the aforementioned at least one third indicator may indicate: the predicted usage of the congestion queue at the starting time point of time t+1, and / or, the predicted value corresponding to the RTT of all messages transmitted within time t.

[0030] In one possible implementation, the specific implementation method of the first device determining the target control information from the control information set based on the first prediction information of each control information may include: the first device determines the target control information from the control information set based on the first prediction information of each control information and the uncertainty corresponding to each of the aforementioned control information; wherein the uncertainty corresponding to each control information represents the uncertainty of the combination of each control information and the first state information.

[0031] Optionally, the uncertainty corresponding to each control information can be determined based on a contextual multi-armed bandit (CMAB) algorithm, wherein the higher the uncertainty score of each control information, the higher the uncertainty of the combination of each control information and the first state information; the lower the uncertainty score of each control information, the lower the uncertainty of the combination of each control information and the first state information.

[0032] In this implementation, not only the advantages and disadvantages of each control information are considered, but also the uncertainty of the combination of each control information and the first state information is considered, which is conducive to actively exploring combinations with high uncertainty, and further conducive to fully exploring the advantages and disadvantages of each control information, so as to increase the probability that the target control information finally obtained is the optimal control information.

[0033] In a possible implementation, the uncertainty corresponding to each control information may be specifically expressed as an uncertainty score for each control information. A specific implementation of the first device determining the target control information from the control information set according to the first prediction information of each control information and the uncertainty corresponding to each of the aforementioned control information may include: the first device weightedly sums the first comprehensive score value (or the second comprehensive score value) of each control information with the uncertainty score of each control information, thereby obtaining a total score for each control information; and then determining the target control information from the control information set according to the total score of each control information.

[0034] In a possible implementation, each control information includes a value of a receiving window RWND and / or a value of a sending window SWND. In this implementation, the specific information carried by the control information is detailed, which is conducive to reducing the difficulty of implementing this solution. In addition, the RWND parameter exists in different versions of the transmission control protocol (TCP). Selecting the value of RWND to be carried in the control information is conducive to improving the universality of this solution, that is, it is conducive to expanding the application scenarios of this solution.

[0035] In a second aspect, an embodiment of the present application provides a data processing device, which can apply artificial intelligence technology to the field of communications, and the data processing device includes: an acquisition module, used to obtain first status information, the first status information indicating the congestion of the network; the acquisition module, also used to obtain first prediction information of first control information, wherein the first control information is any one in a control information set, the control information set includes multiple control information, each of the multiple control information indicates the data sending rate, and the first prediction information of the first control information indicates the pros and cons of using the first control information under the network state indicated by the first status information; a determination module, used to determine the target control information from the control information set according to the first prediction information of each control information.

[0036] In the second aspect, the data processing device can also execute the steps performed by the first device in the first aspect and various possible implementation methods of the first aspect. The specific implementation methods, meanings of terms and beneficial effects brought about by the steps in each possible implementation method of the second aspect of this application can all be referred to the first aspect and will not be repeated here.

[0037] In a third aspect, an embodiment of the present application provides a device, including a processor and a memory, wherein the processor is coupled to the memory, the memory is used to store programs; the processor is used to execute the programs in the memory, so that the device executes the data processing method of the first aspect mentioned above.

[0038] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored. When the computer-readable storage medium is run on a computer, the computer executes the data processing method of the first aspect.

[0039] In a fifth aspect, an embodiment of the present application provides a computer program product, which includes a program. When the program is run on a computer, the computer executes the data processing method of the first aspect.

[0040] In a sixth aspect, the present application provides a chip system, which includes a processor for supporting the implementation of the functions involved in the above aspects, for example, sending or processing the data and / or information involved in the above methods. In one possible design, the chip system also includes a memory, which is used to store program instructions and data necessary for a terminal device or a communication device. The chip system can be composed of a chip, or it can include a chip and other discrete devices. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 A schematic diagram of an architecture of a communication system provided in an embodiment of the present application;

[0042] Figure 2 A flowchart of a data processing method provided in an embodiment of the present application;

[0043] Figure 3 Another flowchart of the data processing method provided in the embodiment of the present application;

[0044] Figure 4 A schematic diagram for determining a first parameter value provided in an embodiment of the present application;

[0045] Figure 5 Another flowchart of the data processing method provided in the embodiment of the present application;

[0046] Figure 6 Another flowchart of the data processing method provided in the embodiment of the present application;

[0047] Figure 7 Another flowchart of the data processing method provided in the embodiment of the present application;

[0048] Figure 8 A schematic diagram of the structure of a data processing device provided in an embodiment of the present application;

[0049] Fig. 9 A schematic diagram of the structure of a device provided in an embodiment of the present application;

[0050] Fig.10A schematic diagram of the structure of a chip provided in an embodiment of the present application. DETAILED DESCRIPTION

[0051] The embodiments of the present application are described below in conjunction with the accompanying drawings. Those skilled in the art will appreciate that, with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0052] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and need not be used to describe a specific order or sequential order. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, which is only to describe the distinction mode adopted by the objects of the same attributes when describing in the embodiments of the present application. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, so that the process, method, system, product or equipment comprising a series of units need not be limited to those units, but may include other units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0053] "Send" and "receive" in the embodiments of the present application indicate the direction of signal transmission. For example, "send information to XX device" can be understood as the destination of the information is XX device, which can include direct transmission through the air interface, and also include indirect transmission through the air interface by other units or modules. "Receive information from YY device" can be understood as the source of the information is YY device, which can include direct reception from YY device through the air interface, and can also include indirect reception from YY device through other units or modules through the air interface. "Send" can also be understood as the "output" of the chip interface, and "receive" can also be understood as the "input" of the chip interface. In other words, sending and receiving can be carried out between devices or within the device, for example, sending or receiving between components, modules, chips, software modules or hardware modules within the device through a bus, a wiring or an interface. It is understandable that the information may be processed as necessary between the source and the destination of the information transmission, such as encoding, modulation, etc., but the destination can understand the valid information from the source. Similar expressions in this application can be understood in a similar way and will not be repeated.

[0054] In the embodiments of the present application, "indication" may include direct indication and indirect indication, and may also include explicit indication and implicit indication. The information indicated by a certain information (such as the indication information described below) is called information to be indicated. In the specific implementation process, there are many ways to indicate the information to be indicated, such as but not limited to, directly indicating the information to be indicated, such as the information to be indicated itself or the index of the information to be indicated. The information to be indicated may also be indirectly indicated by indicating other information, wherein there is an association between the other information and the information to be indicated; it may also be possible to indicate only a part of the information to be indicated, while the other part of the information to be indicated is known or agreed in advance, for example, the indication of specific information can be realized by means of the arrangement order of each information agreed in advance (such as predefined by the protocol), thereby reducing the indication overhead to a certain extent. The present application does not limit the specific method of indication. It is understandable that, for the sender of the indication information, the indication information can be used to indicate the information to be indicated, and for the receiver of the indication information, the indication information can be used to determine the information to be indicated.

[0055] The method provided in this application can be applied in the field of communications. For example, it can be applied in the application scenario of data transmission to adjust the data transmission rate of the transmitting end; optionally, a machine learning model can be used in the process of determining the aforementioned data transmission rate. Before describing the method provided in this application in detail, the scenario in which the method provided in the embodiment of this application is applied is first introduced.

[0056] Please read first Figure 1 , Figure 1 The method provided by the present application can be applied to a wide area network, which is a remote network connecting local area networks or metropolitan area networks in different regions, such as Figure 1 As shown, the sending end and the receiving end are connected to each other through a wide area network, and when data is transmitted between the sending end and the receiving end, it may be forwarded through multiple transit devices.

[0057] For example, the transit device may include but is not limited to a router, a firewall, a gateway or other network device with a forwarding function, etc., which are not limited in this application.

[0058] For example, the sending end (or receiving end) may be specifically manifested as a server, a server cluster, or other devices.

[0059] For another example, the transmitting end (or receiving end) may be a handheld device, a vehicle-mounted device, a wearable device, a computing device or other processing device with wireless communication function, which are not exhaustive here. The transmitting end (or receiving end) may communicate with one or more core networks. For example, the transmitting end (or receiving end) may be a portable, pocket-sized, handheld, computer-built-in or vehicle-mounted mobile device, which exchanges voice and / or data with the wireless access network. Exemplarily, the transmitting end (or receiving end) may be a user agent, a cellular phone, a smart phone, a personal digital assistant (PDA), a tablet PC (Tablet Personal Computer), a wireless modem, a handheld device (handset), a laptop computer, a personal communication service (PCS) phone, a remote station (remotestation), an access point (AP), a remote terminal, an access terminal, customer premises equipment (CPE), a terminal, user equipment (UE) or a mobile terminal (MT), etc.

[0060] For another example, the transmitting end (or receiving end) can also be a wearable device, which is a general term for the use of wearable technology to intelligently design and develop wearable devices for daily wear, such as glasses, gloves, watches, clothing and shoes. A wearable device is a portable device that is worn directly on the body or integrated into the user's clothes or accessories. Wearable devices are not just hardware devices, but also achieve powerful functions through software support, data interaction, and cloud interaction. Broadly speaking, wearable smart devices include full-featured, large-sized, and independent of smartphones to achieve complete or partial functions, such as smart watches or smart glasses, as well as those that only focus on a certain type of application function and need to be used in conjunction with other devices such as smartphones, such as various smart bracelets, smart helmets, and smart jewelry for vital sign monitoring.

[0061] For another example, the transmitting end (or receiving end) may also be a drone, a robot, a terminal device in device-to-device (D2D) communication, a terminal device in vehicle to everything (V2X), a virtual reality (VR) device, an augmented reality (AR) device, a wireless terminal in industrial control, a terminal device in self driving, a terminal device in remote medical, a terminal device in a smart grid, a wireless terminal in a smart city, a terminal device in a smart home, etc.

[0062] In addition to the above Figure 1 In addition to the scenarios described above, the method provided by the embodiments of the present application can also be applied to other communication system scenarios. For example, in a smart factory scenario, different devices (such as smart robots, lathes, transport vehicles, etc.) are connected to the same communication network through a router and transmit data to each other through the communication network; the control information determined by the router can indicate the data transmission rate of the aforementioned devices, etc. The embodiments of the present application do not limit the specific scenarios in which the data processing method is applied.

[0063] In the above application scenarios, when data is transmitted between the transmitter and the receiver, in order to explore a better data transmission rate, the present application provides a data processing method, which is executed by a first device. Optionally, the first device can be a transit device, and the first device can be connected to one or more transmitters. For example, the first device can be specifically manifested as a switch, a router, a firewall, a base station or other network device with forwarding function, etc. The specific deployment in which the device is deployed can be flexibly determined in combination with the actual application scenario, and is not limited in the embodiments of the present application. For details, please refer to Figure 2 , Figure 2 A schematic diagram of a data processing method provided in an embodiment of the present application. The data processing method provided in the present application may include:

[0064] 201. Obtain first status information, where the first status information indicates a congestion situation of a network.

[0065] In an embodiment of the present application, the first device may obtain first status information. Exemplarily, the first status information may indicate the congestion situation of the network at time t, where t is an integer greater than or equal to 1. For example, the first status information may include any one or more of the following information: the actual usage of the congested queue at the starting time point of time t, the queue entry rate and queue exit rate of the congested queue at time t-1, the average value of the round-trip time (RRT) of all data transmission units transmitted at time t-1, the median value of the RRT of all data transmission units transmitted at time t-1, the maximum value of the RRT of all data transmission units transmitted at time t-1, or other information that can reflect the congestion situation of the network, etc. The specific information to be used can be flexibly determined in combination with the actual application scenario and is not limited here.

[0066] Exemplarily, "the actual usage of the congested queue at the starting time point of time t" may include the actual number of data transmission units to be transmitted in the congested queue, and / or the ratio between the actual number and the total length of the congested queue, and / or other parameter values ​​that can reflect the actual usage of the congested queue at the starting time point of time t; "the total length of the congested queue" can also be understood as "the total number of data transmission units that the congested queue can accommodate".

[0067] The "data transmission unit" can be specifically expressed as a message, a data packet or other types of data transmission units in the network, etc. The specific form of the "data transmission unit" can be determined in combination with the actual application scenario, and is not limited in this application; it should be noted that in the subsequent embodiments of this application, only the "data transmission unit" expressed as a message is used as an example for explanation. For the case where the "data transmission unit" is expressed in other forms, you can refer to the case where the "data transmission unit" is expressed as a message for understanding, and will not be repeated one by one in the embodiments of this application.

[0068] Both "time t-1" and "time t" can be expressed as a time period with a first duration. For example, the first duration can be 5 seconds, 2 seconds, 1 second, 1500 milliseconds, 800 milliseconds or other lengths, etc. The examples given here are only for the convenience of understanding this solution and are not used to limit this solution.

[0069] The RRT of a data transmission unit represents the total delay from the time when the sender starts sending the data transmission unit to the time when the sender receives the information sent by the receiver confirming the receipt of the data transmission unit.

[0070] 202. Obtain first prediction information of first control information, wherein the first control information is any one of a control information set, the control information set includes multiple control information, each of the multiple control information indicates a data sending rate, and the first prediction information of the first control information indicates the pros and cons of using the first control information under the network state indicated by the first state information.

[0071] In an embodiment of the present application, the first device can obtain a control information set, which includes multiple control information; after determining the first state information, the first prediction information of any one control information in the control information set (that is, the first control information) can be obtained; the first device repeats the aforementioned steps multiple times, and can obtain the first prediction information of each control information in the control information set. The "first prediction information of each control information" can also be understood as the prediction performance information of each control information, and the first prediction information of each control information indicates the pros and cons of adopting the control information under the network state indicated by the first state information.

[0072] Exemplarily, in one case, if each control information indicates a message transmission rate, each control information may include a value of RWND, and / or a value of a send window (SWND). If the control information carries a value of RWND, the sender can determine the final SWND value based on the value of RWND carried in the control information and the value of CWND determined by the sender. If the control information carries a value of SWND, the sender can determine the final SWND value based on the value of SWND carried in the control information. In the embodiment of the present application, the specific information carried by the control information is refined, which is conducive to reducing the difficulty of implementing the present solution, and the parameter RWND exists in different versions of the transmission control protocol (TCP). Selecting the value of RWND carried in the control information is conducive to improving the universality of the present solution, that is, it is conducive to expanding the application scenarios of the present solution.

[0073] In another case, if each control message indicates the sending rate of the data packet, each control message may also carry other information. The specific information carried in each control message can be determined in combination with the actual application scenario and is not limited here.

[0074] Regarding the meaning of "first prediction information of the first control information", in one case, the first prediction information of the first control information may indicate the difference between the predicted state information at time t+1 and the expected state information when the first control information is adopted under the network state indicated by the first state information (i.e., the network state at time t); and / or, the first prediction information of the first control information may include the difference between the predicted number and the expected number of messages successfully sent at time t.

[0075] For example, the first prediction information of the first control information may be a first score, and the first score may include a score value corresponding to the parameter value of each first indicator and / or a score value corresponding to the parameter value of each second indicator. Alternatively, the first score may be a first comprehensive score value obtained by weighted summing the score value corresponding to each first indicator and / or the score value corresponding to each second indicator.

[0076] The parameter value of the at least one first indicator mentioned above may indicate any one or more of the following information: the difference between the predicted usage and the expected usage of the congested queue at the starting time point of time t+1, the difference between the predicted value and the expected value corresponding to the RTT of all packets transmitted within time t, or other information, etc., that is, the expected state information may include any one or more of the following information: the expected usage of the congested queue, the expected value of RTT or other information, etc., which are not exhaustive here.

[0077] Furthermore, "the difference between the predicted usage and the expected usage of the congested queue at the starting time point of time t+1" can be specifically reflected by any one or more of the following first indicators: the difference between the predicted number and the expected number of messages to be transmitted by the congested queue at time t+1; the ratio between the predicted number and the expected number of messages to be transmitted by the congested queue at time t+1; the difference between the first ratio and the second ratio, the first ratio represents the ratio between the predicted number of messages to be transmitted by the congested queue at time t+1 and the total length of the congested queue, and the second ratio represents the ratio between the expected number of messages to be transmitted in the congested queue and the total length of the congested queue; or other indicators, etc., which are not exhaustive here.

[0078] "The predicted value corresponding to the RTT of all messages transmitted within time t" may include any one or more of the following: the average value of the RRT of all messages transmitted within time t, the median value of the RRT of all messages transmitted within time t, the maximum value of the RRT of all messages transmitted within time t, or other values ​​obtained based on the values ​​of the RRT of all messages transmitted within time t, etc.; correspondingly, "the expected value of RTT" may include any one or more of the following: the average value of the RRT of all messages transmitted within a unit time, the median value of the RRT of all messages transmitted within a unit time, the maximum value of the RRT of all messages transmitted within a unit time, or other values ​​obtained based on the values ​​of the RRT of all messages transmitted within a unit time, etc., which are not limited in the embodiments of the present application.

[0079] "The difference between the predicted value and the expected value corresponding to the RTT of all messages transmitted within time t" can be reflected by any one or more of the following first indicators: the difference between the predicted value and the expected value corresponding to the RTT of all messages transmitted within time t, the ratio between the aforementioned predicted value and the expected value, or other indicators, etc., which are not exhaustive here.

[0080] "The difference between the predicted number and the expected number of messages successfully sent at time t" may include the parameter value of each indicator in at least one second indicator, and the aforementioned at least one second indicator may include any one or more of the following: the difference between the predicted number and the expected number of messages successfully sent at time t, the ratio between the aforementioned predicted number and the expected number, or other indicators that can reflect the "difference between the aforementioned predicted number and the expected number", etc., which are not enumerated here.

[0081] Optionally, the higher the score value corresponding to each first indicator (or each second indicator), the better the performance of using the first control information in the evaluation dimension of the indicator; the higher the score value corresponding to each first indicator (or each second indicator), the worse the performance of using the first control information in the evaluation dimension of the indicator.

[0082] Exemplarily, a higher score value corresponding to the indicator "the difference between the predicted number of packets to be transmitted by the congested queue at time t+1 and the expected number" indicates that the difference between the predicted number of packets to be transmitted by the congested queue at time t+1 and the expected number is smaller, indicating that the performance of using the first control information in the evaluation dimension of "the difference between the predicted number of packets to be transmitted by the congested queue at time t+1 and the expected number" is better.

[0083] The higher the score corresponding to the indicator "the ratio between the predicted number of messages successfully sent within time t and the expected number", the larger the parameter value of the indicator "the ratio between the predicted number of messages successfully sent within time t and the expected number", which means that the performance of using the first control information in the evaluation dimension of "the ratio between the predicted number of messages successfully sent within time t and the expected number" is better, and so on. The examples here are only for the convenience of understanding the meaning represented by the score corresponding to a certain indicator. In the embodiments of the present application, each indicator in all the first indicators and the second indicators is no longer combined for examples one by one, nor is this scheme limited.

[0084] Optionally, a higher first comprehensive score value indicates that it is better to adopt the first control information in the network indicated by the first state information; a lower first comprehensive score value indicates that it is worse to adopt the first control information in the network indicated by the first state information.

[0085] It should be noted that the meanings of “each first indicator”, “each second indicator”, “the scoring value corresponding to each first indicator” and “the scoring value corresponding to each second indicator” can be found above and will not be repeated here.

[0086] In an embodiment of the present application, the first prediction information of the first control information indicates the difference between the predicted state information and the expected state information of the network at time t+1 when the first control information is adopted in the network state indicated by the first state information. Since the expected state information indicates the expected congestion situation of the network, the advantages and disadvantages of adopting the first control information are measured with the help of the expected state information, which is conducive to narrowing the similarity between the actual congestion situation of the network and the expected congestion situation, thereby facilitating users to more conveniently regulate the congestion situation of the network.

[0087] In another case, the first prediction information of the first control information may only include the predicted number of messages successfully sent within time t.

[0088] In another case, the first prediction information of the first control information may include a second score; for example, the second score may include a score value corresponding to a parameter value of each third indicator in at least one third indicator, and / or a score value corresponding to a predicted number of messages successfully sent at time t. At least one third indicator may indicate the predicted congestion of the network at the starting time point of time t+1. For example, at least one indicator may indicate any one or more of the following information: the predicted usage of the congestion queue at the starting time point of time t+1, the predicted value corresponding to the RTT of all messages transmitted at time t, or other information that can reflect the congestion of the network at the starting time point of time t+1; it should be understood that which indicators can reflect the "usage of the congestion queue at the starting time point of time t+1" and the "predicted value corresponding to the RTT of all messages transmitted at time t" can be understood in combination with the above description, and will not be repeated here.

[0089] Optionally, the higher the score value corresponding to each third indicator (or "the predicted number of messages successfully sent within time t"), the better the performance of using the first control information in this evaluation dimension of the indicator; the higher the score value corresponding to each third indicator (or "the predicted number of messages successfully sent within time t"), the worse the performance of using the first control information in this evaluation dimension of the indicator.

[0090] The higher the score value corresponding to the predicted number of messages successfully sent at time t, the more messages are successfully sent at time t; the lower the score value corresponding to the predicted number of messages successfully sent at time t, the fewer messages are successfully sent at time t.

[0091] Alternatively, the second score may be a second comprehensive score obtained by weighted summing the score corresponding to each third indicator and / or the score corresponding to the predicted number of messages successfully sent within time t. Optionally, a higher second comprehensive score indicates that the first control information is better used in the network indicated by the first state information; a lower second comprehensive score indicates that the first control information is worse used in the network indicated by the first state information.

[0092] It should be noted that the meaning of “the first prediction information of other control information in the control information set” is similar to the meaning of “the first prediction information of the first control information” and can be understood by reference and will not be repeated in the embodiments of the present application.

[0093] Optionally, the first device may use a machine learning model to perform step 202. Exemplarily, in one implementation, a first mapping relationship between the combination information and the predicted performance information of the aforementioned combination information is modeled using a machine learning model. Any combination information (hereinafter referred to as "target combination information" for the convenience of description) may include any state information (hereinafter referred to as "third state information" for the convenience of description) and any control information in the control information set (hereinafter referred to as "third control information" for the convenience of description). The aforementioned third state information is used to indicate the congestion of the network; the predicted performance information of the target combination information indicates the pros and cons of using the third control information under the network state indicated by the third state information.

[0094] It should be noted that the concepts of "second state information" and "second control information" will be used in subsequent descriptions and will not be introduced here; the meaning of "target state information" is similar to the above-mentioned "first state information", and the meaning of "predicted performance information" is similar to the above-mentioned "first prediction information". You can refer to the above description for understanding and will not repeat them here.

[0095] Then step 202 may include: the first device inputs the first state information and the first control information into the machine learning model, and obtains the first prediction information of the first control information output by the machine learning model. In the embodiment of the present application, the input of the machine learning model includes the first state information and the first control information, and the output of the machine learning model is directly the first prediction information of the first control information, which is conducive to quickly obtaining the first prediction information of the first control information after obtaining the first state information and the first control information, thereby facilitating improving the speed of finally obtaining the target control information.

[0096] In another implementation, a first distribution space is modeled using a machine learning model, and the first distribution space represents a distribution space where at least one first information pair is located, and each first information pair may include a combination information and predicted state information corresponding to the combination information. It should be noted that the meaning of "combination information" can be found in the above description and will not be repeated here.

[0097] Then step 202 may include: the first device inputs the first state information and the first control information into the machine learning model to obtain the third prediction information output by the machine learning model, and the third prediction information indicates the congestion condition of the network at time t+1 when the first control information is adopted in the network state indicated by the first state information. The first device can determine the pros and cons of adopting the first control information in the network state indicated by the first state information (that is, the first prediction information) based on the congestion condition of the network at time t+1 when the first control information is adopted in the network state indicated by the first state information (that is, the third prediction information). It should be noted that the concept of "second prediction information" will be used in subsequent descriptions and will not be introduced here.

[0098] Optionally, if the first prediction information of the first state information indicates the difference between the predicted state information and the expected state information of the network at time t+1 when the first control information is adopted in the network state indicated by the first state information, the first device determines the first prediction information based on the third prediction information, which may include: the first device determines the first prediction information based on the third prediction information and the expected state information.

[0099] In an embodiment of the present application, another scheme for obtaining first prediction information of first control information is provided, which improves the implementation flexibility of the scheme, and firstly predicts the congestion situation of the network at time t+1 based on the first state information and the first control information using a machine learning model, and then determines the pros and cons of using the first control information under the network state indicated by the first state information based on the congestion situation of the network at time t+1, so that the generation process of the first prediction information of the first control information is more refined, which is conducive to obtaining more accurate first prediction information of the first control information, and then is conducive to obtaining target control information that is more adapted to the network at time t, and is conducive to more effective use of the network bandwidth for data transmission within time t.

[0100] The third machine learning model is used to model the first distribution space. The first distribution space represents the distribution space of information pairs consisting of (state information, control information) and the state information at the next moment. The first distribution space is used to determine the first prediction information of each control information. That is, a specific implementation scheme for determining the first prediction information of each control information is provided, which helps to reduce the difficulty of implementing this scheme.

[0101] The first device may repeatedly perform step 202 for multiple times to obtain first prediction information of each control information in the control information set, that is, to determine the pros and cons of using each control information under the network state indicated by the first state information.

[0102] 203. Determine target control information from a control information set according to first prediction information of each control information.

[0103] In an embodiment of the present application, after the first device obtains the first prediction information of each control information in the control information set, it can select at least one target control information from the multiple control information included in the control information set according to the pros and cons of adopting each control information under the network state indicated by the first state information; the meanings of "target control information" and "first control information" are similar, and can be understood by referring to the above description, and will not be repeated here.

[0104] Exemplarily, in one implementation, the target control information is optimal control information under the network state indicated by the first state information, determined based on the first prediction information of each control information.

[0105] Optionally, step 203 may include: the first device determines the target control information from the control information set based on the first prediction information of each control information and the uncertainty corresponding to each control information, and the uncertainty corresponding to each control information represents the uncertainty of the combination of each control information and the first state information.

[0106] Exemplarily, the uncertainty corresponding to each control information can be specifically expressed as an uncertainty score for each control information; when the first prediction information of the first control information is expressed as a first comprehensive score value or a second comprehensive score value, the first device can perform a weighted summation of the first comprehensive score value (or the second comprehensive score value) of each control information and the uncertainty score of each control information to obtain a total score for each control information; and then determine the target control information from the control information set based on the total score of each control information.

[0107] Optionally, the target control information is control information with the highest total score among multiple control information included in the control information set.

[0108] Optionally, the uncertainty corresponding to each control information can be determined based on a contextual multi-armed bandit (CMAB) algorithm, wherein the higher the uncertainty score of each control information, the higher the uncertainty of the combination of each control information and the first state information; the lower the uncertainty score of each control information, the lower the uncertainty of the combination of each control information and the first state information.

[0109] Optionally, if a machine learning model is used during the execution of step 202, the uncertainty corresponding to each control information may also represent the uncertainty of a combination of each control information, the first state information, and the machine learning model used in step 202.

[0110] To further understand the present solution, an example of a formula used by the first device when executing step 203 is disclosed as follows:

[0111] a t = argmax a θ t [s t ;a]+cf(s t ,a;θ t ); (1)

[0112] Among them, a t represents the target control information, s t represents the first state information, a represents the control information set; θ t [s t ; a] represents obtaining the first comprehensive score value (or second comprehensive score value) corresponding to each control information in the control information set; θ t represents the machine learning model for processing the first state information in step 202; cf(s t ,a;θ t ) represents the acquisition of s t ,θ t The uncertainty score of the combination with each control information in the control information set; θ t [s t ;a]+cf(s t ,a;θ t ) represents adding the first comprehensive score value (or the second comprehensive score value) corresponding to each control information in the control information set to the uncertainty score to obtain the total score of each control information; argmax a θ t [s t ;a]+cf(s t ,a;θ t ) represents obtaining the target control information with the highest total score in the control information set. It should be understood that the example in formula (1) is only for the convenience of understanding this solution and is not used to limit this solution.

[0113] In the embodiment of the present application, not only the advantages and disadvantages of each control information are considered, but also the uncertainty of the combination of each control information and the first state information is considered, which is conducive to actively exploring combinations with high uncertainty, and further conducive to fully exploring the advantages and disadvantages of each control information, so as to increase the probability that the target control information finally obtained is the optimal control information.

[0114] Optionally, the first device may also send the aforementioned target control information to each sending end connected to the first device for communication, and the target control information is used by each sending end to determine a message sending rate.

[0115] In this implementation, each control information in the control information set indicates a data transmission rate. In the process of determining the target control information finally selected, the first prediction information of each control information is first obtained. Since the first prediction information of the first control information can indicate the pros and cons of using the first control information under the network state indicated by the first state information, the pros and cons of using each control information under the network state indicated by the first state information is obtained, and then a target control information is selected from the control information set based on the pros and cons of each control information. Through the above scheme, a more refined congestion control method is provided. Since in this scheme, each time the target control information to be used is determined, the pros and cons of each control information are evaluated in combination with the congestion situation of the network, which is conducive to improving the adaptability between the target control information and the congestion situation of the network, and is conducive to making the finally determined data transmission rate more consistent with the control information of the actual congestion situation of the network, that is, it is conducive to exploring a better data transmission rate.

[0116] In combination with the above description, it can be known that the process of acquiring the first prediction information of each control information in the control information set can be assisted by a machine learning model. Optionally, in one case, the first device can not only execute the steps of the reasoning phase of the aforementioned machine learning model, but also execute the steps of the training phase of the aforementioned machine learning model, that is, update the parameters of the aforementioned machine learning model. In another case, the first device only executes the steps of the reasoning phase of the aforementioned machine learning model, and other devices (hereinafter referred to as the second device for the convenience of description) execute the steps of the training phase of the aforementioned machine learning model, that is, the second device updates the parameters of the aforementioned machine learning model. The specific implementation processes of the aforementioned two cases are different. The detailed implementation processes of the aforementioned two cases are introduced below.

[0117] 1. The first device performs the steps of the inference phase of the machine learning model and updates the parameters of the machine learning model

[0118] In the embodiments of the present application, refer to the above Figure 2 It can be seen from the description in the corresponding embodiment that there may be two situations for the objects modeled by the machine learning model. The above two situations are described below respectively.

[0119] (1) Using machine learning models to model the mapping relationship between combined information and the predicted performance information of the aforementioned combined information

[0120] For details, please refer to Figure 3 , Figure 3 Another flow chart of the data processing method provided in the embodiment of the present application is as follows: Figure 3 As shown, the data processing method provided by the present application may include:

[0121] 301. The first device obtains the second state information at time t-1, the second control information used at time t-1, and the actual performance information of the second control information. The second state information at time t-1 indicates the congestion situation of the network at time t-1, and the actual performance information of the second control information indicates the advantages and disadvantages of using the second control information in the network at time t-1.

[0122] In the embodiment of the present application, step 301 is an optional step. Before using the machine learning model (hereinafter referred to as the "first machine learning model" for the convenience of distinction) to determine the second control information used at time t, the second state information at time t-1, the second control information used at time t-1, and the actual performance information of the second control information can also be obtained.

[0123] The meaning of "second state information" is similar to that of "first state information", except that t-1 in the above description of the first state information is replaced by t-2; the meaning of "second control information" is similar to that of "first control information", and the meanings of "second state information" and "second control information" can refer to the above description of the first state information. Figure 2 The corresponding embodiments can be understood and will not be described in detail here.

[0124] The information type included in the "actual performance information of the second control information" is similar to the information type included in the "first prediction information of the first control information". The difference is that the "actual performance information of the second control information" and the "first prediction information of the first control information" are generated in different ways. The "actual performance information of the second control information" is determined based on the first status information and / or the number of messages successfully sent at time t-1.

[0125] In one case, the actual performance information of the second control information may indicate the difference between the first state information and the expected state information used to reflect the congestion situation of the network at time t when the second control information is used under the network state indicated by the second state information (i.e., the network state at time t-1); and / or, the actual performance information of the second control information may include the difference between the actual number and the expected number of messages successfully sent at time t-1.

[0126] The meaning of "expected state information" can be found in the above Figure 2 The description in the corresponding embodiment is not repeated here.

[0127] In one implementation, the "actual performance information of the second control information" may include a score value corresponding to a parameter value of each fourth indicator in at least one fourth indicator, and / or a score value corresponding to a parameter value of each fifth indicator in at least one fifth indicator.

[0128] The parameter value of at least one of the fourth indicators mentioned above may indicate any one or more of the following information: the difference between the actual usage and the expected usage of the congestion queue at the starting time point of time t, the difference between the actual value and the expected value corresponding to the RTT of all packets transmitted within time t-1, or other information, etc., which is not limited here.

[0129] Which indicators can be combined with the above? Figure 2 The description of "at least one first indicator" in the corresponding embodiment is understood as follows: Figure 2 The time t+1 in the description of the corresponding embodiment is replaced by the time t, and Figure 2 The predicted number of messages to be transmitted at time t+1 in the corresponding embodiment is replaced by the actual number of messages to be transmitted at time t. Figure 2 The time t in the corresponding embodiment is replaced by the time t-1, and the specific indicators of the at least one fourth indicator will no longer be displayed and described here.

[0130] "The difference between the actual number and the expected number of messages successfully sent within time t-1" may include the parameter value of each indicator in at least one fifth indicator, and the aforementioned at least one fifth indicator may include any one or more of the following: the difference between the actual number and the expected number of messages successfully sent within time t-1, the ratio between the aforementioned actual number and the expected number, or other indicators that can reflect the "difference between the aforementioned actual number and the expected number", etc., which are not enumerated here.

[0131] Then in step 301, after obtaining the first status information indicating the congestion situation of the network at time t, the first device can determine the parameter value of each fourth indicator based on the first status information and the expected status information, and then obtain the score value corresponding to the parameter value of each fourth indicator; and / or, after determining the parameter value of each fifth indicator based on the actual number and expected number of messages successfully sent at time t-1, determine the score value corresponding to the parameter value of each fifth indicator.

[0132] Among them, the "score value corresponding to the parameter value of the fourth indicator" may also be referred to as the "score value corresponding to the fourth indicator", and the "score value corresponding to the parameter value of the fifth indicator" may also be referred to as the "score value corresponding to the fifth indicator". It should be noted that the meaning of the "score value corresponding to the fourth indicator" can be understood by referring to the description of the "score value corresponding to the first indicator", and the meaning of the "score value corresponding to the fifth indicator" can be understood by referring to the description of the "score value corresponding to the second indicator", which will not be repeated here.

[0133] In another implementation, the actual performance information of the second control information may include a third comprehensive score value obtained by weighted summing the score values ​​corresponding to each fourth indicator and / or the score values ​​corresponding to each fifth indicator. Then, in step 301, after determining the score value corresponding to each fourth indicator and / or the score value corresponding to each fifth indicator, the first device may also perform weighted summing of all the aforementioned score values ​​to obtain the third comprehensive score value.

[0134] In another case, the actual performance information of the second control information may include the actual number of messages successfully sent within time t-1.

[0135] In another case, the actual performance information of the second control information may include a third score; for example, the third score may include a score value corresponding to the parameter value of each sixth indicator in at least one sixth indicator, and / or a score value corresponding to the actual number of messages successfully sent at time t-1.

[0136] For example, at least one sixth indicator may indicate the actual congestion situation of the network at the starting time point of time t. For example, at least one sixth indicator may indicate any one or more of the following information: the actual usage of the congestion queue at the starting time point of time t, the actual value corresponding to the RTT of all packets transmitted within time t-1, or other information that can reflect the degree of congestion of the network, etc.; these are not exhaustive here.

[0137] The "actual usage of the congestion queue at the starting time point of time t" can be reflected by any one or more of the following sixth indicators: the actual number of data transmission units to be transmitted in the congestion queue, the ratio between the actual number and the total length of the congestion queue, or other indicators.

[0138] The "actual value corresponding to the RTT of all messages transmitted within time t-1" can be reflected by any one or more of the following sixth indicators: the average value, median value, maximum value or other indicators corresponding to the RTT of all messages transmitted within time t-1.

[0139] Optionally, the higher the score value corresponding to each sixth indicator (or "the actual number of messages successfully sent within time t-1"), the better the performance of using the second control information in this evaluation dimension of the indicator; the higher the score value corresponding to each sixth indicator (or "the actual number of messages successfully sent within time t-1"), the worse the performance of using the second control information in this evaluation dimension of the indicator.

[0140] Then in step 301, the first device can determine the score value corresponding to the parameter value of each sixth indicator and / or the score value corresponding to the actual number of messages successfully sent within time t-1 based on the first status information and / or the actual number of messages successfully sent within time t-1, thereby obtaining the actual performance information of the second control information.

[0141] Alternatively, the third score may be a fourth comprehensive score obtained by weighted summing the score corresponding to each sixth indicator and / or the score corresponding to the actual number of messages successfully sent within time t-1. Then, in step 301, after determining the score corresponding to the parameter value of each sixth indicator and / or the score corresponding to the actual number of messages successfully sent within time t-1, the first device may generate a fourth comprehensive score, that is, obtain the actual performance information of the second control information.

[0142] 302. The first device inputs the second state information and the second control information into N second machine learning models corresponding to the N second parameter values, and obtains N second prediction information of the second control information, where the second prediction information represents the pros and cons of using the second control information in the network indicated by the second state information, and N is an integer greater than or equal to 1.

[0143] In the embodiment of the present application, step 302 is an optional step. N second parameter values ​​are deployed in the first device, each second parameter value includes a parameter value adopted by a weight parameter of the machine learning model at one of the N moments before the moment t, and N is an integer greater than or equal to 1; when the weight parameter of the machine learning model adopts the second parameter value, it can be called a second machine learning model, and there are N second machine learning models corresponding to the N second parameter values.

[0144] The first device can input the second state information and the second control information into each of the N second machine learning models to obtain a second prediction information output by each second machine learning model, that is, to obtain N second prediction information output by the N second machine learning models, each of the N second prediction information represents the pros and cons of adopting the second control information in the network indicated by the second state information; wherein the specific expression form of the "second prediction information" is similar to the specific expression form of the "first prediction information", and can be understood by referring to the above description, which will not be repeated here.

[0145] 303. The first device updates the third parameter value to obtain the first parameter value according to the N second prediction information and the actual performance information. The third parameter value is one of the N second parameter values. When the weight parameter of the machine learning model adopts the first parameter value, it is the first machine learning model. The first machine learning model is used to process the first state information at time t.

[0146] In the embodiment of the present application, step 303 is an optional step. Optionally, N is an integer greater than or equal to 3, and illustratively, the value of N can be 3, 4, 5 or other values.

[0147] Exemplarily, the third parameter value may be a set of parameter values ​​adopted by the machine learning model when determining the second control information adopted at time t-1; or, the third parameter value may also be any one of the N second parameter values. The selection of the specific "third parameter value" may be flexibly determined in combination with the actual application scenario, and is not limited in the embodiments of the present application.

[0148] Exemplarily, the first parameter value is obtained by updating the third parameter value based on the second state information at time t-1, the second control information used at time t-1, and the actual performance information of the second control information.

[0149] Exemplarily, step 303 may include: the first device may calculate the function value of the first loss function based on N second prediction information corresponding to the second control information and the actual performance information of the second control information, and based on the function value of the first loss function, use the back propagation algorithm to perform gradient update on the third parameter value to obtain the first parameter value.

[0150] For example, when each second prediction information is expressed as a comprehensive score value, the first device can obtain the distance between each second prediction information in the N second prediction information and the actual performance information, that is, obtain N distances corresponding to the N second prediction information one by one; obtain the weight value corresponding to each second prediction information in the N second prediction information, and perform weighted summation of the N distances according to the weight value of each second prediction information to obtain the sum result, that is, obtain the function value of the first loss function; wherein the first loss function indicates the similarity between the N second prediction information and the actual performance information, and the goal of training using the first loss function includes improving the similarity between the N second prediction information and the actual performance information.

[0151] To further understand the present solution, an example of a formula used in the process of determining the first parameter value is disclosed as follows:

[0152]

[0153] Among them, L1 represents the first loss function, θ N Including N second machine learning models corresponding to the N moments before moment t; (θ N [s t ; a t ]-y t ) 2represents calculating the square of the difference between each second prediction information in the N second prediction information and the actual performance information (that is, an example of the distance between each second prediction information and the actual performance information); represents N weights corresponding to the N second prediction information one by one, represents the weighted summation of the N distances corresponding to the N second prediction information, that is, the function value of the first loss function is obtained; argmin θ The purpose of updating the weight parameters of the machine learning model includes reducing the value of the first loss function. It should be understood that the example in formula (2) is only for the convenience of understanding this solution and is not used to limit this solution.

[0154] Exemplarily, in one case, the weight values ​​of the N second prediction information decrease successively, that is, the weight value of the second prediction information corresponding to time t-1 is the largest, and the weight value of the second prediction information corresponding to time tN is the smallest.

[0155] In another case, the weight value corresponding to each of the N second prediction information is obtained based on the first state information at time t, the N state information at N moments before time t, and a regression algorithm, and the weight value corresponding to each of the second prediction information is consistent with the weight value corresponding to each of the N state information. Exemplarily, multiple groups of weight values ​​are deployed in the first device, each group of weight values ​​includes N weight values ​​corresponding one-to-one to the N second prediction information, and when the information obtained by weighted summing the N state information using a certain group of weight values ​​(hereinafter referred to as the "target group" for convenience of description) is most similar to the first state information, the target group weight value is determined as the weight of the N second prediction information.

[0156] It should be noted that the example of "the weight value of each second prediction information" here is only to prove the feasibility of this solution. The specific implementation method can be flexibly determined in combination with the actual application scenario and is not limited here.

[0157] For a more intuitive understanding of this solution, please refer to Figure 4 , Figure 4 A schematic diagram of determining a first parameter value provided in an embodiment of the present application. Figure 4 As shown, Figure 4Taking the value of N as 3 as an example, the three second machine learning models include the second machine learning model 1 corresponding to time t-1, the second machine learning model 2 corresponding to time t-2, and the second machine learning model 3 corresponding to time t-3. The first device inputs the second state information and the second control information into the second machine learning model 1, the second machine learning model 2, and the second machine learning model 3 to obtain the second prediction information 1 output by the second machine learning model 1, the second prediction information 2 output by the second machine learning model 2, and the second prediction information 3 output by the second machine learning model 3.

[0158] According to the second prediction information 1, the second prediction information 2, the second prediction information 3 and the actual performance information, the function value of the first loss function is generated; based on the function value of the first loss function, the value of the weight parameter in the second machine learning model 1 (that is, the third parameter value) is updated to obtain the first parameter value, that is, the first machine learning model adopted at time t is obtained. It should be understood that Figure 4 The examples are only for facilitating the understanding of this solution and are not intended to limit this solution.

[0159] In an embodiment of the present application, the advantages and disadvantages of adopting the second control information are predicted respectively based on the N second machine learning models before time t (that is, N second prediction information), and then the third parameter value is updated based on the actual performance information of the N second prediction information and the second control information, so as to obtain the first machine learning model adopted at time t, that is, the process of determining the value of the weight parameter of the first machine learning model combines the second machine learning model adopted at N moments before time t, and the second machine learning model adopted at N moments before time t has seen N kinds of congestion conditions of the network, which is conducive to obtaining a first machine learning model with better performance, and then conducive to obtaining better target control information, so as to improve the utilization rate of network communication resources while minimizing network congestion.

[0160] In the embodiment of the present application, steps 302 and 303 are both optional steps. If steps 302 and 303 are not performed, the first device may also input the second state information and the second control information into the target machine learning model to obtain a second prediction information output by the target machine learning model; wherein the target machine learning model is one of the N second machine learning models; among the N state information processed by the N second machine learning models, the state information processed by the target machine learning model has the greatest similarity to the first state information, and the third parameter value is a set of parameter values ​​used by the weight parameter of the target machine learning model.

[0161] The first device can generate a function value of a second loss function based on a second prediction information and actual performance information output by the target machine learning model, and use a back-propagation algorithm to update the value of a weight parameter of the target machine learning model (that is, a third parameter value) to obtain a first parameter value, that is, a first machine learning model.

[0162] In an embodiment of the present application, the first machine learning model adopted at time t is determined based on the historical data at time t-1 and the second machine learning model adopted at N times before time t. That is, the first parameter value adopted by the first machine learning model is timely updated using the latest historical data, which is conducive to improving the performance of the first machine learning model and thus helping to obtain better target control information.

[0163] 304. The first device obtains first status information, where the first status information indicates a congestion situation of the network.

[0164] 305. The first device obtains first prediction information of first control information, wherein the first control information is any one in a control information set, the control information set includes multiple control information, each of the multiple control information indicates a data sending rate, and the first prediction information of the first control information indicates the advantages and disadvantages of using the first control information under the network state indicated by the first state information.

[0165] In the embodiment of the present application, after the first device determines the first parameter value used when processing the first state information at time t through steps 301 to 303, it also determines the first machine learning model used when processing the first state information at time t; in step 305, the first state information and the first control information can be input into the first machine learning model to obtain the first prediction information of the first control information output by the first machine learning model; the detailed meaning of "the first prediction information of the first control information" can be referred to above Figure 2 The description in the corresponding embodiment is not repeated here.

[0166] The first device repeatedly executes step 305 multiple times to obtain first prediction information of each control information in the control information set.

[0167] 306. The first device selects second control information from the control information set according to the first prediction information of each control information.

[0168] In the embodiment of the present application, the specific implementation of steps 304 and 306 can be found in Figure 2 The description in the corresponding embodiment is not repeated here.

[0169] (2) Modeling a first distribution space using a machine learning model, where the first distribution space represents a distribution space where at least one first information pair is located, and each first information pair may include a combination information and prediction state information corresponding to the combination information

[0170] For details, please refer to Figure 5 , Figure 5 Another flow chart of the data processing method provided in the embodiment of the present application is as follows: Figure 5 As shown, the data processing method provided by the present application may include:

[0171] 501. A first device obtains second state information at time t-1 and second control information used at time t-1, where the state information at time t-1 indicates a congestion situation of the network at time t-1.

[0172] In the embodiment of the present application, step 501 and subsequent step 502 are both optional steps. The specific implementation of step 501 and the meaning of the terms in step 501 can be found in Figure 3 The description in the corresponding embodiment is not repeated here.

[0173] 502. The first device updates the posterior distribution of the third machine learning model according to the state information at time t-1, the second control information and the first state information to obtain an updated third machine learning model.

[0174] For the sake of convenience in distinction, in the embodiment of the present application, the machine learning model for modeling the first distribution space is referred to as the third machine learning model. The third machine learning model may adopt a Gaussian model, or may adopt other models for describing the distribution space, etc., which is not limited in the embodiment of the present application.

[0175] Exemplarily, the state information at time t-1, the second control information, and the first state information constitute a first information pair; the first device can use the maximum a posteriori probability algorithm to update the posterior distribution of the third machine learning model based on the state information at time t-1, the second control information, and the first state information to obtain an updated third machine learning model.

[0176] 503. The first device obtains first status information, where the first status information indicates a congestion situation of the network.

[0177] 504. The first device obtains first prediction information of first control information, wherein the first control information is any one in a control information set, the control information set includes multiple control information, each of the multiple control information indicates a data sending rate, and the first prediction information of the first control information indicates the advantages and disadvantages of using the first control information under the network state indicated by the first state information.

[0178] In the embodiment of the present application, the first state information and the first control information constitute a first combination of information, and the second combination of information is a combination of information;

[0179] Since steps 501 and 502 are optional steps, if steps 501 and 502 are executed, the first device can obtain second combination information that is most similar to the first combination information from the first distribution space indicated by the updated third machine learning model; if steps 501 and 502 are not executed, the first device can obtain second combination information that is most similar to the first combination information from the first distribution space indicated by the third machine learning model.

[0180] The first device determines the predicted state information corresponding to the second combined information as the predicted state information corresponding to the first combined information, and determines the first predicted state information of the first control information based on the predicted state information corresponding to the first combined information (that is, the predicted state information used to indicate the congestion status of the network at the starting time point of time t+1).

[0181] The specific implementation of the above steps can be combined with Figure 2 For understanding the description in the corresponding embodiment, the relationship between the "prediction status information indicating the congestion status of the network at the starting time point of time t+1" and the "first prediction information of the first control information" can be referred to. Figure 2 The description in the corresponding embodiments will not be repeated here.

[0182] The first device repeatedly executes step 504 multiple times until first prediction information of each control information in the control information set is obtained.

[0183] 505. The first device determines target control information from the control information set according to the first prediction information of each control information.

[0184] In the embodiment of the present application, the specific implementation of steps 504 and 505 can be found in Figure 2 The description in the corresponding embodiment will not be repeated here. After obtaining the historical data at time t-1, the historical data at time t-1 will be used in a timely manner to update the posterior distribution of the third machine learning model, which is beneficial to improve the similarity between the first distribution space indicated by the third machine learning model and the real distribution space where at least one information pair is located, that is, it is beneficial to model the real distribution space where at least one information pair is located through the third machine learning model, which is beneficial to obtain more accurate prediction state information with the help of the updated third machine learning model, and then it is beneficial to obtain more accurate first prediction information, so as to select better target control information.

[0185] 2. The first device only performs the steps of the inference phase of the machine learning model, and the second device updates the parameters of the machine learning model

[0186] In the embodiments of the present application, refer to the above Figure 2 It can be seen from the description in the corresponding embodiment that there may be two situations for the objects modeled by the machine learning model. The above two situations are described below respectively.

[0187] (1) Using machine learning models to model the mapping relationship between combined information and the predicted performance information of the aforementioned combined information

[0188] For details, please refer to Figure 6 , Figure 6 Another flow chart of the data processing method provided in the embodiment of the present application is as follows: Figure 3 As shown, the data processing method provided by the present application may include:

[0189] 601. The second device obtains a first sample, where the first sample includes fourth state information, fourth control information, and actual performance information of the fourth control information, where the first state information indicates network congestion, and the actual performance information of the fourth control information indicates pros and cons of using the fourth control information under the network state indicated by the fourth state information.

[0190] 602. The second device inputs the fourth state information and the fourth control information into the first machine learning model to obtain predicted performance information of the fourth control information output by the first machine learning model.

[0191] In the embodiment of the present application, in the process of executing the steps of the inference phase using the first machine learning model, multiple samples for training can be obtained. In step 601, multiple samples for training can be deployed in the second device, and the second device can obtain one or more first samples; wherein, the meaning of "fourth state information" is similar to the meaning of "second state information", the meaning of "fourth control information" is similar to the meaning of "second control information", and the meaning of "actual performance information of the fourth control information" is similar to the meaning of "actual performance information of the second control information", and both can be referred to. Figure 3 The description of step 301 in the corresponding embodiment. The specific implementation of steps 601 and 602 can also be found in Figure 3 The descriptions in steps 301 and 302 in the corresponding embodiment are not repeated here.

[0192] 603. The second device updates the first machine learning model using a third loss function according to actual performance information of the fourth control information and predicted performance information of the fourth control information, where the third loss function indicates the similarity between the actual performance information of the fourth control information and the predicted performance information of the fourth control information.

[0193] In the embodiment of the present application, steps 601 to 603 are optional steps. Exemplarily, step 603 may include: the second device generates a function value of a third loss function according to the actual performance information of the fourth control information and the predicted performance information of the fourth control information; and according to the function value of the third loss function, a back propagation algorithm is used to perform a gradient update on the value of the weight parameter of the first machine learning model to implement a training of the first machine learning model.

[0194] The second device may repeat steps 601 to 603 multiple times until the convergence condition is met, thereby obtaining a first machine learning model that has performed a training operation, and the first machine learning model that has performed a training operation will be deployed to the first device; exemplarily, the convergence condition may include satisfying a convergence condition of a third loss function and / or the number of training times reaching a preset number of times.

[0195] 604. The first device obtains first status information, where the first status information indicates a congestion situation of the network.

[0196] 605. The first device obtains first prediction information of first control information, wherein the first control information is any one in a control information set, the control information set includes multiple control information, each of the multiple control information indicates a data sending rate, and the first prediction information of the first control information indicates the advantages and disadvantages of using the first control information under the network state indicated by the first state information.

[0197] 606. The first device selects second control information from the control information set according to the first prediction information of each control information.

[0198] In the embodiment of the present application, the specific implementation of steps 604 to 606 can be found in Figure 3 The description of steps 304 to 306 in the corresponding embodiment will not be repeated in the embodiment of the present application.

[0199] It should be noted that if steps 601 to 603 are executed, after executing steps 604 to 606 multiple times, steps 601 to 603 can be executed again to update the values ​​of the weight parameters of the first machine learning model again, to obtain the first machine learning model that has been trained again, and the first machine learning model that has been trained again can be deployed to the first device to execute steps 604 to 606 using the first machine learning model that has been trained again; the foregoing cycle can be repeated in actual application.

[0200] (2) Modeling a first distribution space using a machine learning model, where the first distribution space represents a distribution space where at least one first information pair is located, and each first information pair may include a combination information and prediction state information corresponding to the combination information

[0201] For details, please refer to Figure 7 , Figure 7 Another flow chart of the data processing method provided in the embodiment of the present application is as follows: Figure 7 As shown, the data processing method provided by the present application may include:

[0202] 701. The second device obtains at least one second information pair, where the second information pair includes third combination information and status information corresponding to the third combination information, the third combination information includes fifth status information and fifth control information, and the status information corresponding to the third combination information indicates a congestion condition of the network at a next moment when the fifth control information is used in the network state indicated by the fifth status information.

[0203] 702. The second device updates the posterior distribution of the third machine learning model based on at least one second information pair.

[0204] In the embodiment of the present application, steps 701 and 702 are optional steps. For example, in the process of executing the steps of the reasoning phase using the third machine learning model, one or more second information pairs can be obtained, the meaning of "second information pair" is similar to the meaning of "first information pair", "third combination information" is a combination information, and the meaning of "state information corresponding to the third combination information" can be understood by referring to the above description of "state information corresponding to the combination information", which will not be repeated here.

[0205] Exemplarily, after obtaining at least one second information pair, the second device can update the posterior distribution of the third machine learning model in sequence based on each second information pair, that is, using at least one second information pair, perform at least one update operation on the posterior distribution of the third machine learning model to obtain an updated third machine learning model.

[0206] 703. The first device obtains first status information, where the first status information indicates a congestion situation of the network.

[0207] 704. The first device obtains first prediction information of first control information, wherein the first control information is any one in a control information set, the control information set includes multiple control information, each of the multiple control information indicates a data sending rate, and the first prediction information of the first control information indicates the advantages and disadvantages of using the first control information under the network state indicated by the first state information.

[0208] 705. The first device selects second control information from the control information set according to the first prediction information of each control information.

[0209] In the embodiment of the present application, the specific implementation of steps 703 to 705 can be referred to Figure 3 The description of steps 303 to 305 in the corresponding embodiment will not be repeated in the embodiment of the present application.

[0210] It should be noted that steps 701 and 702 are optional steps. If steps 701 and 702 are executed, steps 701 and 702 can be executed again after executing steps 703 to 705 multiple times to update the values ​​of the weight parameters of the third machine learning model again, to obtain the third machine learning model that has been trained again, and the third machine learning model that has been trained again can be deployed to the third device to execute steps 703 to 705 using the third machine learning model that has been trained again; the foregoing cycle can be repeated in actual application.

[0211] exist Figures 1 to 7 On the basis of the corresponding embodiments, in order to better implement the above solutions of the embodiments of the present application, the following also provides related devices for implementing the above solutions. Figure 8 , Figure 8 A structural schematic diagram of a data processing device provided in an embodiment of the present application, the data processing device 800 is applied to a first device, and the data processing device 800 includes: an acquisition module 801, used to obtain first status information, the first status information indicates the congestion status of the network; the acquisition module 801 is also used to obtain first prediction information of first control information, wherein the first control information is any one of a control information set, the control information set includes multiple control information, each of the multiple control information indicates the data sending rate, and the first prediction information of the first control information indicates the pros and cons of using the first control information under the network status indicated by the first status information; a determination module 802 is used to determine the target control information from the control information set according to the first prediction information of each control information.

[0212] Optionally, the acquisition module 801 is specifically used to input the first state information and the first control information into the first machine learning model to obtain the first prediction information output by the first machine learning model.

[0213] Optionally, the first state information indicates the congestion condition of the network at time t, and when the weight parameter of the machine learning model adopts the first parameter value, it is the first machine learning model, and t is an integer greater than or equal to 11; the data processing device 800 is applied to the first device, and N second parameter values ​​are deployed in the first device, each second parameter value includes the parameter value adopted by the weight parameter of the machine learning model at one of the N moments before time t, N is an integer greater than or equal to 11, and the first parameter value is obtained by updating the third parameter value based on the second state information at time t-11, the second control information adopted at time t-11, and the actual performance information of the second control information; wherein the third parameter value is one of the N second parameter values, the state information at time t-11 indicates the congestion condition of the network at time t-11, the second control information indicates the data sending rate in the network at time t-11, and the actual performance information of the second control information represents the pros and cons of adopting the second control information in the network at time t-11.

[0214] Optionally, when the weight parameter of the machine learning model adopts the second parameter value, it is a second machine learning model, and the data processing device also includes: an input module 803, which is used to input the second state information and the second control information into N second machine learning models corresponding to the N second parameter values ​​one by one, to obtain N second prediction information of the second control information, the second prediction information representing the pros and cons of adopting the second control information in the network indicated by the second state information; an update module 804, which is used to update the third parameter value according to the N second prediction information and actual performance information to obtain the first parameter value.

[0215] Optionally, the first state information indicates the congestion status of the network at time t, where t is an integer greater than or equal to 11, and the acquisition module 801 is specifically used to: input the first state information and the first control information into a third machine learning model to obtain third prediction information output by the third machine learning model, the third prediction information indicating the congestion status of the network at time t+11 when the first control information is used in the network state indicated by the first state information; and determine the first prediction information based on the third prediction information.

[0216] Optionally, the first distribution space is modeled by a third machine learning model, and the first distribution space represents the distribution space where at least one information pair is located. Each information pair includes a combination information and predicted state information corresponding to the combination information. A combination information includes a third state information and a third control information. The third state information indicates the congestion condition of the network. The third control information is any one in the control information set. The state information corresponding to the combination information indicates the congestion condition of the network at the next moment when the third control information is adopted in the network state indicated by the third state information.

[0217] Optionally, the acquisition module 801 is also used to acquire the second state information at time t-11 and the second control information used at time t-11, the state information at time t-11 indicating the congestion status of the network at time t-11; the data processing device 800 also includes: an update module 804, used to update the posterior distribution of the third machine learning model according to the state information at time t-11, the second control information and the first state information, to obtain an updated third machine learning model; the acquisition module 801 is specifically used to acquire third prediction information from the distribution space indicated by the updated third machine learning model according to the first state information and the first control information.

[0218] Optionally, the first state information indicates the congestion situation of the network at time t, and the first prediction information of the first state information indicates the difference between the predicted state information and the expected state information of the network at time t+11 when the first control information is adopted in the network state indicated by the first state information; wherein, the predicted state information of the network at time t+11 represents the predicted congestion situation of the network at time t+11, and the expected state information represents the expected congestion situation of the network.

[0219] Optionally, the determination module 802 is specifically used to determine the target control information from the control information set based on the first prediction information of each control information and the uncertainty corresponding to each control information, and the uncertainty corresponding to each control information represents the uncertainty of the combination of each control information and the first state information.

[0220] Optionally, each control information includes a value of a receiving window RWND and / or a value of a sending window SWND.

[0221] It should be noted that the information interaction, execution process, etc. between the modules / units in the data processing device 800 are the same as those in the present application. Figures 2 to 7 The corresponding method embodiments are based on the same concept. For specific contents, please refer to the description in the method embodiments shown above in this application, which will not be repeated here.

[0222] See also Fig. 9 , Fig. 9 A schematic diagram of a structure of a device provided in an embodiment of the present application. When the first device or the second device involved in the above embodiment is specifically a network device, it can be used Fig. 9 Implementation of the device 900 shown in FIG. 1; illustratively, when the first device (or the second device) is a router, a firewall, a gateway or other transit device, it can be used Fig. 9 Alternatively, when the second device is a server or a server cluster, the Fig. 9 The device 900 shown in FIG.

[0223] The structure of the communication device can refer to Fig. 9 The structure shown. The device 900 includes at least one processor 911 and at least one network interface 912. Further optionally, the communication device also includes at least one memory 914, at least one transceiver 913 and one or more antennas 915. The processor 911, the transceiver 913, the memory 914 and the network interface 912 are connected, for example, through a bus. In an embodiment of the present application, the connection may include various interfaces, transmission lines or buses, etc., which are not limited in this embodiment. The antenna 915 is connected to the transceiver 913. The network interface 912 is used to enable the communication device to communicate with other communication devices through a communication link. For example, the network interface 912 may include a network interface between the communication device and the core network device, such as an S1 interface, and the network interface may include a network interface between the communication device and other communication devices (such as other network devices or core network devices), such as an X2 or Xn interface.

[0224] The processor 911 is mainly used to process the communication protocol and communication data, and to control the entire communication device, execute the software program, and process the data of the software program, for example, to support the communication device to perform the actions described in the embodiment. The communication device may include a baseband processor and a central processing unit. The baseband processor is mainly used to process the communication protocol and communication data, and the central processing unit is mainly used to control the entire terminal device, execute the software program, and process the data of the software program. Fig. 9 The processor 911 in the embodiment can integrate the functions of the baseband processor and the central processor. It can be understood by those skilled in the art that the baseband processor and the central processor can also be independent processors, which are interconnected through technologies such as buses. It can be understood by those skilled in the art that the terminal device can include multiple baseband processors to adapt to different network formats, and the terminal device can include multiple central processors to enhance its processing capabilities. The various components of the terminal device can be connected through various buses. The baseband processor can also be described as a baseband processing circuit or a baseband processing chip. The central processor can also be described as a central processing circuit or a central processing chip. The function of processing the communication protocol and the communication data can be built into the processor, or it can be stored in the memory in the form of a software program, and the processor executes the software program to realize the baseband processing function.

[0225] The memory is mainly used to store software programs and data. The memory 914 can exist independently and be connected to the processor 911. Optionally, the memory 914 can be integrated with the processor 911, for example, integrated into a chip. Among them, the memory 914 can store program codes for executing the technical solutions of the embodiments of the present application, and the execution is controlled by the processor 911. The various types of computer program codes executed can also be regarded as drivers of the processor 911.

[0226] Fig. 9 Only one memory and one processor are shown. In an actual network device, there may be multiple processors and multiple memories. The memory may also be referred to as a storage medium or a storage device, etc. The memory may be a storage element on the same chip as the processor, i.e., an on-chip storage element, or an independent storage element, which is not limited in the embodiments of the present application.

[0227] The transceiver 913 can be used to support the reception or transmission of radio frequency signals between the communication device and the terminal, and the transceiver 913 can be connected to the antenna 915. The transceiver 913 includes a transmitter Tx and a receiver Rx. Specifically, one or more antennas 915 can receive radio frequency signals, and the receiver Rx of the transceiver 913 is used to receive the radio frequency signal from the antenna, convert the radio frequency signal into a digital baseband signal or a digital intermediate frequency signal, and provide the digital baseband signal or the digital intermediate frequency signal to the processor 911, so that the processor 911 further processes the digital baseband signal or the digital intermediate frequency signal, such as demodulation processing and decoding processing. In addition, the transmitter Tx in the transceiver 913 is also used to receive a modulated digital baseband signal or a digital intermediate frequency signal from the processor 911, and convert the modulated digital baseband signal or the digital intermediate frequency signal into a radio frequency signal, and send the radio frequency signal through one or more antennas 915. Specifically, the receiver Rx can selectively perform one or more stages of down-mixing and analog-to-digital conversion processing on the RF signal to obtain a digital baseband signal or a digital intermediate frequency signal, and the order of the down-mixing and analog-to-digital conversion processing is adjustable. The transmitter Tx can selectively perform one or more stages of up-mixing and digital-to-analog conversion processing on the modulated digital baseband signal or digital intermediate frequency signal to obtain a RF signal, and the order of the up-mixing and digital-to-analog conversion processing is adjustable. The digital baseband signal and the digital intermediate frequency signal can be collectively referred to as a digital signal.

[0228] The transceiver 913 may also be referred to as a transceiver unit, a transceiver, a transceiver device, etc. Optionally, a device in the transceiver unit for implementing a receiving function may be regarded as a receiving unit, and a device in the transceiver unit for implementing a sending function may be regarded as a sending unit, that is, the transceiver unit includes a receiving unit and a sending unit, the receiving unit may also be referred to as a receiver, an input port, a receiving circuit, etc., and the sending unit may be referred to as a transmitter, a transmitter, or a transmitting circuit, etc.

[0229] For example, it is possible to use Fig. 9 The device 900 shown in the figure implements the above Figures 2 to 7 The steps implemented by the first device (or the second device) in the corresponding method embodiment, Fig. 9The specific implementation methods of the device 900 shown in the figure to perform the aforementioned steps can all be referred to the description in the aforementioned method embodiment, and will not be described in detail here.

[0230] The present application also provides a computer-readable storage medium in which a program for signal processing is stored. When the program is run on a computer, the computer executes the above-mentioned Figures 2 to 7 The steps performed by the first device in the method described in the embodiment shown, or causing the computer to perform the above Figures 2 to 7 The illustrated embodiment describes the steps performed by the second device in the method.

[0231] The present application also provides a computer program product which, when executed on a computer, enables the computer to execute the above Figures 2 to 7 The steps performed by the first device in the method described in the embodiment shown, or causing the computer to perform the above Figures 2 to 7 The illustrated embodiment describes the steps performed by the second device in the method.

[0232] The first device or the second device provided in the embodiment of the present application may be a chip, and the chip includes: a processing unit and a communication unit, the processing unit may be a processor, and the communication unit may be an input / output interface, a pin or a circuit, etc. The processing unit may execute the computer execution instructions stored in the storage unit to enable the chip to execute the above Figures 2 to 7 The data processing method described in the embodiment shown. Optionally, the storage unit is a storage unit in the chip, such as a register, a cache, etc., and the storage unit can also be a storage unit located outside the chip in the wireless access device, such as a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM), etc.

[0233] For details, please refer to Fig.10 , Fig.10 A schematic diagram of the structure of a chip provided in an embodiment of the present application, the chip can be expressed as a neural network processor NPU 100, NPU 100 is mounted on the host CPU (Host CPU) as a coprocessor, and the host CPU assigns tasks. The core part of the NPU is the operation circuit 1003, which is controlled by the controller 1004 to extract matrix data from the memory and perform multiplication operations.

[0234] In some implementations, the operation circuit 1003 includes multiple processing units (Process Engine, PE) inside. In some implementations, the operation circuit 1003 is a two-dimensional systolic array. The operation circuit 1003 can also be a one-dimensional systolic array or other electronic circuits capable of performing mathematical operations such as multiplication and addition. In some implementations, the operation circuit 1003 is a general-purpose matrix processor.

[0235] For example, assume there is an input matrix A, a weight matrix B, and an output matrix C. The operation circuit takes the corresponding data of matrix B from the weight memory 1002 and caches it on each PE in the operation circuit. The operation circuit takes the matrix A data from the input memory 1001 and performs matrix operation with matrix B, and the partial result or final result of the matrix is ​​stored in the accumulator 1008.

[0236] The unified memory 1006 is used to store input data and output data. The weight data is directly transferred to the weight memory 1002 through the direct memory access controller (DMAC) 1005. The input data is also transferred to the unified memory 1006 through the DMAC.

[0237] BIU stands for Bus Interface Unit, i.e., bus interface unit 1010 , which is used for interaction between AXI bus, DMAC and instruction fetch buffer (IFB) 1009 .

[0238] The bus interface unit 1010 (Bus Interface Unit, BIU for short) is used for the instruction fetch memory 1009 to obtain instructions from the external memory, and is also used for the storage unit access controller 1005 to obtain the original data of the input matrix A or the weight matrix B from the external memory.

[0239] DMAC is mainly used to transfer input data in the external memory DDR to the unified memory 1006 or to transfer weight data to the weight memory 1002 or to transfer input data to the input memory 1001.

[0240] The vector calculation unit 1007 includes multiple operation processing units, and further processes the output of the operation circuit when necessary, such as vector multiplication, vector addition, exponential operation, logarithmic operation, size comparison, etc. It is mainly used for non-convolutional / fully connected layer network calculations in neural networks, such as Batch Normalization, pixel-level summation, upsampling of feature planes, etc.

[0241] In some implementations, the vector calculation unit 1007 can store the processed output vector to the unified memory 1006. For example, the vector calculation unit 1007 can apply a linear function and / or a nonlinear function to the output of the operation circuit 1003, such as linear interpolation of the feature plane extracted by the convolution layer, and then, for example, a vector of accumulated values ​​to generate an activation value. In some implementations, the vector calculation unit 1007 generates a normalized value, a pixel-level summed value, or both. In some implementations, the processed output vector can be used as an activation input to the operation circuit 1003, for example, for use in a subsequent layer in a neural network.

[0242] An instruction fetch buffer 1009 connected to the controller 1004 is used to store instructions used by the controller 1004;

[0243] Unified memory 1006, input memory 1001, weight memory 1002 and instruction fetch memory 1009 are all on-chip memories. External memories are private to the NPU hardware architecture.

[0244] In which, the operations of each layer in the first machine learning model, the second machine learning model and / or the third machine learning model can be performed by the operation circuit 1003 or the vector calculation unit 1007.

[0245] The processor mentioned in any of the above places may be a general-purpose central processing unit, a microprocessor, an ASIC, or one or more integrated circuits for controlling the execution of the program of the above-mentioned first aspect method.

[0246] It should also be noted that the device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed over multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment. In addition, in the drawings of the device embodiments provided by the present application, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines.

[0247] Through the description of the above implementation mode, the technicians in the field can clearly understand that the present application can be implemented by means of software plus necessary general hardware, and of course, it can also be implemented by special hardware including special integrated circuits, special CPUs, special memories, special components, etc. In general, all functions completed by computer programs can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be various, such as analog circuits, digital circuits or special circuits. However, for the present application, software program implementation is a better implementation mode in more cases. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer floppy disk, a U disk, a mobile hard disk, a ROM, a RAM, a disk or an optical disk, etc., including a number of instructions to enable a computer device (which can be a personal computer, a training device, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0248] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0249] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, a computer, a training device, or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, training device, or data center. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device, a data center, etc. that includes one or more available media integrations. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)), etc.

Claims

1. A data processing method, characterized in that: The method comprises: Acquire first status information, where the first status information indicates a congestion condition of a network; Acquire first prediction information of first control information, wherein the first control information is any one of a control information set, the control information set includes multiple control information, each of the multiple control information indicates a data transmission rate, and the first prediction information of the first control information indicates the pros and cons of using the first control information under the network state indicated by the first state information; According to the first prediction information of each control information, target control information is determined from the control information set.

2. The method according to claim 1, characterized in that The acquiring first prediction information of the first control information includes: The first state information and the first control information are input into a first machine learning model to obtain the first prediction information output by the first machine learning model.

3. The method according to claim 2, characterized in that The first state information indicates the congestion status of the network at time t, when the weight parameter of the machine learning model adopts the first parameter value, it is the first machine learning model, and t is an integer greater than or equal to 1; The method is applied to a first device, wherein N second parameter values ​​are deployed in the first device, each of the second parameter values ​​includes a parameter value adopted by a weight parameter of the machine learning model at one of N moments before the moment t, N is an integer greater than or equal to 1, and the first parameter value is obtained by updating a third parameter value based on second state information at moment t-1, second control information adopted at moment t-1, and actual performance information of the second control information; Among them, the third parameter value is one of the N second parameter values, the status information at time t-1 indicates the congestion situation of the network at time t-1, the second control information indicates the data sending rate in the network at time t-1, and the actual performance information of the second control information represents the advantages and disadvantages of using the second control information in the network at time t-1.

4. The method according to claim 3, characterized in that When the weight parameter of the machine learning model adopts the second parameter value, it is a second machine learning model, and the method further includes: Inputting the second state information and the second control information into N second machine learning models corresponding to the N second parameter values ​​one by one, to obtain N second prediction information of the second control information, where the second prediction information represents the pros and cons of adopting the second control information in the network indicated by the second state information; According to the N second prediction information and the actual performance information, the third parameter value is updated to obtain the first parameter value.

5. The method according to claim 1, characterized in that The first state information indicates the congestion status of the network at time t, where t is an integer greater than or equal to 1, and the acquiring the first prediction information of the first control information includes: Inputting the first state information and the first control information into a third machine learning model to obtain third prediction information output by the third machine learning model, wherein the third prediction information indicates a congestion condition of the network at time t+1 when the first control information is used under the network state indicated by the first state information; The first prediction information is determined according to the third prediction information.

6. The method according to claim 5, characterized in that The first distribution space is modeled by the third machine learning model, and the first distribution space represents the distribution space where at least one information pair is located, each of the information pairs includes a combination information and predicted state information corresponding to the combination information, one combination information includes a third state information and a third control information, the third state information indicates the congestion condition of the network, the third control information is any one of the control information set, and the state information corresponding to the combination information indicates the congestion condition of the network at the next moment when the third control information is adopted under the network state indicated by the third state information.

7. The method according to claim 6, characterized in that The method further comprises: Acquire second state information at time t-1 and second control information used at the time t-1, wherein the state information at time t-1 indicates a congestion condition of the network at the time t-1; According to the state information at time t-1, the second control information and the first state information, the posterior distribution of the third machine learning model is updated to obtain the updated third machine learning model; The obtaining of third prediction information based on a third machine learning model according to the first state information and the first control information includes: obtaining the third prediction information from the distribution space indicated by the updated third machine learning model according to the first state information and the first control information.

8. The method according to any one of claims 1 to 7, characterized in that: The first state information indicates a congestion condition of the network at time t, and the first prediction information of the first state information indicates a difference between predicted state information and expected state information of the network at time t+1 when the first control information is used in the network state indicated by the first state information; The predicted state information of the network at time t+1 represents the predicted congestion situation of the network at time t+1, and the expected state information represents the expected congestion situation of the network.

9. The method according to any one of claims 1 to 7, characterized in that: The determining target control information from the control information set according to the first prediction information of each control information includes: The target control information is determined from the control information set according to the first prediction information of each control information and the uncertainty corresponding to each control information, wherein the uncertainty corresponding to each control information represents the uncertainty of a combination of each control information and the first state information.

10. The method according to any one of claims 1 to 7, characterized in that: Each of the control information includes the value of the receiving window RWND and / or the value of the sending window SWND.

11. A data processing device, characterized in that: The device comprises: An acquisition module, configured to acquire first status information, where the first status information indicates a congestion condition of a network; The acquisition module is further configured to acquire first prediction information of first control information, wherein the first control information is any one of a control information set, the control information set includes a plurality of control information, each of the plurality of control information indicates a data transmission rate, and the first prediction information of the first control information indicates the pros and cons of using the first control information under the network state indicated by the first state information; A determination module is used to determine target control information from the control information set according to the first prediction information of each control information.

12. The device according to claim 11, characterized in that The acquisition module is specifically used to input the first state information and the first control information into a first machine learning model to obtain the first prediction information output by the first machine learning model.

13. The device according to claim 12, characterized in that The first state information indicates the congestion status of the network at time t, when the weight parameter of the machine learning model adopts the first parameter value, it is the first machine learning model, and t is an integer greater than or equal to 11; The apparatus is applied to a first device, wherein N second parameter values ​​are deployed in the first device, each of the second parameter values ​​includes a parameter value adopted by a weight parameter of the machine learning model at one of N moments before the moment t, N is an integer greater than or equal to 11, and the first parameter value is obtained by updating a third parameter value based on the second state information at the moment t-11, the second control information adopted at the moment t-11, and the actual performance information of the second control information; Among them, the third parameter value is one of the N second parameter values, the status information at time t-11 indicates the congestion situation of the network at time t-11, the second control information indicates the data sending rate in the network at time t-11, and the actual performance information of the second control information represents the advantages and disadvantages of using the second control information in the network at time t-11.

14. The device according to claim 13, characterized in that When the weight parameter of the machine learning model adopts the second parameter value, it is a second machine learning model, and the device further includes: an input module, configured to input the second state information and the second control information into N second machine learning models corresponding one-to-one to the N second parameter values, to obtain N second prediction information of the second control information, wherein the second prediction information represents the pros and cons of adopting the second control information in the network indicated by the second state information; An updating module is used to update the third parameter value to obtain the first parameter value according to the N second prediction information and the actual performance information.

15. The device according to claim 11, characterized in that The first status information indicates the congestion status of the network at time t, where t is an integer greater than or equal to 11, and the acquisition module is specifically used to: Inputting the first state information and the first control information into a third machine learning model to obtain third prediction information output by the third machine learning model, wherein the third prediction information indicates a congestion condition of the network at time t+11 when the first control information is used under the network state indicated by the first state information; The first prediction information is determined according to the third prediction information.

16. The device according to claim 15, characterized in that The first distribution space is modeled by the third machine learning model, and the first distribution space represents the distribution space where at least one information pair is located, each of the information pairs includes a combination information and predicted state information corresponding to the combination information, one combination information includes a third state information and a third control information, the third state information indicates the congestion condition of the network, the third control information is any one of the control information set, and the state information corresponding to the combination information indicates the congestion condition of the network at the next moment when the third control information is adopted under the network state indicated by the third state information.

17. The device according to claim 16, characterized in that The acquisition module is further used to acquire the second state information at time t-11 and the second control information used at time t-11, wherein the state information at time t-11 indicates the congestion status of the network at time t-11; The device further includes: an updating module, configured to update the posterior distribution of the third machine learning model according to the state information at time t-11, the second control information, and the first state information, to obtain the updated third machine learning model; The acquisition module is specifically used to acquire the third prediction information from the distribution space indicated by the updated third machine learning model based on the first state information and the first control information.

18. The device according to any one of claims 11 to 17, characterized in that The first state information indicates a congestion condition of the network at time t, and the first prediction information of the first state information indicates a difference between predicted state information and expected state information of the network at time t+11 when the first control information is used in the network state indicated by the first state information; The predicted state information of the network at time t+11 represents the predicted congestion situation of the network at time t+11, and the expected state information represents the expected congestion situation of the network.

19. The device according to any one of claims 11 to 17, characterized in that The determination module is specifically used to determine the target control information from the control information set based on the first prediction information of each control information and the uncertainty corresponding to each control information, and the uncertainty corresponding to each control information represents the uncertainty of the combination of each control information and the first state information.

20. The device according to any one of claims 11 to 17, characterized in that Each of the control information includes the value of the receiving window RWND and / or the value of the sending window SWND.

21. A device, characterized in that comprising a processor and a memory, the processor being coupled to the memory, The memory is used to store programs; The processor is configured to execute the program in the memory so that the device performs the method according to any one of claims 1 to 10.

22. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a program, and when the program is executed on a computer, the computer is caused to execute the method according to any one of claims 1 to 10.

23. A computer program product, characterized in that The computer program product comprises a program, and when the program is run on a computer, the computer is caused to execute the method according to any one of claims 1 to 10 .

Images