Data processing method and related device

Abstract

A data processing method and a related device. The method enables the application of artificial intelligence technology to the field of communication. The method comprises: acquiring first state information indicating the congestion condition of a network (201); obtaining first prediction information of first control information, wherein the first control information is any one from a control information set, the control information set comprises a plurality of pieces of control information, each piece of control information indicates the sending rate of data, and the first prediction information of the first control information indicates the pros and cons of the use of the first control information in the network state indicated by the first state information (202); and determining target control information from the control information set on the basis of the first prediction information of each piece of control information (203). Evaluating the pros and cons of each piece of control information in conjunction with the congestion condition of the network in the present solution is conducive to improving the adaptability between the target control information and the congestion condition of the network and exploring a more preferred sending rate of data.

Description

A data processing method and related equipment

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office on September 28, 2023, with application number 202311294635.2 and invention name “A data processing method and related equipment”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of artificial intelligence, and in particular to a data processing method and related equipment. Background Art

[0003] With the continuous development of the information industry, the demand for data in various application scenarios is increasing. For example, in application scenarios such as data express and edge cloud computing, the amount of data transmitted is highly variable. This requires the network to be able to detect changes in transmission throughput as quickly as possible and adaptively adjust the data transmission rate. To address the challenges brought about by these changes, congestion control methods are currently being used. The goal of congestion control is to adjust the data transmission rate to alleviate or avoid congestion, thereby meeting the low-latency requirements of data transmission to a certain extent.

[0004] Specifically, during data transmission, the initial value of the receive window (RWND) can be set to infinity, and the value of the congestion window (CWND) continues to increase at a first growth rate until the length of the receive queue reaches the first length, the value of CWND continues to increase at a second growth rate, and the value of RWND is modified to 1. Since the data sending rate is determined based on the minimum value of CWND and RWND, the data sending rate is greatly reduced after the value of RWND is modified to 1; when the length of the receive queue is reduced to the second length, the value of RWND is modified to infinity again, and the value of CWND continues to increase at the first growth rate again; the above steps are repeated at least once until a packet loss event occurs, the value of CWND is halved, and the value of RWND is modified to 1, that is, one cycle is completed to continue to the next cycle.

[0005] However, the value of RWND in the above congestion control algorithm is either 1 or infinite. The congestion control algorithm is relatively rough as a whole, which is not conducive to exploring a better data transmission rate.

[0006] Summary of the Invention

[0007] The embodiments of the present application provide a data processing method and related equipment, which solve the problem in the prior art that the congestion control algorithm is rough and it is difficult to obtain a better data transmission rate.

[0008] The embodiments of this application provide the following technical solutions:

[0009] In a first aspect, an embodiment of the present application provides a data processing method that can apply artificial intelligence technology to the field of communications. In this method, a first device obtains first status information, which indicates the congestion status of the network; the first device obtains first prediction information of first control information; and then, based on the first prediction information of each control information in the control information set, the target control information can be determined from the control information set, and the target control information indicates a data transmission rate. The first control information is any one of the control information sets, and the control information set includes multiple control information, which indicates the data transmission rate, and the first prediction information of the first control information indicates the pros and cons of using the first control information under the network status indicated by the first status information.

[0010] In this implementation, each control information in the control information set indicates a data transmission rate. In the process of determining the target control information finally selected, the first prediction information of each control information is first obtained. Since the first prediction information of the first control information can indicate the pros and cons of adopting the first control information under the network state indicated by the first state information, the pros and cons of adopting each control information under the network state indicated by the first state information are obtained, and then a target control information is selected from the control information set based on the pros and cons of each control information. Through the above scheme, a more refined congestion control method is provided. Since in this scheme, each time the target control information to be adopted is determined, the pros and cons of each control information are evaluated in combination with the congestion situation of the network, it is beneficial to improve the adaptability between the target control information and the congestion situation of the network, and it is beneficial to make the finally determined data transmission rate more consistent with the control information of the actual congestion situation of the network, that is, it is beneficial to explore a better data transmission rate.

[0011] In one possible implementation, the first status information may indicate the congestion status of the network at time t, where t is an integer greater than or equal to 1. For example, the first status information may include any one or more of the following information: the actual usage of the congested queue at the starting time point of time t, the enqueue rate and the dequeue rate of the congested queue at time t-1, the average round-trip time (RRT) of all data transmission units transmitted at time t-1, the median value of the RRTs of all data transmission units transmitted at time t-1, the maximum value of the RRTs of all data transmission units transmitted at time t-1, or other information that can reflect the congestion status of the network.

[0012] Both "time t-1" and "time t" can be expressed as a time period with a first duration. For example, the first duration can be 5 seconds, 2 seconds, 1 second, 1500 milliseconds, 800 milliseconds or other lengths.

[0013] In one possible implementation, a specific implementation method for the first device to obtain the first prediction information of the first control information may include: the first device inputting the first state information and the first control information into a first machine learning model, and obtaining the first prediction information output by the first machine learning model. In this implementation, the input of the machine learning model includes the first state information and the first control information, and the output of the machine learning model is directly the first prediction information of the first control information. This facilitates quickly obtaining the first prediction information of the first control information after obtaining the first state information and the first control information, thereby facilitating the ultimate acquisition of the target control information at a faster speed.

[0014] In one possible implementation, the first state information is used to indicate the congestion status of the network at time t, where t is an integer greater than or equal to 1. When the weight parameter of the machine learning model adopts the first parameter value, it is the first machine learning model. N second parameter values ​​can be deployed in the first device, each second parameter value includes the parameter value adopted by the weight parameter of the machine learning model at one of the N moments before time t, where N is an integer greater than or equal to 1. Exemplarily, the first parameter value adopted in the first machine learning model can be obtained by updating the third parameter value based on the second state information at time t-1, the second control information adopted at time t-1, and the actual performance information obtained after the second control information is adopted at time t-1; wherein the third parameter value is one of the N second parameter values, the state information at time t-1 indicates the congestion status of the network at time t-1, the second control information indicates the data transmission rate adopted in the network state at time t-1, and the actual performance information obtained after the second control information is adopted at time t-1 represents the pros and cons of adopting the second control information in the network at time t-1.

[0015] Exemplarily, the third parameter value may be a set of parameter values ​​adopted by the machine learning model when determining the second control information adopted at time t-1; or, the third parameter value may also be any one of the N second parameter values.

[0016] In this implementation, the first machine learning model adopted at time t is determined based on the historical data at time t-1 and the second machine learning model adopted N times before time t. That is, the first parameter value adopted by the first machine learning model is updated in a timely manner using the latest historical data, which is conducive to improving the performance of the first machine learning model and thus helping to obtain better target control information.

[0017] In one possible implementation, when the weight parameter of the machine learning model adopts the second parameter value, it is a second machine learning model, that is, the difference between the first machine learning model and the second machine learning model is that different weight parameters are adopted. The method also includes: the first device inputs the second state information and the second control information into N second machine learning models corresponding to the N second parameter values, and can obtain a second prediction information of the second control information generated by each second machine learning model in the N second machine learning models, that is, can obtain N second prediction information corresponding to the second control information generated by the N second machine learning models; then the first device can update the third parameter value according to the N second prediction information and the actual performance information to obtain the first parameter value, wherein the second prediction information represents the pros and cons of adopting the second control information in the network indicated by the second state information.

[0018] Exemplarily, the first device can calculate the function value of the first loss function based on the N second prediction information corresponding to the second control information and the actual performance information brought about by adopting the second control information at time t-1, and use the back propagation algorithm to perform gradient update on the third parameter value according to the function value of the first loss function to obtain the first parameter value.

[0019] In this implementation, the pros and cons of adopting the second control information are predicted respectively based on the N second machine learning models before time t (that is, N second prediction information), and then the third parameter value is updated based on the actual performance information of the N second prediction information and the second control information, so as to obtain the first machine learning model adopted at time t, that is, the process of determining the value of the weight parameter of the first machine learning model is combined with the second machine learning model adopted at N moments before time t, and the second machine learning model adopted at N moments before time t has seen N types of congestion conditions in the network, which is conducive to obtaining a first machine learning model with better performance, and then conducive to obtaining better target control information, so as to improve the utilization rate of network communication resources while minimizing network congestion.

[0020] In one possible implementation, when each second prediction information is expressed as a comprehensive score value, the first device can obtain the distance between each second prediction information in the N second prediction information and the actual performance information, that is, it can obtain N distances corresponding one-to-one to the N second prediction information; the first device can also obtain the weight value corresponding to each second prediction information in the N second prediction information, and perform weighted summation on the N distances according to the weight value of each second prediction information to obtain the sum result, that is, the function value of the first loss function; wherein the first loss function indicates the similarity between the N second prediction information and the actual performance information, and the goal of training using the first loss function includes improving the similarity between the N second prediction information and the actual performance information.

[0021] In one possible implementation, the first status information indicates the congestion status of the network at time t, where t is an integer greater than or equal to 1. A specific implementation method for the first device to obtain the first prediction information of the first control information may include: the first device inputs the first status information and the first control information into a third machine learning model, obtains third prediction information output by the third machine learning model, and then determines the first prediction information based on the third prediction information; wherein the third prediction information indicates the congestion status of the network at time t+1 when the first control information is adopted in the network state indicated by the first status information.

[0022] Optionally, in one case, the first prediction information corresponding to the first state information indicates the difference between the predicted state information and the expected state information of the network at time t+1 when the first control information is adopted in the network state indicated by the first state information. The specific implementation method of the first device determining the first prediction information based on the third prediction information may include: the first device determines the first prediction information based on the third prediction information and the expected state information.

[0023] In this implementation, another scheme for obtaining the first prediction information of the first control information is provided, which improves the implementation flexibility of this scheme. The congestion situation of the network at time t+1 is first predicted by using a machine learning model based on the first state information and the first control information, and then the pros and cons of using the first control information under the network state indicated by the first state information are determined based on the congestion situation of the network at time t+1, so that the generation process of the first prediction information of the first control information is more refined, which is conducive to obtaining more accurate first prediction information of the first control information, and then is conducive to obtaining target control information that is more adapted to the network at time t, and is conducive to more effective use of the network bandwidth for data transmission within time t.

[0024] In one possible implementation, a first distribution space is modeled by a third machine learning model, and the first distribution space represents the distribution space where at least one information pair is located. Each information pair in the aforementioned at least one information pair includes a combination information and predicted state information corresponding to the combination information. The aforementioned combination information includes a third state information and a third control information. The third state information indicates the congestion of the network. The third control information is any one in the control information set. The state information corresponding to the combination information indicates the congestion of the network at the next moment when the third control information is adopted in the network state indicated by the third state information.

[0025] In this implementation, the third machine learning model is used to model the first distribution space. The first distribution space represents the distribution space of the information pairs consisting of (state information, control information) and the state information at the next moment. The first distribution space is used to determine the first prediction information of each control information. That is, a specific implementation scheme for determining the first prediction information of each control information is provided, which is conducive to reducing the implementation difficulty of this scheme.

[0026] In one possible implementation, in this method, the first device may also obtain second state information at time t-1 and second control information used at time t-1, and then update the posterior distribution of the third machine learning model based on the state information at time t-1, the second control information, and the first state information to obtain an updated third machine learning model; wherein the aforementioned state information at time t-1 indicates the congestion status of the network at time t-1. A specific implementation method for the first device to obtain third prediction information based on the third machine learning model according to the first state information and the first control information may include: the first device obtains the third prediction information from the distribution space indicated by the updated third machine learning model according to the first state information and the first control information.

[0027] In this implementation, after obtaining the historical data at time t-1, the historical data at time t-1 will be used in a timely manner to update the posterior distribution of the third machine learning model, which is conducive to improving the similarity between the first distribution space indicated by the third machine learning model and the real distribution space where at least one information pair is located, that is, it is conducive to modeling the real distribution space where at least one information pair is located through the third machine learning model, and is conducive to obtaining more accurate prediction state information with the help of the updated third machine learning model, and then is conducive to obtaining more accurate first prediction information, which is conducive to selecting better target control information.

[0028] In one possible implementation, the first state information indicates the congestion situation of the network at time t, and the first prediction information of the first state information indicates the difference between the predicted state information and the expected state information of the network at time t+1 when the first control information is adopted in the network state indicated by the first state information; wherein the aforementioned predicted state information of the network at time t+1 represents the predicted congestion situation of the network at time t+1, and the aforementioned expected state information represents the expected congestion situation of the network.

[0029] For example, the first prediction information of the first control information may be a first score, and the first score may include a score value corresponding to the parameter value of each first indicator and / or a score value corresponding to the parameter value of each second indicator. Alternatively, the first score may be a first comprehensive score value obtained by weighted summing the score values ​​corresponding to each first indicator and / or the score values ​​corresponding to each second indicator.

[0030] In this implementation, the first prediction information of the first control information indicates the difference between the predicted state information and the expected state information of the network at time t+1 when the first control information is adopted in the network state indicated by the first state information. Since the expected state information indicates the expected congestion situation of the network, the advantages and disadvantages of adopting the first control information are measured with the help of the expected state information, which is conducive to narrowing the similarity between the actual congestion situation of the network and the expected congestion situation, thereby helping users to more conveniently regulate the congestion situation of the network.

[0031] In one possible implementation, the first prediction information of the first control information may include only the predicted number of packets successfully sent within time t. Alternatively, the first prediction information of the first control information may include a second score; for example, the second score may include a score value corresponding to the parameter value of each third indicator in at least one third indicator, and / or a score value corresponding to the predicted number of packets successfully sent within time t. Exemplarily, the aforementioned at least one third indicator may be used to indicate the predicted congestion situation of the network at the starting time point of time t+1, for example, the aforementioned at least one third indicator may indicate: the predicted usage of the congestion queue at the starting time point of time t+1, and / or the predicted value corresponding to the RTT of all packets transmitted within time t.

[0032] In one possible implementation, the specific implementation method of the first device determining the target control information from the control information set based on the first prediction information of each control information may include: the first device determines the target control information from the control information set based on the first prediction information of each control information and the uncertainty corresponding to each of the aforementioned control information; wherein the uncertainty corresponding to each control information represents the uncertainty of the combination of each control information and the first state information.

[0033] Optionally, the uncertainty corresponding to each control information may be determined based on a contextual multi-armed bandit (CMAB) algorithm, wherein a higher uncertainty score for each control information indicates a higher uncertainty for a combination of each control information and the first state information; and a lower uncertainty score for each control information indicates a lower uncertainty for a combination of each control information and the first state information.

[0034] In this implementation, not only the advantages and disadvantages of each control information are considered, but also the uncertainty of the combination of each control information and the first state information is considered. This is conducive to actively exploring combinations with high uncertainty, and further conducive to fully exploring the advantages and disadvantages of each control information, so as to increase the probability that the target control information finally obtained is the optimal control information.

[0035] In one possible implementation, the uncertainty corresponding to each piece of control information may be specifically represented by an uncertainty score for each piece of control information. A specific implementation method for the first device to determine target control information from the set of control information based on the first prediction information of each piece of control information and the uncertainty corresponding to each piece of control information may include: the first device performing a weighted summation of the first comprehensive score (or second comprehensive score) of each piece of control information and the uncertainty score of each piece of control information, thereby obtaining a total score for each piece of control information; and then determining the target control information from the set of control information based on the total score of each piece of control information.

[0036] In one possible implementation, each control message includes a receive window RWND value and / or a send window SWND value. This implementation specifies the specific information carried by the control message, which helps reduce the implementation difficulty of this solution. Furthermore, the RWND parameter exists in all versions of the Transmission Control Protocol (TCP). Choosing to carry the RWND value in the control message helps improve the universality of this solution, and thus expands its application scenarios.

[0037] In a second aspect, an embodiment of the present application provides a data processing device, which can apply artificial intelligence technology to the field of communications. The data processing device includes: an acquisition module, used to obtain first status information, the first status information indicates the congestion of the network; the acquisition module is also used to obtain first prediction information of first control information, wherein the first control information is any one in a control information set, the control information set includes multiple control information, each of the multiple control information indicates the data sending rate, and the first prediction information of the first control information indicates the pros and cons of using the first control information under the network state indicated by the first status information; a determination module is used to determine the target control information from the control information set based on the first prediction information of each control information.

[0038] In the second aspect, the data processing device can also execute the steps performed by the first device in the first aspect and various possible implementation methods of the first aspect. The specific implementation methods, meanings of terms and beneficial effects brought about by the steps in each possible implementation method of the second aspect of this application can all be referred to the first aspect and will not be repeated here.

[0039] In a third aspect, an embodiment of the present application provides a device comprising a processor and a memory, wherein the processor is coupled to the memory, the memory is used to store programs, and the processor is used to execute the programs in the memory, so that the device executes the data processing method of the first aspect mentioned above.

[0040] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored. When the computer-readable storage medium is run on a computer, the computer executes the data processing method of the first aspect.

[0041] In a fifth aspect, an embodiment of the present application provides a computer program product, which includes a program. When the program is run on a computer, it enables the computer to execute the data processing method of the first aspect mentioned above.

[0042] In a sixth aspect, the present application provides a chip system, which includes a processor for supporting the implementation of the functions involved in the above aspects, for example, sending or processing the data and / or information involved in the above methods. In one possible design, the chip system also includes a memory, which is used to store program instructions and data necessary for the terminal device or communication device. The chip system can be composed of a chip or can include a chip and other discrete devices. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] FIG1 is a schematic diagram of an architecture of a communication system provided in an embodiment of the present application;

[0044] FIG2 is a flow chart of a data processing method provided in an embodiment of the present application;

[0045] FIG3 is another flow chart of a data processing method according to an embodiment of the present application;

[0046] FIG4 is a schematic diagram of determining a first parameter value according to an embodiment of the present application;

[0047] FIG5 is another flow chart of a data processing method according to an embodiment of the present application;

[0048] FIG6 is another flow chart of a data processing method according to an embodiment of the present application;

[0049] FIG7 is another flow chart of a data processing method according to an embodiment of the present application;

[0050] FIG8 is a schematic structural diagram of a data processing device provided in an embodiment of the present application;

[0051] FIG9 is a schematic structural diagram of a device provided in an embodiment of the present application;

[0052] FIG10 is a schematic diagram of the structure of a chip provided in an embodiment of the present application. DETAILED DESCRIPTION

[0053] The embodiments of the present application are described below in conjunction with the accompanying drawings. Those skilled in the art will appreciate that, with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0054] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequential order. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, and this is merely a way of distinguishing the objects of the same attributes when describing them in the embodiments of the present application. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, so that the process, method, system, product or equipment comprising a series of units need not be limited to those units, but may include other units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0055] In the embodiments of the present application, "sending" and "receiving" indicate the direction of signal transmission. For example, "sending information to XX device" can be understood as the destination of the information being XX device, which can include direct sending through the air interface, as well as indirect sending through the air interface by other units or modules. "Receiving information from YY device" can be understood as the source of the information being YY device, which can include direct receiving from YY device through the air interface, as well as indirect receiving from YY device through the air interface from other units or modules. "Sending" can also be understood as the "output" of the chip interface, and "receiving" can also be understood as the "input" of the chip interface. In other words, sending and receiving can be performed between devices or within a device, for example, between components, modules, chips, software modules or hardware modules within the device through a bus, trace or interface. It is understandable that information may undergo necessary processing, such as encoding, modulation, etc., between the source and destination of the information, but the destination can understand the valid information from the source. Similar expressions in this application can be understood similarly and will not be repeated.

[0056] In the embodiments of the present application, "indication" may include direct indication and indirect indication, and may also include explicit indication and implicit indication. The information indicated by a certain information (such as the indication information described below) is called information to be indicated. In the specific implementation process, there are many ways to indicate the information to be indicated, such as but not limited to, the information to be indicated can be directly indicated, such as the information to be indicated itself or the index of the information to be indicated. The information to be indicated can also be indirectly indicated by indicating other information, wherein there is an association between the other information and the information to be indicated; it is also possible to indicate only a part of the information to be indicated, while the other parts of the information to be indicated are known or agreed in advance, for example, the indication of specific information can be achieved with the help of the arrangement order of each information agreed in advance (such as predefined by the protocol), thereby reducing the indication overhead to a certain extent. The present application does not limit the specific method of indication. It is understandable that, for the sender of the indication information, the indication information can be used to indicate the information to be indicated, and for the receiver of the indication information, the indication information can be used to determine the information to be indicated.

[0057] The method provided in this application can be applied in the field of communications. For example, it can be applied in the application scenario of data transmission to adjust the data transmission rate of the transmitting end; optionally, a machine learning model can be used in the process of determining the aforementioned data transmission rate. Before describing the method provided in this application in detail, the scenario in which the method provided in the embodiment of this application is applied is first introduced.

[0058] Please refer to Figure 1, which is a schematic diagram of the architecture of a communication system provided in an embodiment of the present application. The method provided in this application can be applied to a wide area network (WAN). A WAN is a remote network that connects local area networks (LANs) or metropolitan area networks (MANs) in different regions. As shown in Figure 1, a communication connection is established between a sender and a receiver via the WAN. When data is transmitted between the sender and the receiver, it may be forwarded through multiple transit devices.

[0059] For example, the transit device may include but is not limited to a router, a firewall, a gateway or other network device with forwarding function, etc., which is not limited in this application.

[0060] For example, the sending end (or receiving end) may be specifically manifested as a server, a server cluster, or other devices.

[0061] For another example, the transmitting end (or receiving end) may be a handheld device, an in-vehicle device, a wearable device, a computing device, or other processing device with wireless communication capabilities, and the list is not exhaustive. The transmitting end (or receiving end) may communicate with one or more core networks. For example, the transmitting end (or receiving end) may be a portable, pocket-sized, handheld, computer-built-in, or in-vehicle mobile device that exchanges voice and / or data with a wireless access network. Exemplarily, the transmitting end (or receiving end) may be a user agent, a cellular phone, a smart phone, a personal digital assistant (PDA), a tablet personal computer (Tablet PC), a wireless modem, a handset, a laptop computer, a personal communication service (PCS) phone, a remote station, an access point (AP), a remote terminal, an access terminal, customer premises equipment (CPE), a terminal, user equipment (UE) or a mobile terminal (MT), etc.

[0062] For another example, the transmitting end (or receiving end) can also be a wearable device, which is a general term for wearable devices that are intelligently designed and developed by applying wearable technology to everyday wear, such as glasses, gloves, watches, clothing and shoes. A wearable device is a portable device that is worn directly on the body or integrated into the user's clothing or accessories. Wearable devices are not just hardware devices, but also achieve powerful functions through software support, data interaction, and cloud interaction. Broadly speaking, wearable smart devices include those that are full-featured, large in size, and can achieve full or partial functions without relying on smartphones, such as smart watches or smart glasses, as well as those that only focus on a certain type of application function and need to be used in conjunction with other devices such as smartphones, such as various smart bracelets, smart helmets, and smart jewelry for vital sign monitoring.

[0063] For another example, the transmitting end (or receiving end) may also be a drone, a robot, a terminal device in device-to-device communication (D2D), a terminal device in vehicle-to-everything (V2X), a virtual reality (VR) device, an augmented reality (AR) device, a wireless terminal in industrial control, a terminal device in self-driving, a terminal device in remote medical care, a terminal device in a smart grid, a wireless terminal in a smart city, a terminal device in a smart home, etc.

[0064] In addition to the scenario described in Figure 1 above, the method provided by the embodiment of the present application can also be applied to other communication system scenarios. For example, in a smart factory scenario, different devices (such as intelligent robots, lathes, and transport vehicles) are connected to the same communication network through a router and transmit data to each other through the communication network; the control information determined by the router can indicate the data transmission rate of the aforementioned devices, etc. The embodiment of the present application does not limit the specific scenario in which the data processing method is applied.

[0065] In the above-mentioned application scenarios, when data is transmitted between the transmitting end and the receiving end, in order to explore a better data transmission rate, the present application provides a data processing method, which is executed by a first device. Optionally, the first device can be a transit device, and the first device can be connected to one or more transmitting ends. For example, the first device can be specifically manifested as a switch, a router, a firewall, a base station or other network device with a forwarding function, etc. The specific deployment in which the device can be flexibly determined in combination with the actual application scenario, and is not limited in the embodiment of the present application. Specifically, please refer to Figure 2, which is a schematic diagram of the data processing method provided in the embodiment of the present application. The data processing method provided by the present application may include:

[0066] 201. Obtain first status information, where the first status information indicates a congestion situation of a network.

[0067] In an embodiment of the present application, the first device may obtain first status information. Exemplarily, the first status information may indicate the congestion status of the network at time t, where t is an integer greater than or equal to 1. For example, the first status information may include any one or more of the following information: the actual usage of the congested queue at the starting time point of time t, the enqueue rate and the dequeue rate of the congested queue at time t-1, the average value of the round-trip time (RRT) of all data transmission units transmitted at time t-1, the median value of the RRT of all data transmission units transmitted at time t-1, the maximum value of the RRT of all data transmission units transmitted at time t-1, or other information that can reflect the congestion status of the network, etc. The specific information to be used can be flexibly determined in combination with the actual application scenario and is not limited here.

[0068] For example, the "actual usage of the congested queue at the starting time point of time t" may include the actual number of data transmission units to be transmitted in the congested queue, and / or the ratio between the actual number and the total length of the congested queue, and / or other parameter values ​​that can reflect the actual usage of the congested queue at the starting time point of time t; the "total length of the congested queue" can also be understood as the "total number of data transmission units that the congested queue can accommodate."

[0069] The "data transmission unit" can be specifically expressed as a message, a data packet or other types of data transmission units in the network, etc. The specific form of the "data transmission unit" can be determined in combination with the actual application scenario, and is not limited in this application; it should be noted that in the subsequent embodiments of this application, only the "data transmission unit" expressed as a message is used as an example for explanation. For the case where the "data transmission unit" is expressed in other forms, you can refer to the case where the "data transmission unit" is expressed as a message for understanding, and will not be repeated one by one in the embodiments of this application.

[0070] Both "t-1 moment" and "t moment" can be expressed as a time period with a first duration. For example, the first duration can be 5 seconds, 2 seconds, 1 second, 1500 milliseconds, 800 milliseconds or other lengths, etc. The examples here are only for the convenience of understanding this solution and are not used to limit this solution.

[0071] The RRT of a data transmission unit represents the total delay from the time the sender starts sending the data transmission unit to the time the sender receives the information from the receiver confirming the receipt of the data transmission unit.

[0072] 202. Obtain first prediction information of first control information, wherein the first control information is any one in a control information set, the control information set includes multiple control information, each control information in the multiple control information indicates a data sending rate, and the first prediction information of the first control information indicates the pros and cons of using the first control information under the network state indicated by the first state information.

[0073] In an embodiment of the present application, the first device is able to obtain a control information set, which includes multiple control information; after determining the first status information, it can obtain the first prediction information of any control information (that is, the first control information) in the control information set; the first device repeats the aforementioned steps multiple times, and is able to obtain the first prediction information of each control information in the control information set. The "first prediction information of each control information" can also be understood as the predicted performance information of each control information. The first prediction information of each control information indicates the pros and cons of adopting the control information under the network state indicated by the first status information.

[0074] For example, in one case, if each control information indicates the sending rate of the message, each control information may include the value of RWND and / or the value of the send window (SWND). If the control information carries the value of RWND, the sender can determine the final SWND value based on the value of RWND carried in the control information and the value of CWND determined by the sender. If the control information carries the value of SWND, the sender can determine the final SWND value based on the value of SWND carried in the control information. In the embodiment of the present application, the specific information carried by the control information is refined, which is conducive to reducing the difficulty of implementing the solution. In different versions of the transmission control protocol (TCP), the parameter RWND exists. Choosing to carry the value of RWND in the control information is conducive to improving the universality of the solution, that is, it is conducive to expanding the application scenarios of the solution.

[0075] In another case, if each control information indicates the sending rate of the data packet, each control information may also carry other information. The specific information carried in each control information can be determined in combination with the actual application scenario and is not limited here.

[0076] Regarding the meaning of "first predicted information of the first control information", in one case, the first predicted information of the first control information may indicate the difference between the predicted state information at time t+1 and the expected state information when the first control information is adopted under the network state indicated by the first state information (i.e., the network state at time t); and / or, the first predicted information of the first control information may include the difference between the predicted number and the expected number of messages successfully sent at time t.

[0077] For example, the first prediction information of the first control information may be a first score, and the first score may include a score value corresponding to the parameter value of each first indicator and / or a score value corresponding to the parameter value of each second indicator. Alternatively, the first score may be a first comprehensive score value obtained by weighted summing the score values ​​corresponding to each first indicator and / or the score values ​​corresponding to each second indicator.

[0078] The parameter value of the above-mentioned at least one first indicator can indicate any one or more of the following information: the difference between the predicted usage and the expected usage of the congested queue at the starting time point of time t+1, the difference between the predicted value and the expected value corresponding to the RTT of all packets transmitted within time t, or other information, etc., that is, the expected state information can include any one or more of the following information: the expected usage of the congested queue, the expected value of RTT or other information, etc., which are not listed here exhaustively.

[0079] Furthermore, "the difference between the predicted usage and the expected usage of the congested queue at the starting time point of time t+1" can be specifically reflected by any one or more of the following first indicators: the difference between the predicted number and the expected number of packets to be transmitted in the congested queue at time t+1; the ratio between the predicted number and the expected number of packets to be transmitted in the congested queue at time t+1; the difference between the first ratio and the second ratio, the first ratio representing the ratio between the predicted number of packets to be transmitted in the congested queue at time t+1 and the total length of the congested queue, and the second ratio representing the ratio between the expected number of packets to be transmitted in the congested queue and the total length of the congested queue; or other indicators, etc., which are not exhaustively listed here.

[0080] The "predicted value corresponding to the RTT of all messages transmitted within time t" may include any one or more of the following: the average value of the RRT of all messages transmitted within time t, the median value of the RRT of all messages transmitted within time t, the maximum value of the RRT of all messages transmitted within time t, or other values ​​obtained based on the values ​​of the RRT of all messages transmitted within time t; correspondingly, the "expected value of RTT" may include any one or more of the following: the average value of the RRT of all messages transmitted within a unit time, the median value of the RRT of all messages transmitted within a unit time, the maximum value of the RRT of all messages transmitted within a unit time, or other values ​​obtained based on the values ​​of the RRT of all messages transmitted within a unit time, etc., which are not limited in the embodiments of the present application.

[0081] "The difference between the predicted value and the expected value corresponding to the RTT of all messages transmitted within time t" can be reflected by any one or more of the following first indicators: the difference between the predicted value and the expected value corresponding to the RTT of all messages transmitted within time t, the ratio between the aforementioned predicted value and the expected value, or other indicators, etc., which are not exhaustive here.

[0082] "The difference between the predicted number and the expected number of messages successfully sent at time t" may include the parameter value of each indicator in at least one second indicator. The aforementioned at least one second indicator may include any one or more of the following: the difference between the predicted number and the expected number of messages successfully sent at time t, the ratio between the aforementioned predicted number and the expected number, or other indicators that can reflect the "difference between the aforementioned predicted number and the expected number", etc., which are not enumerated here.

[0083] Optionally, the higher the score value corresponding to each first indicator (or each second indicator), the better the performance of using the first control information in the evaluation dimension of the indicator; the higher the score value corresponding to each first indicator (or each second indicator), the worse the performance of using the first control information in the evaluation dimension of the indicator.

[0084] For example, a higher score corresponding to the indicator "the difference between the predicted number of packets to be transmitted by the congested queue at time t+1 and the expected number" indicates a smaller difference between the predicted number of packets to be transmitted by the congested queue at time t+1 and the expected number, and a better performance of the first control information in the evaluation dimension of "the difference between the predicted number of packets to be transmitted by the congested queue at time t+1 and the expected number".

[0085] The higher the score value corresponding to the indicator "the ratio between the predicted number of messages successfully sent within time t and the expected number", the larger the parameter value of the indicator "the ratio between the predicted number of messages successfully sent within time t and the expected number", which means that the performance of the first control information in the evaluation dimension "the ratio between the predicted number of messages successfully sent within time t and the expected number" is better, and so on. The examples here are only for the convenience of understanding the meaning represented by the score value corresponding to a certain indicator. In the embodiments of the present application, each indicator in all the first indicators and the second indicators is no longer combined for examples one by one, nor is this scheme limited.

[0086] Optionally, a higher first comprehensive score value indicates that it is better to adopt the first control information in the network indicated by the first status information; a lower first comprehensive score value indicates that it is worse to adopt the first control information in the network indicated by the first status information.

[0087] It should be noted that the meanings of “each first indicator”, “each second indicator”, “the scoring value corresponding to each first indicator” and “the scoring value corresponding to each second indicator” can be found above and will not be repeated here.

[0088] In an embodiment of the present application, the first prediction information of the first control information indicates the difference between the predicted state information and the expected state information of the network at time t+1 when the first control information is adopted in the network state indicated by the first state information. Since the expected state information indicates the expected congestion situation of the network, the advantages and disadvantages of adopting the first control information are measured with the help of the expected state information, which is conducive to narrowing the similarity between the actual congestion situation of the network and the expected congestion situation, thereby helping users to more conveniently regulate the congestion situation of the network.

[0089] In another case, the first prediction information of the first control information may only include the predicted number of messages successfully sent within time t.

[0090] In another embodiment, the first prediction information of the first control information may include a second score; for example, the second score may include a score corresponding to a parameter value of each third indicator in the at least one third indicator, and / or a score corresponding to a predicted number of packets successfully transmitted at time t. The at least one third indicator may indicate a predicted congestion condition of the network at the starting time point of time t+1. For example, the at least one indicator may indicate any one or more of the following information: a predicted usage of a congestion queue at the starting time point of time t+1, a predicted value corresponding to the RTT of all packets transmitted at time t, or other information that can reflect the congestion condition of the network at the starting time point of time t+1. It should be understood that which indicators can reflect the "usage of the congestion queue at the starting time point of time t+1" and the "predicted value corresponding to the RTT of all packets transmitted at time t" can be understood in conjunction with the above description and will not be elaborated here.

[0091] Optionally, the higher the score value corresponding to each third indicator (or "the predicted number of messages successfully sent within time t"), the better the performance of using the first control information in the evaluation dimension of the indicator; the higher the score value corresponding to each third indicator (or "the predicted number of messages successfully sent within time t"), the worse the performance of using the first control information in the evaluation dimension of the indicator.

[0092] A higher score corresponding to the predicted number of messages successfully sent at time t indicates that more messages were successfully sent at time t; a lower score corresponding to the predicted number of messages successfully sent at time t indicates that fewer messages were successfully sent at time t.

[0093] Alternatively, the second score may be a second comprehensive score obtained by weighted summing the score corresponding to each third indicator and / or the score corresponding to the predicted number of messages successfully sent at time t. Optionally, a higher second comprehensive score indicates a more favorable adoption of the first control information in the network indicated by the first state information; a lower second comprehensive score indicates a less favorable adoption of the first control information in the network indicated by the first state information.

[0094] It should be noted that the meaning of "the first prediction information of other control information in the control information set" is similar to the meaning of "the first prediction information of the first control information", which can be understood by reference and will not be repeated in the embodiments of this application.

[0095] Optionally, the first device may utilize a machine learning model to perform step 202. For example, in one implementation, a machine learning model is utilized to model a first mapping relationship between the combination information and the predicted performance information of the aforementioned combination information. Any combination information (hereinafter referred to as "target combination information" for ease of description) may include a combination of any state information (hereinafter referred to as "third state information" for ease of description) and any control information in the control information set (hereinafter referred to as "third control information" for ease of description). The aforementioned third state information is used to indicate the congestion status of the network; the predicted performance information of the target combination information indicates the pros and cons of using the third control information under the network status indicated by the third state information.

[0096] It should be noted that the concepts of "second state information" and "second control information" will be used in subsequent descriptions and will not be introduced here; the meaning of "target state information" is similar to the above-mentioned "first state information", and the meaning of "predicted performance information" is similar to the above-mentioned "first prediction information". You can refer to the above description for understanding and will not repeat them here.

[0097] Step 202 may include: the first device inputting the first state information and the first control information into a machine learning model, and obtaining first prediction information of the first control information output by the machine learning model. In an embodiment of the present application, the input of the machine learning model includes the first state information and the first control information, and the output of the machine learning model is directly the first prediction information of the first control information. This facilitates quickly obtaining the first prediction information of the first control information after obtaining the first state information and the first control information, thereby facilitating a faster acquisition of the target control information.

[0098] In another implementation, a machine learning model is used to model a first distribution space. The first distribution space represents the distribution space of at least one first information pair. Each first information pair may include a combination of information and predicted state information corresponding to the combination of information. It should be noted that the meaning of "combination of information" can be found in the above description and will not be further elaborated here.

[0099] Then step 202 may include: the first device inputs the first state information and the first control information into the machine learning model to obtain the third prediction information output by the machine learning model, the third prediction information indicating the congestion situation of the network at time t+1 when the first control information is adopted in the network state indicated by the first state information. The first device can determine the pros and cons of adopting the first control information (i.e., the first prediction information) in the network state indicated by the first state information based on the congestion situation of the network at time t+1 when the first control information is adopted in the network state indicated by the first state information (i.e., the third prediction information). It should be noted that the concept of "second prediction information" will be used in subsequent descriptions and will not be introduced here.

[0100] Optionally, if the first prediction information of the first state information indicates the difference between the predicted state information and the expected state information of the network at time t+1 when the first control information is adopted in the network state indicated by the first state information, the first device determines the first prediction information based on the third prediction information, which may include: the first device determines the first prediction information based on the third prediction information and the expected state information.

[0101] In an embodiment of the present application, another scheme for obtaining first prediction information of first control information is provided, which improves the implementation flexibility of this scheme, and firstly predicts the congestion situation of the network at time t+1 based on the first state information and the first control information using a machine learning model, and then determines the pros and cons of using the first control information under the network state indicated by the first state information based on the congestion situation of the network at time t+1, so that the generation process of the first prediction information of the first control information is more refined, which is conducive to obtaining more accurate first prediction information of the first control information, and then is conducive to obtaining target control information that is more adapted to the network at time t, and is conducive to more effective use of the network bandwidth for data transmission within time t.

[0102] The third machine learning model is used to model the first distribution space. The first distribution space represents the distribution space of the information pairs consisting of (state information, control information) and the state information at the next moment. The first distribution space is used to determine the first prediction information of each control information. That is, a specific implementation scheme for determining the first prediction information of each control information is provided, which is conducive to reducing the implementation difficulty of this scheme.

[0103] The first device may repeatedly perform step 202 multiple times to obtain first prediction information of each control information in the control information set, that is, to determine the pros and cons of using each control information under the network state indicated by the first state information.

[0104] 203. Determine target control information from the control information set according to the first prediction information of each control information.

[0105] In an embodiment of the present application, after obtaining the first prediction information of each control information in the control information set, the first device can select at least one target control information from the multiple control information included in the control information set based on the pros and cons of adopting each control information under the network state indicated by the first state information; the meanings of "target control information" and "first control information" are similar, and can be understood by referring to the above description, and will not be repeated here.

[0106] Exemplarily, in one implementation, the target control information is optimal control information under the network state indicated by the first state information, determined based on the first prediction information of each control information.

[0107] Optionally, step 203 may include: the first device determines the target control information from the control information set based on the first prediction information of each control information and the uncertainty corresponding to each control information, and the uncertainty corresponding to each control information represents the uncertainty of the combination of each control information and the first state information.

[0108] Exemplarily, the uncertainty corresponding to each control information can be specifically expressed as an uncertainty score for each control information; when the first prediction information of the first control information is expressed as a first comprehensive score value or a second comprehensive score value, the first device can perform a weighted summation of the first comprehensive score value (or the second comprehensive score value) of each control information and the uncertainty score of each control information to obtain a total score for each control information; and then determine the target control information from the control information set based on the total score of each control information.

[0109] Optionally, the target control information is the control information with the highest total score among the multiple control information included in the control information set.

[0110] Optionally, the uncertainty corresponding to each control information may be determined based on a contextual multi-armed bandit (CMAB) algorithm, wherein a higher uncertainty score for each control information indicates a higher uncertainty for a combination of each control information and the first state information; and a lower uncertainty score for each control information indicates a lower uncertainty for a combination of each control information and the first state information.

[0111] Optionally, if a machine learning model is utilized during the execution of step 202, the uncertainty corresponding to each control information may also represent the uncertainty of a combination of each control information, the first state information, and the machine learning model used in step 202.

[0112] To further understand this solution, an example of a formula used by the first device when executing step 203 is disclosed below: t =argmax a θ t [s t ;a]+cf(s t ,a;θ t );(1)

[0113] Among them, a t Represents target control information, s t represents the first state information, a represents the control information set; θ t [s t ; a] represents obtaining the first comprehensive score value (or second comprehensive score value) corresponding to each control information in the control information set; θ t represents the machine learning model for processing the first state information in step 202; cf(s t ,a;θ t ) represents the acquisition of s t ,θ t The uncertainty score of the combination with each control information in the control information set; θ t [s t ;a]+cf(s t ,a;θ t ) represents adding the first comprehensive score value (or the second comprehensive score value) corresponding to each control information in the control information set to the uncertainty score to obtain the total score of each control information; argmax a θ t [s t ;a]+cf(s t ,a;θ t ) represents obtaining the target control information with the highest total score in the control information set. It should be understood that the example in formula (1) is only for the convenience of understanding this solution and is not used to limit this solution.

[0114] In the embodiment of the present application, not only the advantages and disadvantages of each control information are considered, but also the uncertainty of the combination of each control information and the first state information is considered, which is conducive to actively exploring combinations with high uncertainty, and further conducive to fully exploring the advantages and disadvantages of each control information, so as to increase the probability that the target control information finally obtained is the optimal control information.

[0115] Optionally, the first device may also send the aforementioned target control information to each sending end connected to the first device for communication, where the target control information is used by each sending end to determine a message sending rate.

[0116] In this implementation, each control information in the control information set indicates a data transmission rate. In the process of determining the target control information finally selected, the first prediction information of each control information is first obtained. Since the first prediction information of the first control information can indicate the pros and cons of adopting the first control information under the network state indicated by the first state information, the pros and cons of adopting each control information under the network state indicated by the first state information are obtained, and then a target control information is selected from the control information set based on the pros and cons of each control information. Through the above scheme, a more refined congestion control method is provided. Since in this scheme, each time the target control information to be adopted is determined, the pros and cons of each control information are evaluated in combination with the congestion situation of the network, it is beneficial to improve the adaptability between the target control information and the congestion situation of the network, and it is beneficial to make the finally determined data transmission rate more consistent with the control information of the actual congestion situation of the network, that is, it is beneficial to explore a better data transmission rate.

[0117] In combination with the above description, it can be seen that the process of acquiring the first prediction information of each control information in the control information set can be assisted by a machine learning model. Optionally, in one case, the first device can not only execute the steps of the reasoning phase of the aforementioned machine learning model, but also execute the steps of the training phase of the aforementioned machine learning model, that is, update the parameters of the aforementioned machine learning model. In another case, the first device only executes the steps of the reasoning phase of the aforementioned machine learning model, and other devices (hereinafter referred to as the second device for the convenience of description) execute the steps of the training phase of the aforementioned machine learning model, that is, the second device updates the parameters of the aforementioned machine learning model. The specific implementation processes of the above two cases are different. The detailed implementation processes of the above two cases are respectively introduced below.

[0118] 1. The first device performs the steps of the inference phase of the machine learning model and updates the parameters of the machine learning model

[0119] In the embodiment of the present application, referring to the description in the embodiment corresponding to Figure 2 above, it can be seen that there may be two situations for the object modeled by the machine learning model. The above two situations are described below respectively.

[0120] (1) Using machine learning models to model the mapping relationship between combined information and the predicted performance information of the aforementioned combined information

[0121] Specifically, please refer to FIG3 , which is another flow chart of a data processing method provided in an embodiment of the present application. As shown in FIG3 , the data processing method provided in the present application may include:

[0122] 301. The first device obtains the second state information at time t-1, the second control information used at time t-1, and the actual performance information of the second control information. The second state information at time t-1 indicates the congestion status of the network at time t-1, and the actual performance information of the second control information indicates the pros and cons of using the second control information in the network at time t-1.

[0123] In this embodiment of the present application, step 301 is optional. Before using the machine learning model (hereinafter referred to as the "first machine learning model" for ease of distinction) to determine the second control information used at time t, the second state information at time t-1, the second control information used at time t-1, and the actual performance information of the second control information may also be obtained.

[0124] The meaning of "second state information" is similar to that of "first state information", except that t-1 in the above description of the first state information is replaced by t-2; the meaning of "second control information" is similar to that of "first control information", and the meanings of "second state information" and "second control information" can be understood by referring to the above embodiment corresponding to Figure 2, and will not be repeated here.

[0125] The information type included in the "actual performance information of the second control information" is similar to the information type included in the "first prediction information of the first control information", with the difference being that the "actual performance information of the second control information" and the "first prediction information of the first control information" are generated in different ways, and the "actual performance information of the second control information" is determined based on the first status information and / or the number of messages successfully sent at time t-1.

[0126] In one case, the actual performance information of the second control information may indicate the difference between the first state information and the expected state information used to reflect the congestion situation of the network at time t when the second control information is used under the network state indicated by the second state information (i.e., the network state at time t-1); and / or, the actual performance information of the second control information may include the difference between the actual number of messages successfully sent at time t-1 and the expected number.

[0127] The meaning of “expected state information” can be found in the description of the embodiment corresponding to FIG2 above, and will not be elaborated here.

[0128] In one implementation, the "actual performance information of the second control information" may include a score value corresponding to the parameter value of each fourth indicator in at least one fourth indicator, and / or a score value corresponding to the parameter value of each fifth indicator in at least one fifth indicator.

[0129] The parameter value of the above-mentioned at least one fourth indicator can indicate any one or more of the following information: the difference between the actual usage and the expected usage of the congestion queue at the starting time point of time t, the difference between the actual value and the expected value corresponding to the RTT of all messages transmitted within time t-1, or other information, etc., which is not limited here.

[0130] Which indicators the at least one fourth indicator specifically includes can be understood in conjunction with the description of the "at least one first indicator" in the embodiment corresponding to Figure 2 above. The difference is that the t+1 moment in the description of the embodiment corresponding to Figure 2 is replaced by the t moment, the predicted number of messages to be transmitted at the t+1 moment in the embodiment corresponding to Figure 2 is replaced by the actual number of messages to be transmitted at the t moment, and the t moment in the embodiment corresponding to Figure 2 is replaced by the t-1 moment. The specific indicators of the at least one fourth indicator will no longer be displayed and described here.

[0131] "The difference between the actual number of messages successfully sent within time t-1 and the expected number" may include the parameter value of each indicator in at least one fifth indicator. The aforementioned at least one fifth indicator may include any one or more of the following: the difference between the actual number of messages successfully sent within time t-1 and the expected number, the ratio between the aforementioned actual number and the expected number, or other indicators that can reflect the "difference between the aforementioned actual number and the expected number", etc., which are not enumerated here.

[0132] Then in step 301, after obtaining the first status information indicating the congestion situation of the network at time t, the first device can determine the parameter value of each fourth indicator based on the first status information and the expected status information, and then obtain the score value corresponding to the parameter value of each fourth indicator; and / or, after determining the parameter value of each fifth indicator based on the actual number and expected number of messages successfully sent at time t-1, determine the score value corresponding to the parameter value of each fifth indicator.

[0133] The "score value corresponding to the parameter value of the fourth indicator" may also be referred to as the "score value corresponding to the fourth indicator," and the "score value corresponding to the parameter value of the fifth indicator" may also be referred to as the "score value corresponding to the fifth indicator." It should be noted that the meaning of the "score value corresponding to the fourth indicator" can be understood by referring to the description of the "score value corresponding to the first indicator," and the meaning of the "score value corresponding to the fifth indicator" can be understood by referring to the description of the "score value corresponding to the second indicator," and these will not be elaborated here.

[0134] In another implementation, the actual performance information of the second control information may include a third comprehensive score value obtained by weighted summing the score values ​​corresponding to each fourth indicator and / or the score values ​​corresponding to each fifth indicator. In step 301, after determining the score value corresponding to each fourth indicator and / or the score value corresponding to each fifth indicator, the first device may further perform a weighted summation of all the aforementioned score values ​​to obtain the third comprehensive score value.

[0135] In another case, the actual performance information of the second control information may include the actual number of messages successfully sent within time t-1.

[0136] In another case, the actual performance information of the second control information may include a third score; for example, the third score may include a score value corresponding to the parameter value of each sixth indicator in at least one sixth indicator, and / or a score value corresponding to the actual number of messages successfully sent at time t-1.

[0137] For example, at least one sixth indicator can indicate the actual congestion situation of the network at the starting time point of time t. For example, at least one sixth indicator can indicate any one or more of the following information: the actual usage of the congestion queue at the starting time point of time t, the actual value corresponding to the RTT of all packets transmitted within time t-1, or other information that can reflect the degree of congestion of the network, etc.; this is not exhaustive.

[0138] The "actual usage of the congestion queue at the starting time point of time t" can be reflected by any one or more of the following sixth indicators: the actual number of data transmission units to be transmitted in the congestion queue, the ratio between the actual number and the total length of the congestion queue, or other indicators.

[0139] The "actual value corresponding to the RTT of all messages transmitted within time t-1" can be reflected by any one or more of the following sixth indicators: the average value, median value, maximum value or other indicators corresponding to the RTT of all messages transmitted within time t-1.

[0140] Optionally, the higher the score value corresponding to each sixth indicator (or "the actual number of messages successfully sent within time t-1"), the better the performance of using the second control information in the evaluation dimension of this indicator; the higher the score value corresponding to each sixth indicator (or "the actual number of messages successfully sent within time t-1"), the worse the performance of using the second control information in the evaluation dimension of this indicator.

[0141] Then in step 301, the first device can determine the score value corresponding to the parameter value of each sixth indicator based on the first status information and / or the actual number of messages successfully sent within time t-1, and / or the score value corresponding to the actual number of messages successfully sent within time t-1, thereby obtaining the actual performance information of the second control information.

[0142] Alternatively, the third score may be a fourth comprehensive score obtained by weighted summing the score corresponding to each sixth indicator and / or the score corresponding to the actual number of messages successfully sent at time t-1. Then, in step 301, after determining the score corresponding to the parameter value of each sixth indicator and / or the score corresponding to the actual number of messages successfully sent at time t-1, the first device may generate the fourth comprehensive score, thereby obtaining the actual performance information of the second control information.

[0143] 302. The first device inputs the second state information and the second control information into N second machine learning models corresponding one-to-one to the N second parameter values ​​to obtain N second prediction information of the second control information, where the second prediction information represents the pros and cons of adopting the second control information in the network indicated by the second state information, and N is an integer greater than or equal to 1.

[0144] In the embodiment of the present application, step 302 is an optional step. N second parameter values ​​are deployed in the first device, each second parameter value includes a parameter value adopted by a weight parameter of the machine learning model at one of the N moments before the moment t, where N is an integer greater than or equal to 1; when the weight parameter of the machine learning model adopts a second parameter value, it can be referred to as a second machine learning model, and there are N second machine learning models corresponding one-to-one to the N second parameter values.

[0145] The first device can input the second state information and the second control information into each second machine learning model of N second machine learning models to obtain a second prediction information output by each second machine learning model, that is, obtain N second prediction information output by N second machine learning models, and each second prediction information in the N second prediction information represents the pros and cons of adopting the second control information in the network indicated by the second state information; wherein, the specific expression form of the "second prediction information" is similar to the specific expression form of the "first prediction information", and you can refer to the above description for understanding, and will not repeat it here.

[0146] 303. The first device updates the third parameter value to obtain the first parameter value based on the N second prediction information and actual performance information. The third parameter value is one of the N second parameter values. When the weight parameter of the machine learning model adopts the first parameter value, it is the first machine learning model. The first machine learning model is used to process the first state information at time t.

[0147] In the embodiment of the present application, step 303 is an optional step. Optionally, N is an integer greater than or equal to 3. For example, the value of N can be 3, 4, 5, or other values.

[0148] Exemplarily, the third parameter value can be a set of parameter values ​​adopted by the machine learning model when determining the second control information adopted at time t-1; or, the third parameter value can also be any one of the N second parameter values. The specific "third parameter value" can be flexibly determined in combination with the actual application scenario, and is not limited in the embodiments of the present application.

[0149] Exemplarily, the first parameter value is obtained by updating the third parameter value based on the second state information at time t-1, the second control information used at time t-1, and the actual performance information of the second control information.

[0150] Exemplarily, step 303 may include: the first device may calculate the function value of the first loss function based on N second prediction information corresponding to the second control information and the actual performance information of the second control information, and use the back propagation algorithm to perform gradient update on the third parameter value to obtain the first parameter value based on the function value of the first loss function.

[0151] For example, when each second prediction information is expressed as a comprehensive score value, the first device can obtain the distance between each second prediction information in the N second prediction information and the actual performance information, that is, obtain N distances corresponding one-to-one to the N second prediction information; obtain the weight value corresponding to each second prediction information in the N second prediction information, and perform weighted summation on the N distances according to the weight value of each second prediction information to obtain the sum result, that is, obtain the function value of the first loss function; wherein the first loss function indicates the similarity between the N second prediction information and the actual performance information, and the goal of training using the first loss function includes improving the similarity between the N second prediction information and the actual performance information.

[0152] To further understand the present solution, an example of a formula used in the process of determining the first parameter value is disclosed below:

[0153] Among them, L1 represents the first loss function, θ N Including N second machine learning models corresponding to the N moments before time t; (θ N [s t ;a t ]-y t ) 2represents calculating the square of the difference between each second prediction information in the N second prediction information and the actual performance information (that is, an example of the distance between each second prediction information and the actual performance information); represents N weights corresponding to the N second prediction information one by one, Represents the weighted summation of the N distances corresponding to the N second prediction information, that is, the function value of the first loss function is obtained; argmin θ The purpose of updating the weight parameters of the machine learning model includes reducing the value of the first loss function. It should be understood that the example in formula (2) is only for the convenience of understanding this solution and is not used to limit this solution.

[0154] Exemplarily, in one case, the weight values ​​of the N second prediction information decrease successively, that is, the weight value of the second prediction information corresponding to time t-1 is the largest, and the weight value of the second prediction information corresponding to time tN is the smallest.

[0155] In another case, the weight value corresponding to each of the N second prediction information is obtained based on the first state information at time t, the N state information at N moments before time t, and a regression algorithm, and the weight value corresponding to each of the second prediction information is consistent with the weight value corresponding to each of the N state information. Exemplarily, multiple groups of weight values ​​are deployed in the first device, each group of weight values ​​includes N weight values ​​corresponding one-to-one to the N second prediction information. When the information obtained by weighted summing the N state information using a certain group of weight values ​​(hereinafter referred to as the "target group" for convenience of description) is most similar to the first state information, the target group weight value is determined as the weight of the N second prediction information.

[0156] It should be noted that the example of "the weight value of each second prediction information" here is only to prove the feasibility of this solution. The specific implementation method can be flexibly determined in combination with the actual application scenario and is not limited here.

[0157] For a more intuitive understanding of this solution, please refer to Figure 4, which is a schematic diagram of determining the first parameter value provided in an embodiment of the present application. As shown in Figure 4, taking the value of N as 3 as an example, the three second machine learning models include a second machine learning model 1 corresponding to time t-1, a second machine learning model 2 corresponding to time t-2, and a second machine learning model 3 corresponding to time t-3. The first device inputs the second state information and the second control information into the second machine learning model 1, the second machine learning model 2, and the second machine learning model 3 to obtain the second prediction information 1 output by the second machine learning model 1, the second prediction information 2 output by the second machine learning model 2, and the second prediction information 3 output by the second machine learning model 3.

[0158] According to the second prediction information 1, the second prediction information 2, the second prediction information 3 and the actual performance information, the function value of the first loss function is generated; based on the function value of the first loss function, the value of the weight parameter in the second machine learning model 1 (that is, the third parameter value) is updated to obtain the first parameter value, that is, the first machine learning model adopted at time t is obtained. It should be understood that the example in Figure 4 is only for the convenience of understanding this solution and is not used to limit this solution.

[0159] In an embodiment of the present application, the pros and cons of adopting the second control information are predicted respectively based on the N second machine learning models before time t (that is, N second prediction information), and then the third parameter value is updated based on the actual performance information of the N second prediction information and the second control information, so as to obtain the first machine learning model adopted at time t, that is, the process of determining the value of the weight parameter of the first machine learning model is combined with the second machine learning model adopted at N moments before time t, and the second machine learning model adopted at N moments before time t has seen N types of congestion conditions of the network, which is conducive to obtaining a first machine learning model with better performance, and then conducive to obtaining better target control information, so as to achieve the improvement of the utilization rate of network communication resources while minimizing network congestion.

[0160] In the embodiment of the present application, steps 302 and 303 are both optional steps. If steps 302 and 303 are not performed, the first device may further input the second state information and the second control information into the target machine learning model to obtain a second prediction information output by the target machine learning model; wherein the target machine learning model is one of N second machine learning models; among the N state information processed by the N second machine learning models, the state information processed by the target machine learning model has the greatest similarity to the first state information, and the third parameter value is a set of parameter values ​​used by the weight parameter of the target machine learning model.

[0161] The first device can generate a function value of a second loss function based on a second prediction information and actual performance information output by the target machine learning model, and use the backpropagation algorithm to update the value of the weight parameter of the target machine learning model (that is, the third parameter value) to obtain the first parameter value, that is, the first machine learning model.

[0162] In an embodiment of the present application, the first machine learning model adopted at time t is determined based on the historical data at time t-1 and the second machine learning model adopted N times before time t. That is, the first parameter value adopted by the first machine learning model is updated in a timely manner using the latest historical data, which is conducive to improving the performance of the first machine learning model and thus helping to obtain better target control information.

[0163] 304. The first device obtains first status information, where the first status information indicates a congestion condition of the network.

[0164] 305. The first device obtains first prediction information of first control information, wherein the first control information is any one in a control information set, the control information set includes multiple control information, each control information in the multiple control information indicates a data sending rate, and the first prediction information of the first control information indicates the pros and cons of using the first control information under the network state indicated by the first state information.

[0165] In an embodiment of the present application, after the first device determines the first parameter value to be used when processing the first state information at time t through steps 301 to 303, it also determines the first machine learning model to be used when processing the first state information at time t; in step 305, the first state information and the first control information can be input into the first machine learning model to obtain the first prediction information of the first control information output by the first machine learning model; the detailed meaning of the "first prediction information of the first control information" can be found in the description in the embodiment corresponding to Figure 2 above, and will not be repeated here.

[0166] The first device repeatedly executes step 305 multiple times to obtain first prediction information for each control information in the control information set.

[0167] 306. The first device selects second control information from the control information set according to the first prediction information of each control information.

[0168] In the embodiment of the present application, the specific implementation of steps 304 and 306 can be found in the description of the corresponding embodiment in Figure 2 and will not be repeated here.

[0169] (2) Modeling a first distribution space using a machine learning model. The first distribution space represents a distribution space where at least one first information pair is located. Each first information pair may include a combination information and predicted state information corresponding to the combination information.

[0170] Specifically, please refer to FIG5 , which is another flowchart of a data processing method provided in an embodiment of the present application. As shown in FIG5 , the data processing method provided in the present application may include:

[0171] 501. A first device obtains second state information at time t-1 and second control information used at time t-1, where the state information at time t-1 indicates a congestion condition of the network at time t-1.

[0172] In the embodiment of the present application, step 501 and subsequent step 502 are optional steps. The specific implementation of step 501 and the meaning of the terms in step 501 can be found in the description of the embodiment corresponding to FIG3 and will not be repeated here.

[0173] 502. The first device updates the posterior distribution of the third machine learning model based on the state information at time t-1, the second control information, and the first state information to obtain an updated third machine learning model.

[0174] For the sake of convenience in distinction, in the embodiment of the present application, the machine learning model for modeling the first distribution space is referred to as the third machine learning model. The third machine learning model can adopt a Gaussian model, or other models for describing the distribution space, etc., which is not limited in the embodiment of the present application.

[0175] Exemplarily, the state information at time t-1, the second control information, and the first state information constitute a first information pair; the first device can use the maximum a posteriori probability algorithm to update the posterior distribution of the third machine learning model based on the state information at time t-1, the second control information, and the first state information to obtain an updated third machine learning model.

[0176] 503. The first device obtains first status information, where the first status information indicates a congestion condition of the network.

[0177] 504. The first device obtains first prediction information of first control information, wherein the first control information is any one in a control information set, the control information set includes multiple control information, each control information in the multiple control information indicates a data sending rate, and the first prediction information of the first control information indicates the pros and cons of using the first control information under the network state indicated by the first state information.

[0178] In the embodiment of the present application, the first state information and the first control information constitute a first combination of information, and the second combination of information is a combination of information;

[0179] Since steps 501 and 502 are optional steps, if steps 501 and 502 are executed, the first device can obtain a second combination information that is most similar to the first combination information from the first distribution space indicated by the updated third machine learning model; if steps 501 and 502 are not executed, the first device can obtain a second combination information that is most similar to the first combination information from the first distribution space indicated by the third machine learning model.

[0180] The first device determines the predicted state information corresponding to the second combined information as the predicted state information corresponding to the first combined information, and determines the first predicted information of the first control information based on the predicted state information corresponding to the first combined information (that is, the predicted state information used to indicate the congestion status of the network at the starting time point t+1).

[0181] Among them, the specific implementation method of the above steps can be understood in combination with the description in the corresponding embodiment of Figure 2. The relationship between "the predicted status information indicating the congestion situation of the network at the starting time point of time t+1" and "the first prediction information of the first control information" can refer to the description in the corresponding embodiment of Figure 2, and will not be repeated here.

[0182] The first device repeatedly performs step 504 multiple times until obtaining first prediction information for each control information in the control information set.

[0183] 505. The first device determines target control information from the control information set according to the first prediction information of each control information.

[0184] In the embodiment of the present application, the specific implementation method of steps 504 and 505 can refer to the description in the embodiment corresponding to Figure 2, which will not be repeated here. After obtaining the historical data at time t-1, the historical data at time t-1 will be used in a timely manner to update the posterior distribution of the third machine learning model, which is conducive to improving the similarity between the first distribution space indicated by the third machine learning model and the real distribution space where at least one information pair is located, that is, it is conducive to modeling the real distribution space where at least one information pair is located through the third machine learning model, and it is conducive to obtaining more accurate prediction state information with the help of the updated third machine learning model, and then it is conducive to obtaining more accurate first prediction information, so as to facilitate the selection of better target control information.

[0185] 2. The first device only performs the steps of the inference phase of the machine learning model, and the second device updates the parameters of the machine learning model

[0186] In the embodiment of the present application, referring to the description in the embodiment corresponding to Figure 2 above, it can be seen that there may be two situations for the object modeled by the machine learning model. The above two situations are described below respectively.

[0187] (1) Using machine learning models to model the mapping relationship between combined information and the predicted performance information of the aforementioned combined information

[0188] Specifically, please refer to FIG6 , which is another flowchart of the data processing method provided in an embodiment of the present application. As shown in FIG3 , the data processing method provided in the present application may include:

[0189] 601. The second device obtains a first sample, where the first sample includes fourth state information, fourth control information, and actual performance information of the fourth control information. The first state information indicates network congestion, and the actual performance information of the fourth control information indicates the pros and cons of using the fourth control information under the network state indicated by the fourth state information.

[0190] 602. The second device inputs the fourth state information and the fourth control information into the first machine learning model to obtain predicted performance information of the fourth control information output by the first machine learning model.

[0191] In an embodiment of the present application, in the process of executing the steps of the inference phase using the first machine learning model, multiple samples for training can be obtained. In step 601, multiple samples for training can be deployed in the second device, and the second device can obtain one or more first samples; wherein, the meaning of "fourth state information" is similar to the meaning of "second state information", the meaning of "fourth control information" is similar to the meaning of "second control information", and the meaning of "actual performance information of the fourth control information" is similar to the meaning of "actual performance information of the second control information", all of which can be found in the description of step 301 in the embodiment corresponding to FIG3. The specific implementation of steps 601 and 602 can also be found in the description of steps 301 and 302 in the embodiment corresponding to FIG3, which will not be repeated here.

[0192] 603. The second device updates the first machine learning model using a third loss function based on the actual performance information of the fourth control information and the predicted performance information of the fourth control information, where the third loss function indicates the similarity between the actual performance information of the fourth control information and the predicted performance information of the fourth control information.

[0193] In the embodiment of the present application, steps 601 to 603 are optional steps. For example, step 603 may include: the second device generating a function value of a third loss function based on the actual performance information of the fourth control information and the predicted performance information of the fourth control information; and using a backpropagation algorithm to perform a gradient update on the values ​​of the weight parameters of the first machine learning model based on the function value of the third loss function, thereby achieving a single training of the first machine learning model.

[0194] The second device can repeat steps 601 to 603 multiple times until the convergence condition is met, and the first machine learning model that has performed the training operation is obtained. The first machine learning model that has performed the training operation will be deployed to the first device; exemplarily, the convergence condition may include satisfying the convergence condition of the third loss function and / or the number of training times reaches a preset number.

[0195] 604. The first device obtains first status information, where the first status information indicates a congestion condition of the network.

[0196] 605. The first device obtains first prediction information of first control information, wherein the first control information is any one in a control information set, the control information set includes multiple control information, each control information in the multiple control information indicates a data sending rate, and the first prediction information of the first control information indicates the pros and cons of using the first control information under the network state indicated by the first state information.

[0197] 606. The first device selects second control information from the control information set according to the first prediction information of each control information.

[0198] In the embodiment of the present application, the specific implementation of steps 604 to 606 can refer to the description of steps 304 to 306 in the embodiment corresponding to Figure 3, and will not be repeated in the embodiment of the present application.

[0199] It should be noted that if steps 601 to 603 are executed, steps 601 to 603 can be executed again after executing steps 604 to 606 multiple times to update the values ​​of the weight parameters of the first machine learning model again, thereby obtaining the first machine learning model that has been trained again, and deploying the first machine learning model that has been trained again to the first device to execute steps 604 to 606 using the first machine learning model that has been trained again; the aforementioned cycle can be repeated during actual application.

[0200] (2) Modeling a first distribution space using a machine learning model. The first distribution space represents a distribution space where at least one first information pair is located. Each first information pair may include a combination information and predicted state information corresponding to the combination information.

[0201] Specifically, please refer to FIG7 , which is another flowchart of a data processing method provided in an embodiment of the present application. As shown in FIG7 , the data processing method provided in the present application may include:

[0202] 701. The second device obtains at least one second information pair, where the second information pair includes third combination information and status information corresponding to the third combination information, the third combination information includes fifth status information and fifth control information, and the status information corresponding to the third combination information indicates a congestion condition of the network at a next moment when the fifth control information is adopted in the network state indicated by the fifth status information.

[0203] 702. The second device updates the posterior distribution of the third machine learning model based on at least one second information pair.

[0204] In the embodiment of the present application, steps 701 and 702 are optional steps. For example, during the process of executing the steps of the inference phase using the third machine learning model, one or more second information pairs may be obtained. The meaning of "second information pair" is similar to the meaning of "first information pair", "third combination information" is a combination information, and the meaning of "state information corresponding to the third combination information" can be understood by referring to the above description of "state information corresponding to the combination information", which will not be repeated here.

[0205] Exemplarily, after obtaining at least one second information pair, the second device can update the posterior distribution of the third machine learning model in sequence based on each second information pair, that is, use at least one second information pair to perform at least one update operation on the posterior distribution of the third machine learning model to obtain an updated third machine learning model.

[0206] 703. The first device obtains first status information, where the first status information indicates a congestion condition of the network.

[0207] 704. The first device obtains first prediction information of first control information, wherein the first control information is any one in a control information set, the control information set includes multiple control information, each control information in the multiple control information indicates a data sending rate, and the first prediction information of the first control information indicates the pros and cons of using the first control information under the network state indicated by the first state information.

[0208] 705. The first device selects second control information from the control information set according to the first prediction information of each control information.

[0209] In the embodiment of the present application, the specific implementation method of steps 703 to 705 can refer to the description of steps 303 to 305 in the embodiment corresponding to Figure 3, and will not be repeated in the embodiment of the present application.

[0210] It should be noted that steps 701 and 702 are optional steps. If steps 701 and 702 are executed, steps 701 and 702 can be executed again after executing steps 703 to 705 multiple times to update the values ​​of the weight parameters of the third machine learning model again, thereby obtaining a third machine learning model that has been trained again, and deploying the third machine learning model that has been trained again to a third device to execute steps 703 to 705 using the third machine learning model that has been trained again. The aforementioned cycle can be repeated during actual application.

[0211] On the basis of the embodiments corresponding to Figures 1 to 7, in order to better implement the above-mentioned scheme of the embodiments of the present application, the following also provides related devices for implementing the above-mentioned scheme. Please refer to Figure 8 in detail. Figure 8 is a structural schematic diagram of a data processing device provided in an embodiment of the present application. The data processing device 800 is applied to the first device. The data processing device 800 includes: an acquisition module 801, which is used to obtain first status information, and the first status information indicates the congestion status of the network; the acquisition module 801 is also used to obtain first prediction information of the first control information, wherein the first control information is any one of the control information set, and the control information set includes multiple control information, each of the multiple control information indicates the data transmission rate, and the first prediction information of the first control information indicates the pros and cons of using the first control information under the network status indicated by the first status information; the determination module 802 is used to determine the target control information from the control information set according to the first prediction information of each control information.

[0212] Optionally, the acquisition module 801 is specifically used to input the first state information and the first control information into the first machine learning model to obtain the first prediction information output by the first machine learning model.

[0213] Optionally, the first status information indicates the congestion status of the network at time t. When the weight parameter of the machine learning model adopts the first parameter value, it is the first machine learning model, and t is an integer greater than or equal to 11; the data processing device 800 is applied to the first device, and N second parameter values ​​are deployed in the first device, each second parameter value includes the parameter value adopted by the weight parameter of the machine learning model at one of the N moments before time t, N is an integer greater than or equal to 11, and the first parameter value is obtained by updating the third parameter value based on the second status information at time t-11, the second control information adopted at time t-11, and the actual performance information of the second control information; wherein the third parameter value is one of the N second parameter values, the status information at time t-11 indicates the congestion status of the network at time t-11, the second control information indicates the data transmission rate in the network at time t-11, and the actual performance information of the second control information represents the pros and cons of adopting the second control information in the network at time t-11.

[0214] Optionally, when the weight parameter of the machine learning model adopts the second parameter value, it is a second machine learning model, and the data processing device also includes: an input module 803, which is used to input the second state information and the second control information into N second machine learning models corresponding to the N second parameter values ​​one by one, to obtain N second prediction information of the second control information, and the second prediction information represents the pros and cons of adopting the second control information in the network indicated by the second state information; an update module 804, which is used to update the third parameter value according to the N second prediction information and actual performance information to obtain the first parameter value.

[0215] Optionally, the first status information indicates the congestion status of the network at time t, where t is an integer greater than or equal to 11. The acquisition module 801 is specifically used to: input the first status information and the first control information into the third machine learning model to obtain third prediction information output by the third machine learning model, the third prediction information indicating the congestion status of the network at time t+11 when the first control information is adopted in the network state indicated by the first status information; and determine the first prediction information based on the third prediction information.

[0216] Optionally, the first distribution space is modeled by a third machine learning model, and the first distribution space represents the distribution space where at least one information pair is located. Each information pair includes a combination information and predicted state information corresponding to the combination information. A combination information includes a third state information and a third control information. The third state information indicates the congestion of the network. The third control information is any one in the control information set. The state information corresponding to the combination information indicates the congestion of the network at the next moment when the third control information is adopted in the network state indicated by the third state information.

[0217] Optionally, the acquisition module 801 is also used to obtain the second state information at time t-11 and the second control information used at time t-11, where the state information at time t-11 indicates the congestion status of the network at time t-11; the data processing device 800 also includes: an update module 804, used to update the posterior distribution of the third machine learning model according to the state information at time t-11, the second control information and the first state information to obtain an updated third machine learning model; the acquisition module 801 is specifically used to obtain third prediction information from the distribution space indicated by the updated third machine learning model according to the first state information and the first control information.

[0218] Optionally, the first state information indicates the congestion situation of the network at time t, and the first prediction information of the first state information indicates the difference between the predicted state information and the expected state information of the network at time t+11 when the first control information is adopted in the network state indicated by the first state information; wherein, the predicted state information of the network at time t+11 represents the predicted congestion situation of the network at time t+11, and the expected state information represents the expected congestion situation of the network.

[0219] Optionally, the determination module 802 is specifically used to determine the target control information from the control information set based on the first prediction information of each control information and the uncertainty corresponding to each control information, and the uncertainty corresponding to each control information represents the uncertainty of the combination of each control information and the first state information.

[0220] Optionally, each piece of control information includes a value of a receiving window RWND and / or a value of a sending window SWND.

[0221] It should be noted that the information interaction, execution process, etc. between the modules / units in the data processing device 800 are based on the same concept as the various method embodiments corresponding to Figures 2 to 7 in this application. For specific contents, please refer to the description in the method embodiments shown above in this application, and will not be repeated here.

[0222] Please refer to Figure 9, which is a schematic diagram of the structure of a device provided in an embodiment of the present application. When the first device or the second device involved in the above embodiment is specifically a network device, it can be implemented using the device 900 shown in Figure 9; illustratively, when the first device (or second device) is a router, firewall, gateway, or other transit device, it can be implemented using the device 900 shown in Figure 9. Alternatively, when the second device is a server or server cluster, it can be implemented using the device 900 shown in Figure 9.

[0223] The structure of the communication device can refer to the structure shown in Figure 9. The device 900 includes at least one processor 911 and at least one network interface 912. Further optionally, the communication device also includes at least one memory 914, at least one transceiver 913 and one or more antennas 915. The processor 911, the transceiver 913, the memory 914 and the network interface 912 are connected, for example, via a bus. In an embodiment of the present application, the connection may include various interfaces, transmission lines or buses, etc., which are not limited in this embodiment. The antenna 915 is connected to the transceiver 913. The network interface 912 is used to enable the communication device to communicate with other communication devices through a communication link. For example, the network interface 912 may include a network interface between the communication device and the core network device, such as an S1 interface, and the network interface may include a network interface between the communication device and other communication devices (such as other network devices or core network devices), such as an X2 or Xn interface.

[0224] Processor 911 is primarily used to process communication protocols and communication data, control the entire communication device, execute software programs, and process software program data, for example, to support the communication device in performing the actions described in the embodiments. A communication device may include a baseband processor and a central processing unit. The baseband processor is primarily used to process communication protocols and communication data, while the central processing unit is primarily used to control the entire terminal device, execute software programs, and process software program data. Processor 911 in Figure 9 may integrate the functions of both a baseband processor and a central processing unit. Those skilled in the art will appreciate that the baseband processor and the central processing unit may also be independent processors interconnected via a bus or other technology. Those skilled in the art will appreciate that a terminal device may include multiple baseband processors to accommodate different network standards, multiple central processing units to enhance its processing capabilities, and various components of the terminal device may be connected via various buses. The baseband processor may also be referred to as a baseband processing circuit or a baseband processing chip. The central processing unit may also be referred to as a central processing circuit or a central processing chip. The functionality for processing communication protocols and communication data may be built into the processor or stored in memory as a software program, which is executed by the processor to implement the baseband processing functionality.

[0225] The memory is primarily used to store software programs and data. Memory 914 can exist independently and be connected to processor 911. Alternatively, memory 914 and processor 911 can be integrated together, for example, within a single chip. Memory 914 can store program code for executing the technical solutions of the embodiments of the present application, and execution is controlled by processor 911. The various computer program codes executed can also be considered drivers for processor 911.

[0226] Figure 9 shows only one memory and one processor. In an actual network device, multiple processors and multiple memories may exist. Memory may also be referred to as a storage medium or storage device. The memory may be a storage element on the same chip as the processor, i.e., an on-chip storage element, or an independent storage element, which is not limited in the present embodiment.

[0227] The transceiver 913 can be used to support the reception or transmission of radio frequency signals between the communication device and the terminal, and the transceiver 913 can be connected to the antenna 915. The transceiver 913 includes a transmitter Tx and a receiver Rx. Specifically, one or more antennas 915 can receive radio frequency signals. The receiver Rx of the transceiver 913 is used to receive the radio frequency signal from the antenna, convert the radio frequency signal into a digital baseband signal or a digital intermediate frequency signal, and provide the digital baseband signal or digital intermediate frequency signal to the processor 911 so that the processor 911 can further process the digital baseband signal or digital intermediate frequency signal, such as demodulation and decoding. In addition, the transmitter Tx in the transceiver 913 is also used to receive a modulated digital baseband signal or digital intermediate frequency signal from the processor 911, convert the modulated digital baseband signal or digital intermediate frequency signal into a radio frequency signal, and transmit the radio frequency signal through one or more antennas 915. Specifically, the receiver Rx can selectively perform one or more stages of down-mixing and analog-to-digital conversion on the RF signal to obtain a digital baseband signal or a digital intermediate frequency signal. The order of the down-mixing and analog-to-digital conversion processes is adjustable. The transmitter Tx can selectively perform one or more stages of up-mixing and digital-to-analog conversion on the modulated digital baseband signal or digital intermediate frequency signal to obtain a RF signal. The order of the up-mixing and digital-to-analog conversion processes is adjustable. The digital baseband signal and the digital intermediate frequency signal may be collectively referred to as digital signals.

[0228] The transceiver 913 may also be referred to as a transceiver unit, a transceiver, a transceiver device, etc. Optionally, a device in the transceiver unit that implements a receiving function may be referred to as a receiving unit, and a device in the transceiver unit that implements a transmitting function may be referred to as a transmitting unit. That is, the transceiver unit includes a receiving unit and a transmitting unit. The receiving unit may also be referred to as a receiver, an input port, a receiving circuit, etc., and the transmitting unit may be referred to as a transmitter, a transmitter, or a transmitting circuit, etc.

[0229] Exemplarily, the device 900 shown in FIG9 can be used to implement the steps implemented by the first device (or second device) in the method embodiments corresponding to FIG2 to FIG7 above. The specific implementation methods of the device 900 shown in FIG9 performing the aforementioned steps can refer to the description in the aforementioned method embodiments and will not be repeated here one by one.

[0230] A computer-readable storage medium is also provided in an embodiment of the present application. The computer-readable storage medium stores a program for signal processing. When the computer-readable storage medium is run on a computer, the computer executes the steps executed by the first device in the method described in the embodiments shown in Figures 2 to 7 above, or the computer executes the steps executed by the second device in the method described in the embodiments shown in Figures 2 to 7 above.

[0231] An embodiment of the present application also provides a computer program product, which, when running on a computer, enables the computer to execute the steps executed by the first device in the method described in the embodiments shown in Figures 2 to 7 above, or enables the computer to execute the steps executed by the second device in the method described in the embodiments shown in Figures 2 to 7 above.

[0232] The first device or the second device provided in the embodiments of the present application may specifically be a chip, and the chip includes: a processing unit and a communication unit. The processing unit may be, for example, a processor, and the communication unit may be, for example, an input / output interface, a pin, or a circuit. The processing unit may execute computer-executable instructions stored in the storage unit so that the chip performs the data processing method described in the embodiments shown in Figures 2 to 7 above. Optionally, the storage unit is a storage unit within the chip, such as a register, a cache, etc. The storage unit may also be a storage unit located outside the chip within the wireless access device, such as a read-only memory (ROM) or other type of static storage device that can store static information and instructions, a random access memory (RAM), etc.

[0233] Specifically, see Figure 10 , which is a schematic diagram of the structure of a chip provided in an embodiment of the present application. The chip can be represented as a neural network processor NPU 100. NPU 100 is mounted on a host CPU (host CPU) as a coprocessor, and the host CPU assigns tasks. The core of the NPU is arithmetic circuit 1003, which is controlled by controller 1004 to extract matrix data from memory and perform multiplication operations.

[0234] In some implementations, the arithmetic circuit 1003 includes multiple processing units (PEs). In some implementations, the arithmetic circuit 1003 is a two-dimensional systolic array. The arithmetic circuit 1003 can also be a one-dimensional systolic array or other electronic circuit capable of performing mathematical operations such as multiplication and addition. In some implementations, the arithmetic circuit 1003 is a general-purpose matrix processor.

[0235] For example, assume there are input matrix A, weight matrix B, and output matrix C. The arithmetic circuit retrieves the corresponding data of matrix B from weight memory 1002 and caches it on each PE in the arithmetic circuit. The arithmetic circuit retrieves the data of matrix A from input memory 1001 and performs a matrix operation on matrix B. The partial or final matrix result is stored in accumulator 1008.

[0236] Unified memory 1006 is used to store input and output data. Weight data is directly transferred to weight memory 1002 through the Direct Memory Access Controller (DMAC) 1005. Input data is also transferred to unified memory 1006 through the DMAC.

[0237] BIU stands for Bus Interface Unit, i.e., bus interface unit 1010 , which is used for interaction between the AXI bus, DMAC, and instruction fetch buffer (IFB) 1009 .

[0238] The bus interface unit 1010 (BIU) is used for the instruction fetch memory 1009 to obtain instructions from the external memory, and is also used for the storage unit access controller 1005 to obtain the original data of the input matrix A or the weight matrix B from the external memory.

[0239] DMAC is mainly used to transfer input data in the external memory DDR to the unified memory 1006 or transfer weight data to the weight memory 1002 or transfer input data to the input memory 1001.

[0240] The vector calculation unit 1007 includes multiple operation processing units. When necessary, it further processes the output of the operation circuit, such as vector multiplication, vector addition, exponential operation, logarithmic operation, size comparison, etc. It is mainly used for non-convolutional / fully connected layer network calculations in neural networks, such as batch normalization, pixel-level summation, and upsampling of feature planes.

[0241] In some implementations, the vector calculation unit 1007 can store the processed output vector to the unified memory 1006. For example, the vector calculation unit 1007 can apply a linear function and / or a nonlinear function to the output of the operation circuit 1003, such as linear interpolation of the feature plane extracted by the convolution layer, or accumulate a vector of values ​​to generate an activation value. In some implementations, the vector calculation unit 1007 generates a normalized value, a pixel-level summed value, or both. In some implementations, the processed output vector can be used as an activation input to the operation circuit 1003, for example, for use in a subsequent layer in a neural network.

[0242] An instruction fetch buffer 1009 connected to the controller 1004 is used to store instructions used by the controller 1004;

[0243] Unified memory 1006, input memory 1001, weight memory 1002, and instruction fetch memory 1009 are all on-chip memories. External memories are private to the NPU hardware architecture.

[0244] In which, the operations of each layer in the first machine learning model, the second machine learning model and / or the third machine learning model can be performed by the operation circuit 1003 or the vector calculation unit 1007.

[0245] The processor mentioned in any of the above places can be a general-purpose central processing unit, a microprocessor, an ASIC, or one or more integrated circuits for controlling the execution of the program of the above-mentioned first aspect method.

[0246] It should also be noted that the device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment. In addition, in the drawings of the device embodiments provided in this application, the connection relationship between the modules indicates that there is a communication connection between them, which can be specifically implemented as one or more communication buses or signal lines.

[0247] Through the description of the above embodiments, those skilled in the art can clearly understand that the present application can be implemented by means of software plus necessary general hardware, and of course can also be implemented by special hardware including application-specific integrated circuits, special CPUs, special memories, special components, etc. In general, all functions performed by computer programs can be easily implemented with corresponding hardware, and the specific hardware structures used to implement the same function can also be diverse, such as analog circuits, digital circuits or special circuits, etc. However, for the present application, software program implementation is a better implementation method in most cases. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer's floppy disk, USB flash drive, mobile hard disk, ROM, RAM, magnetic disk or optical disk, etc., and includes a number of instructions to enable a computer device (which can be a personal computer, training equipment, or network equipment, etc.) to execute the methods described in each embodiment of the present application.

[0248] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware, or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0249] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website, a computer, a training device or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website, a computer, a training device or a data center. The computer-readable storage medium can be any available medium that a computer can store or a data storage device such as a training device, a data center, etc. that includes one or more available media integrations. The available medium can be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)).

Claims

1. A data processing method, characterized in that: The method comprises: Acquire first status information, where the first status information indicates a congestion condition of a network; Acquire first prediction information of first control information, wherein the first control information is any one of a control information set, the control information set includes multiple control information, each of the multiple control information indicates a data transmission rate, and the first prediction information of the first control information indicates the pros and cons of using the first control information under the network state indicated by the first state information; According to the first prediction information of each control information, target control information is determined from the control information set.

2. The method according to claim 1, characterized in that The acquiring first prediction information of the first control information includes: The first state information and the first control information are input into a first machine learning model to obtain the first prediction information output by the first machine learning model.

3. The method according to claim 2, characterized in that The first state information indicates the congestion status of the network at time t, when the weight parameter of the machine learning model adopts the first parameter value, it is the first machine learning model, and t is an integer greater than or equal to 1; The method is applied to a first device, wherein N second parameter values ​​are deployed in the first device, each of the second parameter values ​​includes a parameter value adopted by a weight parameter of the machine learning model at one of N moments before the moment t, N is an integer greater than or equal to 1, and the first parameter value is obtained by updating a third parameter value based on second state information at moment t-1, second control information adopted at moment t-1, and actual performance information of the second control information; Among them, the third parameter value is one of the N second parameter values, the status information at time t-1 indicates the congestion situation of the network at time t-1, the second control information indicates the data sending rate in the network at time t-1, and the actual performance information of the second control information represents the advantages and disadvantages of using the second control information in the network at time t-1.

4. The method according to claim 3, characterized in that When the weight parameter of the machine learning model adopts the second parameter value, it is a second machine learning model, and the method further includes: Inputting the second state information and the second control information into N second machine learning models corresponding to the N second parameter values ​​one by one, to obtain N second prediction information of the second control information, where the second prediction information represents the pros and cons of adopting the second control information in the network indicated by the second state information; According to the N second prediction information and the actual performance information, the third parameter value is updated to obtain the first parameter value.

5. The method according to claim 1, characterized in that The first state information indicates the congestion status of the network at time t, where t is an integer greater than or equal to 1, and the acquiring the first prediction information of the first control information includes: Inputting the first state information and the first control information into a third machine learning model to obtain third prediction information output by the third machine learning model, wherein the third prediction information indicates a congestion condition of the network at time t+1 when the first control information is used under the network state indicated by the first state information; The first prediction information is determined according to the third prediction information.

6. The method according to claim 5, characterized in that The first distribution space is modeled by the third machine learning model, and the first distribution space represents the distribution space where at least one information pair is located, each of the information pairs includes a combination information and predicted state information corresponding to the combination information, one combination information includes a third state information and a third control information, the third state information indicates the congestion condition of the network, the third control information is any one of the control information set, and the state information corresponding to the combination information indicates the congestion condition of the network at the next moment when the third control information is adopted under the network state indicated by the third state information.

7. The method according to claim 6, characterized in that The method further comprises: Acquire second state information at time t-1 and second control information used at the time t-1, wherein the state information at time t-1 indicates a congestion condition of the network at the time t-1; According to the state information at time t-1, the second control information and the first state information, the posterior distribution of the third machine learning model is updated to obtain the updated third machine learning model; The obtaining of third prediction information based on a third machine learning model according to the first state information and the first control information includes: obtaining, from the distribution space indicated by the updated third machine learning model, according to the first state information and the first control information The third prediction information.

8. The method according to any one of claims 1 to 7, characterized in that: The first state information indicates a congestion condition of the network at time t, and the first prediction information of the first state information indicates a difference between predicted state information and expected state information of the network at time t+1 when the first control information is used in the network state indicated by the first state information; The predicted state information of the network at time t+1 represents the predicted congestion situation of the network at time t+1, and the expected state information represents the expected congestion situation of the network.

9. The method according to any one of claims 1 to 7, characterized in that: The determining target control information from the control information set according to the first prediction information of each control information includes: The target control information is determined from the control information set according to the first prediction information of each control information and the uncertainty corresponding to each control information, wherein the uncertainty corresponding to each control information represents the uncertainty of a combination of each control information and the first state information.

10. The method according to any one of claims 1 to 7, characterized in that: Each of the control information includes the value of the receiving window RWND and / or the value of the sending window SWND.

11. A data processing device, characterized in that: The device comprises: An acquisition module, configured to acquire first status information, where the first status information indicates a congestion condition of a network; The acquisition module is further configured to acquire first prediction information of first control information, wherein the first control information is any one of a control information set, the control information set includes a plurality of control information, each of the plurality of control information indicates a data transmission rate, and the first prediction information of the first control information indicates the pros and cons of using the first control information under the network state indicated by the first state information; A determination module is used to determine target control information from the control information set according to the first prediction information of each control information.

12. The device according to claim 11, characterized in that The acquisition module is specifically used to input the first state information and the first control information into a first machine learning model to obtain the first prediction information output by the first machine learning model.

13. The device according to claim 12, characterized in that The first state information indicates the congestion status of the network at time t, when the weight parameter of the machine learning model adopts the first parameter value, it is the first machine learning model, and t is an integer greater than or equal to 11; The apparatus is applied to a first device, wherein N second parameter values ​​are deployed in the first device, each of the second parameter values ​​includes a parameter value adopted by a weight parameter of the machine learning model at one of N moments before the moment t, N is an integer greater than or equal to 11, and the first parameter value is obtained by updating a third parameter value based on the second state information at the moment t-11, the second control information adopted at the moment t-11, and the actual performance information of the second control information; Among them, the third parameter value is one of the N second parameter values, the status information at time t-11 indicates the congestion situation of the network at time t-11, the second control information indicates the data sending rate in the network at time t-11, and the actual performance information of the second control information represents the advantages and disadvantages of using the second control information in the network at time t-11.

14. The device according to claim 13, characterized in that When the weight parameter of the machine learning model adopts the second parameter value, it is a second machine learning model, and the device further includes: an input module, configured to input the second state information and the second control information into N second machine learning models corresponding one-to-one to the N second parameter values, to obtain N second prediction information of the second control information, wherein the second prediction information represents the pros and cons of adopting the second control information in the network indicated by the second state information; An updating module is used to update the third parameter value to obtain the first parameter value according to the N second prediction information and the actual performance information.

15. The device according to claim 11, characterized in that The first status information indicates the congestion status of the network at time t, where t is an integer greater than or equal to 11, and the acquisition module is specifically used to: Inputting the first state information and the first control information into a third machine learning model to obtain third prediction information output by the third machine learning model, wherein the third prediction information indicates a congestion condition of the network at time t+11 when the first control information is used under the network state indicated by the first state information; The first prediction information is determined according to the third prediction information.

16. The device according to claim 15, characterized in that The first distribution space is modeled by the third machine learning model, and the first distribution space represents the distribution space where at least one information pair is located, each of the information pairs includes a combination information and predicted state information corresponding to the combination information, one combination information includes a third state information and a third control information, the third state information indicates the congestion condition of the network, the third control information is any one of the control information set, and the state information corresponding to the combination information indicates the congestion condition of the network at the next moment when the third control information is adopted under the network state indicated by the third state information.

17. The device according to claim 16, characterized in that The acquisition module is further used to acquire the second state information at time t-11 and the second control information used at time t-11, wherein the state information at time t-11 indicates the congestion status of the network at time t-11; The device further includes: an updating module, configured to update the posterior distribution of the third machine learning model according to the state information at time t-11, the second control information, and the first state information, to obtain the updated third machine learning model; The acquisition module is specifically used to acquire the third prediction information from the distribution space indicated by the updated third machine learning model based on the first state information and the first control information.

18. The device according to any one of claims 11 to 17, characterized in that The first state information indicates a congestion condition of the network at time t, and the first prediction information of the first state information indicates a difference between predicted state information and expected state information of the network at time t+11 when the first control information is used in the network state indicated by the first state information; The predicted state information of the network at time t+11 represents the predicted congestion situation of the network at time t+11, and the expected state information represents the expected congestion situation of the network.

19. The device according to any one of claims 11 to 17, characterized in that The determination module is specifically used to determine the target control information from the control information set based on the first prediction information of each control information and the uncertainty corresponding to each control information, and the uncertainty corresponding to each control information represents the uncertainty of the combination of each control information and the first state information.

20. The device according to any one of claims 11 to 17, characterized in that Each of the control information includes the value of the receiving window RWND and / or the value of the sending window SWND.

21. A device, characterized in that comprising a processor and a memory, the processor being coupled to the memory, The memory is used to store programs; The processor is configured to execute the program in the memory so that the device performs the method according to any one of claims 1 to 10.

22. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a program, and when the program is executed on a computer, the computer is caused to execute the method according to any one of claims 1 to 10.

23. A computer program product, characterized in that The computer program product comprises a program, and when the program is run on a computer, the computer is caused to execute the method according to any one of claims 1 to 10 .

Images