Load flow distribution method and device

By constructing a reinforcement learning model to predict future load information and determine traffic allocation strategies, the problems of network performance losses and poor load traffic allocation effects caused by data discarding in the existing technology are solved, and balanced distribution of load traffic and improved user experience are achieved.

CN114916012BActive Publication Date: 2024-11-01CHINA TELECOM CORP LTD
View PDF 6 Cites 0 Cited by

Patent Information

Application Number
CN202110181787.6
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-02-08
Publication Date
2024-11-01
Estimated Expiration
2041-02-08

AI Technical Summary

Technical Problem

The prior art relieves network congestion through data discarding, resulting in network performance loss and inability to adapt to future load traffic changes, resulting in poor load traffic allocation effect.

Method used

By building a reinforcement learning model, predict future load information and determine traffic allocation strategies, avoid data discarding, adapt to the load traffic changes of UPF entities, and achieve balanced allocation of load traffic.

Benefits of technology

There is no need to discard data, avoid network performance losses, adapt to load traffic changes, improve load traffic allocation effects, alleviate network congestion, and improve user experience.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114916012B_ABST
    Figure CN114916012B_ABST
Patent Text Reader

Abstract

The present disclosure relates to a method and device for distributing load traffic, and to the field of communication technology. The distribution method includes: predicting user plane load information at future moments based on user plane load information at historical moments and user plane load information at current moments; constructing a reinforcement learning model for determining the traffic distribution strategy of each UPF entity based on each user plane load information; and determining the traffic distribution strategy of each UPF entity at current moments using the reinforcement learning model.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to the field of communication technology, and in particular to a load flow distribution method, a load flow distribution device and a non-volatile computer-readable storage medium. Background Art

[0002] With the development of various communication services and the increase in the number of terminal connections, the currently deployed 5G network can no longer meet the rapidly growing demand for data from terminal users. Since the network resources that network operators can provide are limited, when a large number of users transmit data through limited network resources, it is easy to cause network congestion.

[0003] Due to the complexity of 5G networks and wireless environments, network congestion can lead to a variety of performance degradations, such as increased PDCP (Packet Data Convergence Protocol) PDU (Protocol Data Unit) discard rate, increased data transmission delay, or reduced PRB (Physical Resource Block) utilization efficiency.

[0004] Therefore, how to solve network congestion and improve user experience has become a pressing issue to be addressed.

[0005] In the related art, data in the data queue of the UPF entity is processed in a "first in, first out" order; when the queue length exceeds a threshold, part of the data is discarded, thereby alleviating network congestion. Summary of the invention

[0006] The inventors of the present disclosure discovered that the above-mentioned related technologies have the following problems: alleviating network congestion by discarding data will not only cause loss of network performance, but also fail to adapt to changes in load flow of the UPF entity in the future, resulting in poor load flow distribution effect.

[0007] In view of this, the present disclosure proposes a technical solution for distributing load traffic, which can avoid network performance loss, adapt to the load traffic changes of the UPF entity at future times, improve the load traffic distribution effect, and thus solve the network congestion problem.

[0008] According to some embodiments of the present disclosure, a method for distributing load traffic is provided, including: predicting user plane load information at future moments based on user plane load information at historical moments and user plane load information at current moments; constructing a reinforcement learning model for determining the traffic distribution strategy of each UPF (User Plane Function) entity based on each user plane load information; and determining the traffic distribution strategy of each UPF entity at the current moment by using the reinforcement learning model.

[0009] In some embodiments, based on the load information of each user plane, a reinforcement learning model is constructed to determine the traffic distribution strategy of each user plane function UPF entity, including: mapping the user plane load information at the current moment and the user plane load information at the future moment to the states at the current moment and the future moment in the reinforcement learning model respectively; mapping various candidate traffic distribution strategies at the current moment to various actions in the reinforcement learning model at the current moment; based on the states and actions, constructing the load traffic distribution process between each UPF entity as a Markov process model as a reinforcement learning model.

[0010] In some embodiments, a reinforcement learning model is used to determine the traffic allocation strategy of each UPF entity at the current moment, including: constructing an objective function of the reinforcement learning model based on the communication performance benefits corresponding to various actions at future moments; solving the objective function under the condition of maximizing the value of the objective function to determine the traffic allocation strategy.

[0011] In some embodiments, based on the communication performance benefits corresponding to various actions at future moments, constructing the objective function of the reinforcement learning model includes: calculating the communication performance benefits corresponding to future moments based on the user experience data and UPF resource utilization efficiency at future moments, and calculating the communication performance benefits corresponding to the current moment based on the user experience data and UPF resource utilization efficiency at the current moment; constructing the objective function based on the weighted sum of the communication performance benefits corresponding to the future moment and the communication performance benefits corresponding to the current moment.

[0012] In some embodiments, the user experience data includes at least one of a packet loss rate and a data transmission delay.

[0013] In some embodiments, predicting user plane load information at future moments based on user plane load information at historical moments and user plane load information at the current moment includes: predicting user plane load information at future moments using an artificial intelligence model based on user plane load information at historical moments and user plane load information at the current moment.

[0014] In some embodiments, the user plane load information includes at least one of the traffic load of each UPF entity, the virtual resource occupancy of each UPF entity, whether the cache queue of each UPF entity is congested, and the time required for each UPF entity to process data in the cache queue.

[0015] In some embodiments, the allocation method further includes: using an SMF (System Management Function) entity to distribute load traffic to each UPF entity according to a current traffic allocation strategy.

[0016] According to some other embodiments of the present disclosure, a load traffic distribution device is provided, including: a prediction unit, used to predict user plane load information at future moments based on user plane load information at historical moments and user plane load information at current moments; a construction unit, used to construct a reinforcement learning model for determining the traffic distribution strategy of each UPF entity based on each user plane load information; and a determination unit, used to determine the traffic distribution strategy of each UPF entity at the current moment by using the reinforcement learning model.

[0017] In some embodiments, the construction unit maps the user plane load information at the current moment and the user plane load information at the future moment to the states in the reinforcement learning model at the current moment and the future moment respectively, and maps the various candidate traffic allocation strategies at the current moment to various actions in the reinforcement learning model at the current moment; based on the states and actions, the load traffic allocation process between each UPF entity is constructed as a Markov process model as a reinforcement learning model.

[0018] In some embodiments, the construction unit constructs the objective function of the reinforcement learning model according to the communication performance benefits corresponding to various actions at future moments, and the determination unit solves the objective function to determine the traffic allocation strategy based on maximizing the value of the objective function.

[0019] In some embodiments, the determination unit calculates the communication performance benefit corresponding to the future moment based on the user experience data and UPF resource utilization efficiency at the future moment, and calculates the communication performance benefit corresponding to the current moment based on the user experience data and UPF resource utilization efficiency at the current moment. The construction unit constructs the objective function based on the weighted sum of the communication performance benefit corresponding to the future moment and the communication performance benefit corresponding to the current moment.

[0020] In some embodiments, the user experience data includes at least one of a packet loss rate and a data transmission delay.

[0021] In some embodiments, the prediction unit predicts the user plane load information at a future moment using an artificial intelligence model based on the user plane load information at a historical moment and the user plane load information at a current moment.

[0022] In some embodiments, the user plane load information includes at least one of the traffic load of each UPF entity, the virtual resource occupancy of each UPF entity, whether the cache queue of each UPF entity is congested, and the time required for each UPF entity to process data in the cache queue.

[0023] In some embodiments, the distribution device also includes: a distribution unit uses the SMF entity to distribute load traffic to each UPF entity according to the traffic distribution strategy at the current moment.

[0024] According to some further embodiments of the present disclosure, a load flow distribution device is provided, comprising: a memory; and a processor coupled to the memory, wherein the processor is configured to execute the load flow distribution method in any of the above embodiments based on instructions stored in the memory device.

[0025] According to some further embodiments of the present disclosure, a non-volatile computer-readable storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, the method for distributing load flow in any of the above embodiments is implemented.

[0026] In the above embodiment, based on the user plane load information of historical moments, current moments, and predicted user plane load information of future moments, a reinforcement learning model is constructed to evenly distribute the traffic of each UPF entity load. In this way, there is no need to discard data, avoid network performance loss, and adapt to the load traffic changes of the UPF entity at future moments, improve the load traffic distribution effect, and thus solve the network congestion problem. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] The accompanying drawings, which constitute a part of the specification, illustrate embodiments of the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0028] The present disclosure may be more clearly understood from the following detailed description with reference to the accompanying drawings:

[0029] Figure 1 A flow chart showing some embodiments of the load flow distribution method disclosed in the present invention;

[0030] Figure 2 Show Figure 1 Flowcharts of some embodiments of step 120;

[0031] Figure 3 Show Figure 1 Flowcharts of some embodiments of step 130;

[0032] Figure 4 A block diagram showing some embodiments of the load flow distribution device disclosed herein;

[0033] Figure 5 A block diagram showing some other embodiments of the load flow distribution device disclosed in the present invention;

[0034] Figure 6 A block diagram showing some further embodiments of the load flow distribution device disclosed in the present invention. DETAILED DESCRIPTION

[0035] Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. It should be noted that the relative arrangement of components and steps, numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present disclosure unless otherwise specifically stated.

[0036] At the same time, it should be understood that for the convenience of description, the sizes of the various parts shown in the drawings are not drawn according to the actual proportional relationship.

[0037] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the present disclosure, its application, or uses.

[0038] Technologies, methods and equipment known to ordinary technicians in the relevant field may not be discussed in detail, but where appropriate, the technologies, methods and equipment should be considered part of the authorization specification.

[0039] In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limiting. Therefore, other examples of the exemplary embodiments may have different values.

[0040] It should be noted that like reference numerals and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0041] As mentioned above, the technology based on data discarding to alleviate network congestion requires queuing the data received by the UPF entity when dealing with network congestion; and forwarding the data in the queue according to the "first in, first out" rule within a period of time.

[0042] When the input rate of the UPF entity is greater than the output rate, the queue length will continue to accumulate until the queue length exceeds its preset threshold, thus forming cache congestion. At this time, it is necessary to discard the data that exceeds the queue length threshold to alleviate network congestion. However, the technology of alleviating network congestion based on data discarding has the following defects:

[0043] Failure to consider users' various service experiences (such as packet loss rate, latency, etc.), resulting in loss of network performance and affecting user experience;

[0044] The resource utilization efficiency of the UPF entity is not considered, and the resources of the UPF entity in the idle state cannot be fully utilized, resulting in low resource utilization efficiency, affecting network performance and user experience;

[0045] The impact of future traffic changes in the UPF entity is not considered, resulting in an inability to respond faster to future changes in network demand, affecting user experience.

[0046] In response to the above technical problems, the present disclosure predicts and evaluates future network traffic based on historical data and current data, and applies a reinforcement learning algorithm to redistribute data among UPF entities. In this way, the load can be evenly distributed among UPF entities, or part of the traffic can be transferred from a congested UPF entity to an idle UPF entity, thereby improving the resource utilization efficiency of the UPF entity, alleviating congestion, and improving the user service experience. For example, the technical solution of the present disclosure can be implemented through the following embodiments.

[0047] Figure 1 A flow chart showing some embodiments of the load flow distribution method of the present disclosure.

[0048] like Figure 1 As shown, in step 110, the user plane load information at the future moment is predicted based on the user plane load information at the historical moment and the user plane load information at the current moment. For example, the user plane load information includes at least one of the traffic load of each UPF entity, the virtual resource occupancy of each UPF entity, whether the cache queue of each UPF entity is congested, and the time required for each UPF entity to process the data in the cache queue.

[0049] In some embodiments, the collected user plane load information may be subjected to data preprocessing, including data cleaning, feature construction, feature data set construction, and the like.

[0050] For example, user plane load information is collected to form a feature data set. The feature data set may include traffic load information of each UPF entity (i.e., user plane load information), virtual resource occupancy of each UPF entity (e.g., virtual resources may include at least one of computing resources, network resources, and storage resources), cache queue status of each UPF entity (i.e., whether the cache queue is congested), length threshold of the cache queue of each UPF entity (i.e., the threshold used to determine whether to perform a data discard operation), and the time required for each UPF entity to process data in the cache queue.

[0051] In some embodiments, based on the user plane load information at historical moments and the user plane load information at the current moment, an artificial intelligence model is used to predict the user plane load information at future moments.

[0052] For example, based on the above feature data, artificial intelligence technology is used to predict the traffic load of each UPF entity, the virtual resource occupancy of each UPF entity, whether the cache queue is congested, and the time required to process data in the cache queue; the feature data set is updated using the predicted data and the length threshold.

[0053] In step 120, a reinforcement learning model is constructed to determine the traffic distribution strategy of each UPF entity based on the load information of each user plane.

[0054] In some embodiments, the Figure 2 The embodiment in implements step 120.

[0055] Figure 2 Show Figure 1 Flowchart of some embodiments of step 120 in FIG.

[0056] like Figure 2 As shown, in step 1210, the user plane load information at the current moment and the user plane load information at the future moment are respectively mapped to the states at the current moment and the future moment in the reinforcement learning model.

[0057] In step 1220, various candidate traffic allocation strategies at the current moment are mapped to various actions in the reinforcement learning model at the current moment.

[0058] In some embodiments, a state set (State) of the reinforcement learning model is defined. The state set S(t) is composed of the traffic load L(t) of each UPF entity, the virtual resource occupancy V(t) of each UPF entity, the cache queue state Q(t) of each UPF entity, and the time T(t) required for each UPF entity to process the cache queue data, that is, (t) = {L(t), V(t), Q(t), T(t)}.

[0059] In step 1230, based on the state and the action, the load flow distribution process among the UPF entities is constructed as a Markov process model as a reinforcement learning model.

[0060] In some embodiments, an action set (Action) is defined. According to the state set S(t), the load flow is redistributed among the UPF entities. For example, the corresponding action set can be defined as A(t)={a i,j (t)},a i,j (t) means that at time t, UPF i The load flow is allocated to UPF j .

[0061] Based on the state set S(t) and action set A(t), the load flow balancing problem can be modeled as an MDP (Markov Decision Process) model. After building the reinforcement learning model, you can use Figure 1 Step 130 in the flow distribution is performed.

[0062] In step 130, the reinforcement learning model is used to determine the traffic allocation strategy of each UPF entity at the current moment.

[0063] In some embodiments, Figure 1 The embodiment in implements step 130.

[0064] Figure 3 Show Figure 1 Flowchart of some embodiments of step 130 in FIG.

[0065] like Figure 3 As shown, in step 1310, the objective function of the reinforcement learning model is constructed according to the communication performance benefits corresponding to various actions at future moments.

[0066] In some embodiments, the communication performance benefit corresponding to the future moment is calculated based on the user experience data and UPF resource utilization efficiency at the future moment; the communication performance benefit corresponding to the current moment is calculated based on the user experience data and UPF resource utilization efficiency at the current moment.

[0067] In some embodiments, the user experience data includes at least one of a packet loss rate and a data transmission delay. For example, in a certain state S(t) at time t, a certain traffic allocation action a is selected. i,j (t), calculate its benefit (a i,j (t)) is

[0068]

[0069] α is a normalization factor that can be adjusted according to timing requirements.

[0070] In some embodiments, according to the future time t n+1 The corresponding communication performance benefit and the current time t n The weighted sum of the corresponding communication performance benefits is used to construct the objective function. For example, for traffic allocation action a i,j (t n ), the objective function is:

[0071] R(a i,j (t n ))=r(a i,j (t n ))+γ·r(a i,j (tn+1 ))

[0072] γ is a conversion factor that can be adjusted according to timing requirements.

[0073] In step 1320, the objective function is solved under the condition of maximizing the value of the objective function to determine the flow distribution strategy. For example, the current time t can be calculated by the above formula n The values ​​of the objective functions corresponding to various actions are selected, and the action corresponding to the maximum objective function value is selected as the traffic distribution strategy.

[0074] In some embodiments, it is determined that t n After the traffic allocation strategy at time t is determined, the above method can be used to continue to determine n+1 For example, you can use t n+1 As the current moment, t n+2 As a future moment, the traffic allocation strategy is determined by solving the objective function using the constructed reinforcement learning model.

[0075] In some embodiments, the SMF entity is used to distribute the load traffic to each UPF entity according to the traffic distribution strategy at the current moment.

[0076] For example, using a reinforcement learning model to determine a load balancing strategy can assist the SMF entity in distributing the load traffic of the UPF entity. Thus, the load traffic can be evenly distributed among the UPF entities as much as possible, or part of the traffic can be transferred from the congested UPF entity to the idle UPF entity to improve the resource utilization efficiency of the UPF entity. In this way, network congestion can be alleviated and user experience can be improved.

[0077] In some embodiments, the technical architecture of the present disclosure may include a data source layer, a load balancing strategy formulation layer, a device layer, etc.

[0078] For example, the data source layer is used for data collection, data preprocessing, feature data set construction, etc. The feature data set may include the UPF entity traffic load, the virtual resource occupancy of the UPF entity, the UPF entity cache queue status (i.e., whether it is congested), the threshold of the UPF entity cache queue, and the time required for the UPF entity to process the cache queue data.

[0079] For example, the load balancing strategy formulation layer uses artificial intelligence technology to predict the UPF traffic load, virtual resource occupancy of UPF entities, and UPF cache queue status based on the feature data set constructed by the data source layer; the prediction results are combined with the data from the data source layer to construct a new feature data set; the reinforcement learning algorithm is used to reallocate data among UPF entities, improve the resource utilization efficiency of UPF entities, alleviate network congestion, and enhance user experience.

[0080] For example, the device layer uses the output of the UPF load balancing strategy formulation layer to assist the SMF entity in distributing the load traffic of the UPF entity. It distributes the load evenly among the UPF entities as much as possible, or diverts some traffic from the congested UPF entity. In this way, the resource utilization efficiency of the UPF entity is improved, network congestion is alleviated, and the user experience is improved.

[0081] In some embodiments, the steps of generating the UPF optimal load balancing strategy by the UPF load balancing strategy formulation layer include steps one to four.

[0082] In step 1, data preprocessing is performed, including data cleaning, feature construction, and feature data set construction. The data set features include UPF traffic load, UPF virtual resource occupancy, UPF cache queue status, UPF cache queue threshold, and the time required for UPF to process cache queue data.

[0083] In step 2, based on the data set in step 1, artificial intelligence technology is used to predict the traffic load of the UPF entity, the virtual resource occupancy of the UPF, and the state of the UPF entity cache queue; the feature data set is updated together with the threshold of the UPF entity cache queue and the time required for the UPF entity to process the cache queue data. For example, the optimal load balancing strategy for the UPF entity can be generated according to the following steps.

[0084] In the content prediction step, the traffic load of the UPF entity, the virtual resource occupancy of the UPF, and the cache queue status of the UPF entity are predicted based on artificial intelligence prediction technology (such as various machine learning models).

[0085] In the step of defining a state set, the state set is composed of the UPF traffic load, the virtual resource occupancy of the UPF, the UPF cache queue state, and the time required for the UPF to process the cache queue data.

[0086] In the action set definition step, traffic will be redistributed between UPF entities according to the above state set, and the corresponding action set will be defined.

[0087] In defining the benefit function, under a certain state, UPF selects a certain traffic distribution action to calculate the benefit; based on the condition of maximizing the objective function, the optimal UPF entity load balancing strategy is calculated and the system status is updated.

[0088] In the above embodiment, the four-tuple {S, A, P s,a ,R},P s,a is the probability of state S and action A. The redistribution action of traffic between UPF entities is mapped to an Actiona i,j (t n )∈A(t), action a will be selectedi,j (t n ) After that, the traffic load of the UPF entity, the virtual resource occupancy of the UPF, the cache queue status of the UPF, and the time required for the UPF to process the cache queue data are mapped into a state S i,j (t n+1 )∈S(t), the formulation of the load balancing strategy of the UPF entity is modeled as an MDP model.

[0089] By maximizing the objective function R(a i,j (t n )), we can get the optimal UPF entity load balancing solution and update the system status. Maximize the objective function R(a i,j (t n )), we can use the dynamic programming algorithm to converge to the optimal strategy.

[0090] In step three, based on the results of the UPF entity's load balancing strategy, the load balancing effect evaluation indicators (i.e., user experience data, UPF resource utilization efficiency, etc.) are fed back to the UPF entity load balancing strategy formulation model to update the value of the benefit function, thereby being used to update the UPF entity's load balancing strategy.

[0091] In step 4, the load balancing strategy developed by the reinforcement learning model is used to assist the SMF entity in distributing traffic to the UPF entity. The load is evenly distributed among the UPF entities as much as possible, or part of the traffic is transferred from the congested UPF entity to the idle UPF entity. In this way, the resource utilization efficiency of the UPF entity can be improved, network congestion can be alleviated, and the user experience can be improved.

[0092] In the above embodiment, taking into account user plane load information such as UPF traffic load, UPF virtual resource occupancy, UPF cache queue status, UPF cache queue threshold, and the time required for UPF to process cache queue data, the traffic redistribution action between UPF entities is mapped to Action; the UPF traffic load after selecting action Action, the UPF cache queue status, the time required for UPF to process cache queue data, etc. are mapped to state State; the UPF load balancing strategy formulation problem is modeled as an MDP problem; and the RL algorithm is used to assist SMF in redistributing data between UPFs, so as to improve the efficiency of UPF resource utilization, alleviate network congestion, and enhance user experience.

[0093] The impact of user service experience (packet loss rate, data transmission delay, etc.) and UPF resource utilization efficiency on the formulation of UPF load balancing solutions is jointly considered to improve UPF resource utilization efficiency and user service experience while alleviating congestion.

[0094] The impact of future UPF traffic changes on load balancing strategy formulation is taken into consideration. Artificial intelligence technology is used to predict and evaluate future UPF traffic changes based on historical data and real-time data, and is used to calculate the objective function of the MDP problem, so that the load balancing strategy can respond more quickly to future changes in network demand.

[0095] Figure 4 A block diagram showing some embodiments of the load flow distribution device of the present disclosure.

[0096] like Figure 4 As shown, the load flow distribution device 4 includes a prediction unit 41 , a construction unit 42 , and a determination unit 43 .

[0097] The prediction unit 41 predicts the user plane load information at a future moment based on the user plane load information at a historical moment and the user plane load information at a current moment. For example, the user plane load information includes at least one of the traffic load of each UPF entity, the virtual resource occupancy of each UPF entity, whether the cache queue of each UPF entity is congested, and the time required for each UPF entity to process data in the cache queue.

[0098] In some embodiments, the prediction unit 41 predicts the user plane load information at a future moment using an artificial intelligence model based on the user plane load information at a historical moment and the user plane load information at a current moment.

[0099] The construction unit 42 constructs a reinforcement learning model for determining the traffic distribution strategy of each UPF entity according to the load information of each user plane.

[0100] In some embodiments, the construction unit 42 maps the user plane load information at the current moment and the user plane load information at the future moment to the states in the reinforcement learning model at the current moment and the future moment respectively; maps various candidate traffic allocation strategies at the current moment to various actions in the reinforcement learning model at the current moment; and constructs the load traffic allocation process between each UPF entity into a Markov process model as a reinforcement learning model based on the states and actions.

[0101] The determination unit 43 uses a reinforcement learning model to determine the flow distribution strategy of each UPF entity at the current moment.

[0102] In some embodiments, the construction unit 42 constructs the objective function of the reinforcement learning model according to the communication performance benefits corresponding to various actions at future moments; the determination unit 43 solves the objective function to determine the traffic allocation strategy under the condition of maximizing the value of the objective function.

[0103] In some embodiments, the determination unit 43 calculates the communication performance benefit corresponding to the future moment according to the user experience data and UPF resource utilization efficiency at the future moment; calculates the communication performance benefit corresponding to the current moment according to the user experience data and UPF resource utilization efficiency at the current moment; the construction unit 42 constructs the objective function according to the weighted sum of the communication performance benefit corresponding to the future moment and the communication performance benefit corresponding to the current moment. For example, the user experience data includes at least one of the packet loss rate and the data transmission delay.

[0104] In some embodiments, the distribution device 4 also includes: a distribution unit 44 uses the SMF entity to distribute the load flow to each UPF entity according to the flow distribution strategy at the current moment.

[0105] Figure 5 A block diagram showing some other embodiments of the load flow distribution device disclosed in the present invention.

[0106] like Figure 5 As shown, the load flow distribution device 5 of this embodiment includes: a memory 51 and a processor 52 coupled to the memory 51, and the processor 52 is configured to execute the load flow distribution method in any embodiment of the present disclosure based on the instructions stored in the memory 51.

[0107] The memory 51 may include, for example, a system memory, a fixed non-volatile storage medium, etc. The system memory may store, for example, an operating system, an application program, a boot loader, a database, and other programs.

[0108] Figure 6 A block diagram showing some further embodiments of the load flow distribution device disclosed in the present invention.

[0109] like Figure 6 As shown, the load flow distribution device 6 of this embodiment includes: a memory 610 and a processor 620 coupled to the memory 610, and the processor 620 is configured to execute the load flow distribution method in any of the above-mentioned embodiments based on the instructions stored in the memory 610.

[0110] The memory 610 may include, for example, a system memory, a fixed non-volatile storage medium, etc. The system memory may store, for example, an operating system, an application program, a boot loader, and other programs.

[0111] The load flow distribution device 6 may also include an input / output interface 630, a network interface 640, a storage interface 650, etc. These interfaces 630, 640, 650, the memory 610, and the processor 620 may be connected, for example, via a bus 660. The input / output interface 630 provides a connection interface for input / output devices such as a display, a mouse, a keyboard, a touch screen, a microphone, and a speaker. The network interface 640 provides a connection interface for various networked devices. The storage interface 650 provides a connection interface for external storage devices such as an SD card and a USB flash drive.

[0112] Those skilled in the art will appreciate that the embodiments of the present disclosure may be provided as methods, systems, or computer program products. Therefore, the present disclosure may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present disclosure may take the form of a computer program product implemented on one or more computer-usable non-transient storage media including but not limited to disk storage, CD-ROM, optical storage, etc., which contain computer-usable program code.

[0113] So far, the load flow distribution method, load flow distribution device and non-volatile computer readable storage medium according to the present disclosure have been described in detail. In order to avoid obscuring the concept of the present disclosure, some details known in the art are not described. Based on the above description, those skilled in the art can fully understand how to implement the technical solution disclosed herein.

[0114] The method and system of the present disclosure may be implemented in many ways. For example, the method and system of the present disclosure may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above order of steps for the method is for illustration only, and the steps of the method of the present disclosure are not limited to the order specifically described above, unless otherwise specifically stated. In addition, in some embodiments, the present disclosure may also be implemented as a program recorded in a recording medium, which includes machine-readable instructions for implementing the method according to the present disclosure. Therefore, the present disclosure also covers a recording medium storing a program for executing the method according to the present disclosure.

[0115] Although some specific embodiments of the present disclosure have been described in detail by way of example, it should be understood by those skilled in the art that the above examples are for illustration only and are not intended to limit the scope of the present disclosure. It should be understood by those skilled in the art that the above embodiments may be modified without departing from the scope and spirit of the present disclosure. The scope of the present disclosure is defined by the appended claims.

Claims

1. A method for distributing load flow, comprising: Predicting user plane load information at a future time based on user plane load information at a historical time and user plane load information at a current time; According to the load information of each user plane, a reinforcement learning model is constructed to determine the traffic allocation strategy of each user plane function UPF entity; Using the reinforcement learning model, determine the flow distribution strategy of each UPF entity at the current moment; Among them, the reinforcement learning model for determining the traffic allocation strategy of each user plane function UPF entity according to the load information of each user plane includes: Mapping the user plane load information at the current moment and the user plane load information at the future moment into the states in the reinforcement learning model at the current moment and the future moment, respectively; Mapping various candidate traffic allocation strategies at the current moment to various actions in the reinforcement learning model at the current moment; According to the state and the action, the load flow distribution process between each UPF entity is constructed as a Markov process model as the reinforcement learning model, The method of using the reinforcement learning model to determine the flow distribution strategy of each UPF entity at the current moment includes: Constructing the objective function of the reinforcement learning model according to the communication performance benefits corresponding to the various actions at future moments; The objective function is solved under the condition of maximizing the value of the objective function to determine the flow distribution strategy.

2. The distribution method according to claim 1, wherein: The objective function of the reinforcement learning model is constructed according to the communication performance benefits corresponding to the various actions at future moments, including: Calculate the communication performance benefit corresponding to the future moment according to the user experience data and UPF resource utilization efficiency at the future moment, and calculate the communication performance benefit corresponding to the current moment according to the user experience data and UPF resource utilization efficiency at the current moment; The objective function is constructed according to the weighted sum of the communication performance benefit corresponding to the future moment and the communication performance benefit corresponding to the current moment.

3. The distribution method according to claim 2, wherein: The user experience data includes at least one of a packet loss rate and a data transmission delay.

4. The allocation method according to any one of claims 1 to 3, wherein: The predicting of user plane load information at a future moment according to user plane load information at a historical moment and user plane load information at a current moment includes: Based on the user plane load information at historical moments and the user plane load information at the current moment, an artificial intelligence model is used to predict the user plane load information at future moments.

5. The allocation method according to any one of claims 1 to 3, wherein: The user plane load information includes at least one of the traffic load of each UPF entity, the virtual resource occupancy of each UPF entity, whether the cache queue of each UPF entity is congested, and the time required for each UPF entity to process data in the cache queue.

6. The allocation method according to any one of claims 1 to 3, further comprising: According to the traffic distribution strategy at the current moment, the system management function SMF entity is used to distribute the load traffic to each UPF entity.

7. A load flow distribution device, comprising: A prediction unit, configured to predict user plane load information at a future moment according to user plane load information at a historical moment and user plane load information at a current moment; A construction unit, used to construct a reinforcement learning model for determining a traffic allocation strategy for each user plane function UPF entity according to the load information of each user plane; A determination unit is used to determine the flow distribution strategy of each UPF entity at the current moment by using the reinforcement learning model, in, The construction unit maps the user plane load information at the current moment and the user plane load information at the future moment to the states in the reinforcement learning model at the current moment and the future moment respectively, maps various candidate traffic allocation strategies at the current moment to various actions in the reinforcement learning model at the current moment, and constructs the load traffic allocation process between each UPF entity into a Markov process model according to the states and the actions as the reinforcement learning model, The construction unit constructs the objective function of the reinforcement learning model according to the communication performance benefits corresponding to the various actions at future moments, The determining unit solves the objective function to determine the flow distribution strategy under the condition of maximizing the value of the objective function.

8. The dispensing device according to claim 7, wherein: The determination unit calculates the communication performance benefit corresponding to the future moment based on the user experience data and UPF resource utilization efficiency at the future moment, and calculates the communication performance benefit corresponding to the current moment based on the user experience data and UPF resource utilization efficiency at the current moment. The construction unit constructs the objective function based on the weighted sum of the communication performance benefit corresponding to the future moment and the communication performance benefit corresponding to the current moment.

9. The dispensing device according to claim 8, wherein: The user experience data includes at least one of a packet loss rate and a data transmission delay.

10. The dispensing device according to any one of claims 7 to 9, wherein: The prediction unit predicts the user plane load information at a future moment by using an artificial intelligence model based on the user plane load information at a historical moment and the user plane load information at a current moment.

11. The dispensing device according to any one of claims 7 to 9, wherein: The user plane load information includes at least one of the traffic load of each UPF entity, the virtual resource occupancy of each UPF entity, whether the cache queue of each UPF entity is congested, and the time required for each UPF entity to process data in the cache queue.

12. The dispensing device according to any one of claims 7 to 9, further comprising: The allocation unit distributes the load flow to each UPF entity using the system management function SMF entity according to the flow distribution strategy at the current moment.

13. A load flow distribution device, comprising: Memory; and A processor coupled to the memory, wherein the processor is configured to execute the load flow distribution method according to any one of claims 1 to 6 based on instructions stored in the memory.

14. A non-volatile computer-readable storage medium having a computer program stored thereon, wherein when the program is executed by a processor, the method for distributing load flow according to any one of claims 1 to 6 is implemented.

Citation Information

Patent Citations

  • Prediction model training method and system

    CN109934512A

  • Method and device for determining background traffic transmission strategy

    CN110519795A

  • Electric energy network coordination control method and system and information data processing terminal

    CN112101651A

  • Redundant transmission determination

    WO2019222905A1

  • A method of managing traffic by a user plane function, UPF, corresponding UPF, session management function and network data analytics function

    WO2020057763A1