Method and device for predicting access to network

Through the reinforcement learning model, the problem of inaccurate prediction of access networks is solved, the network resources are fully utilized, and different service needs are met.

CN114554573BActive Publication Date: 2024-11-26CHINA TELECOM CORP LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202011344349.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2020-11-26
Publication Date
2024-11-26
Estimated Expiration
2040-11-26

AI Technical Summary

Technical Problem

The prior art cannot objectively predict the access network of the current user based on different situations, resulting in inaccurate prediction of the access network and inability to make full use of network resources.

Method used

Using reinforcement learning model, a feature vector is generated based on the terminal information, service information and relevant information of the access network of the user to be accessed, the average return value of each behavior is calculated, the optimal access network is selected, and the prediction result is determined by minimizing the objective function. , improve the accuracy of access network prediction.

Benefits of technology

It realizes objectively predicting the access network of the current user based on different situations, improving the accuracy of access network prediction, thereby making full use of network resources to meet consumers' different needs for new and traditional services.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114554573B_ABST
    Figure CN114554573B_ABST
Patent Text Reader

Abstract

The present disclosure relates to a prediction method and device for access networks, and to the field of communication technology. The method includes: taking the selection of each access network of the user to be accessed as a behavior, generating a feature vector of each behavior according to the terminal information, service information and related information of each access network of the user to be accessed; taking the service experience of the user to be accessed as the benefit, modeling the access network prediction problem as a reinforcement learning model; inputting each feature vector into the reinforcement learning model to determine the prediction result of the access network.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to the field of communication technology, and in particular to a method and device for predicting an access network, and a non-volatile computer-readable storage medium. Background Art

[0002] With the development of communication technology, operators can provide consumers with various new services in addition to traditional services such as voice services, such as augmented reality, virtual reality, etc. These new services require higher uplink throughput than traditional services.

[0003] Compared with 4G E-UTRAN (Evolved UMTS Terrestrial Radio Access Network), 5G NR (New Radio) has higher bandwidth and lower latency, which can meet the needs of new services. When users use the above new services, 5G NR can provide users with a better service experience; when users use traditional services, they can fall back to 4G E-UTRAN.

[0004] Many operators will perform frequency reorganization and deploy NR with high and low frequencies. Low frequencies have a wider coverage than high frequencies. When the terminal uses services in a specific area without moving between areas of interest, 5G NR with high frequencies can provide users with a good service experience; when the terminal needs to move between different areas of interest, the network needs to have a wide coverage, so 5G NR with low frequencies can provide users with a good service experience.

[0005] Therefore, it is necessary to select RAT (Radio Access Technology) and frequency layer to maximize the use of network resources and meet consumers' different service demands for new services and traditional services.

[0006] In the related technology, the current 5G standard specification already supports the RFSP (RAT / frequency Selection Policy) mechanism. The PCF (Policy Control Function) derives the appropriate RFSP index based on the service analysis data provided by the NWDAF (Network Data Analytics Function) to support wireless resource management. Summary of the invention

[0007] The inventors of the present disclosure have discovered that the above-mentioned related technologies have the following problems: it is impossible to objectively predict the access network of the current user according to different situations, resulting in inaccurate access network prediction, thereby failing to fully utilize network resources.

[0008] In view of this, the present disclosure proposes a technical solution for predicting access networks, which can improve the accuracy of access network prediction and thus make full use of network resources.

[0009] According to some embodiments of the present disclosure, a method for predicting an access network is provided, comprising: taking the selection of each access network by a user to be accessed as a behavior, and generating a feature vector for each behavior according to terminal information, service information, and related information of each access network of the user to be accessed; taking the service experience of the user to be accessed as a benefit, modeling the access network prediction problem as a reinforcement learning model; and inputting each feature vector into the reinforcement learning model to determine a prediction result of the access network.

[0010] In some embodiments, the inputting of each feature vector into the reinforcement learning model to determine the prediction result of the access network includes: calculating, based on each feature vector, the mean of the benefits of each of the behaviors up to the current test moment; selecting the access network corresponding to the behavior with the largest mean benefit as the candidate access network at the current test moment; determining the prediction result based on the total benefits of the candidate access networks at each test moment and the preset expected benefits, the total benefit being the sum of the means of the benefits of the candidate access networks.

[0011] In some embodiments, the calculation of the mean of the benefit of each of the behaviors up to the current test moment based on each feature vector includes: obtaining the historical mean of the benefit of each of the behaviors up to the previous test moment and the number of times it was selected as a candidate access network; determining the estimated value of the benefit of each of the behaviors at the current test moment based on each feature vector; and calculating the mean of the benefit of each of the behaviors based on the historical mean, the number of times, and the estimated value.

[0012] In some embodiments, determining the prediction result based on the total revenue of the candidate access network at each test moment and the preset expected revenue includes: taking the prediction result as a variable, constructing an objective function according to the difference between the expectation of the total revenue and the expectation of the expected revenue; and solving the objective function to determine the prediction result under the condition of minimizing the objective function.

[0013] In some embodiments, the terminal information includes at least one of the terminal's mobility and the type of location; the service information includes at least one of the service time period and the service type; and the access network related information includes the access frequency or RAT.

[0014] In some embodiments, the prediction method further includes: sending the prediction result to a RAN (Radio Access Network) so that the RAN selects an access network for the user to access.

[0015] According to some other embodiments of the present disclosure, there is provided a prediction device for access network, comprising: a generation unit, for taking the selection of each access network of the user to be accessed as a behavior, and generating a feature vector for each behavior according to the terminal information, service information and related information of the user to be accessed; a modeling unit, for modeling the prediction problem of the access network as a reinforcement learning model with the service experience of the user to be accessed as a benefit; and a prediction unit, for inputting each feature vector into the reinforcement learning model to determine the prediction result of the access network.

[0016] In some embodiments, the prediction unit calculates the mean of the benefits of each of the behaviors up to the current test moment based on each feature vector, selects the access network corresponding to the behavior with the largest mean benefit as the candidate access network for the current test moment, and determines the prediction result based on the total benefits of the candidate access networks at each test moment and the preset expected benefits, where the total benefit is the sum of the means of the benefits of the candidate access networks.

[0017] In some embodiments, the prediction unit obtains the historical mean of the benefits of each of the behaviors and the number of times they are selected as candidate access networks up to the previous test moment, determines the estimated value of the benefits of each of the behaviors at the current test moment based on each feature vector, and calculates the mean of the benefits of each of the behaviors based on the historical mean, the number of times, and the estimated value.

[0018] In some embodiments, the prediction unit uses the prediction result as a variable, constructs an objective function based on the difference between the expectation of the total revenue and the expectation of the expected revenue, and solves the objective function to determine the prediction result under the condition of minimizing the objective function.

[0019] In some embodiments, the terminal information includes at least one of the terminal's mobility and the type of location; the service information includes at least one of the service time period and the service type; and the access network related information includes the access frequency or RAT.

[0020] In some embodiments, the prediction device further includes: a sending unit, configured to send the prediction result to the RAN so that the RAN selects an access network for the user to access.

[0021] According to some further embodiments of the present disclosure, there is provided a device for predicting an access network, comprising: a memory; and a processor coupled to the memory, wherein the processor is configured to execute the method for predicting an access network in any one of the above embodiments based on instructions stored in the memory device.

[0022] According to some further embodiments of the present disclosure, a non-volatile computer-readable storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, the method for predicting access to a network in any of the above embodiments is implemented.

[0023] In the above embodiment, the relationship between the user's terminal information, service information, and related information of each access network and the user's service experience is jointly considered, the selection of the access network is mapped to behavior, and the access network prediction problem is modeled using a reinforcement algorithm to achieve access network prediction. In this way, the current user's access network can be objectively predicted according to different situations to improve the accuracy of access network prediction, thereby making full use of network resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The accompanying drawings, which constitute a part of the specification, illustrate embodiments of the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0025] The present disclosure can be more clearly understood from the following detailed description with reference to the accompanying drawings.

[0026] in:

[0027] Figure 1 A flowchart showing some embodiments of the method for predicting access to a network disclosed in the present invention;

[0028] Figure 2 Show Figure 1 Flowcharts of some embodiments of step 130;

[0029] Figure 3 Show Figure 2 Flowchart of some embodiments of step 1310;

[0030] Figure 4 A block diagram showing some embodiments of the apparatus for predicting access to a network disclosed in the present invention;

[0031] Figure 5 A block diagram showing some other embodiments of the apparatus for predicting access to a network disclosed in the present invention;

[0032] Figure 6 A block diagram showing some further embodiments of the apparatus for predicting access to a network according to the present disclosure. DETAILED DESCRIPTION

[0033] Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. It should be noted that the relative arrangement of components and steps, numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present disclosure unless otherwise specifically stated.

[0034] At the same time, it should be understood that for the convenience of description, the sizes of the various parts shown in the drawings are not drawn according to the actual proportional relationship.

[0035] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the present disclosure, its application, or uses.

[0036] Technologies, methods, and apparatus known to ordinary technicians in the relevant field may not be discussed in detail, but where appropriate, such technologies, methods, and apparatus should be considered part of the authorization specification.

[0037] In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limiting. Therefore, other examples of the exemplary embodiments may have different values.

[0038] It should be noted that like reference numerals and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0039] As mentioned above, helping RAN redirect the terminal to different RAT or frequency layers mainly involves two scenarios: RAT selection from 5GNR to 4G E-UTRAN; frequency selection from high-frequency NR to low-frequency NR.

[0040] For the above two scenarios, the present disclosure considers the service type, UE mobility, location type and service experience of the terminal within a certain time period, and applies the reinforcement learning algorithm to predict the access frequency or RAT; the prediction result is transmitted to the RAN through the AMF (Access and Mobility Management Function) entity to assist the RAN in resource management. In this way, network resources are utilized to the greatest extent and consumers' different service requirements for new services and traditional services are met. For example, the technical solution of the present disclosure can be implemented through the following embodiments.

[0041] Figure 1 A flow chart showing some embodiments of the method for predicting access to a network disclosed in the present invention.

[0042] like Figure 1As shown, in step 110, the selection of each access network of the user to be accessed is taken as an action, and a feature vector of each action is generated according to the terminal information, service information and related information of each access network of the user to be accessed.

[0043] In some embodiments, the terminal information includes at least one of the terminal's mobility and the type of location; the service information includes at least one of the service time period and the service type; and the relevant information of the access network includes the access frequency or RAT.

[0044] For example, service types may include eMBB (Enhanced Mobile Broadband), URLLC (Ultra-reliable and Low Latency Communications), etc.; location types may include CBD (Central Business District), residential areas, schools, etc.; the mobility of the terminal may be the moving range of the terminal within a certain period of time.

[0045] In step 120, the prediction problem of accessing the network is modeled as a reinforcement learning model with the service experience of the user to be accessed as the benefit. For example, the service experience can be the MoS (Mean Opinion Score) of the average observed service.

[0046] In step 130, each feature vector is input into a reinforcement learning model to determine a prediction result of accessing the network.

[0047] In some embodiments, the prediction result is sent to the RAN so that the RAN selects an access network for the user to access.

[0048] In some embodiments, Figure 2 The embodiment in implements step 130.

[0049] Figure 2 Show Figure 1 Flowchart of some embodiments of step 130 in FIG.

[0050] like Figure 2 As shown, in step 1310, the mean of the benefit of each behavior up to the current test time is calculated based on each feature vector.

[0051] In some embodiments, Figure 3 The embodiment in implements step 1310.

[0052] Figure 3 Show Figure 2 Flowchart of some embodiments of step 1310 in FIG.

[0053] like Figure 3 As shown, in step 310, the historical average of the benefit of each behavior and the number of times it is selected as a candidate access network up to the last test time are obtained.

[0054] In step 320, based on each feature vector, an estimated value of the benefit of each behavior at the current test moment is determined.

[0055] In step 330, the mean of the benefit of each behavior is calculated based on the historical mean, the number of times, and the estimated value.

[0056] In some embodiments, the Action collection The mean value of the benefit of any Action a is:

[0057]

[0058] r t,a represents the mean value of the return of an Action a up to time t, r t-1,a represents the mean (historical mean) of the return of an Action a up to time t-1, d t-1,a It indicates the number of times an Action a is selected up to time t-1. It represents the estimated value of the benefit of an Action a at time t.

[0059] After calculating the mean benefit of each behavior, we can Figure 2 The remaining steps in are used to perform access network prediction.

[0060] In step 1320, the access network corresponding to the behavior with the largest mean benefit is selected as the candidate access network at the current test moment.

[0061] In some embodiments, at the current test time t, the Actiona corresponding to the selected candidate access network t for:

[0062]

[0063] a t It means that at time t, the Action determined by the mean return is the Action with the larger mean return.

[0064] In step 1330, a prediction result is determined according to the total revenue of the candidate access networks at each test time and the preset expected revenue. The total revenue is the sum of the averages of the revenues of the candidate access networks.

[0065] In some embodiments, in T trials (i.e., T trial moments), the total revenue is:

[0066]

[0067] In some embodiments, the prediction result is used as a variable, and an objective function is constructed according to the difference between the expectation of the total revenue and the expectation of the expected revenue; and the prediction result is determined by solving the objective function under the condition of minimizing the objective function.

[0068] For example, mapping the selection of access frequency or RAT to an Action Therefore, there are 2 actions for frequency (NR high frequency, NR low frequency) or RAT (4G, 5G).

[0069] In some embodiments, the access network may be predicted by setting the following Algorithm X. For example, taking the user's service experience as the benefit, a linear relationship is constructed using the expected value of the benefit of Action a and the d-dimensional feature vector x corresponding to Action a:

[0070]

[0071] are the coefficients that describe this linear relationship.

[0072] In the tth test (i.e., test time t), the estimated values ​​of the benefits of each action can be collected and used as samples (a total of m). The eigenvector matrix is ​​recorded as X a (dimension is m×d), feedback is denoted by c a .c a It is the feedback generated based on the actual selected Action in the t-1th trial, that is, the actual benefit generated (the feedback of the Action not selected is defined as 0).

[0073] The training data (X a ,c a ) Input the logistic regression model and solve the following loss formula:

[0074] Loss(θ a )=||c a -X a θ a || 2 +λ||θ a || 2

[0075] In this way, we can get The solution is:

[0076]

[0077] I d is the identity matrix of d×d dimensions.

[0078] In some embodiments, the above problem can be solved using the LinUCB method to obtain the selected Action in the tth trial:

[0079]

[0080] In the above frequency and RAT prediction problems, frequency and RAT are regarded as actions in the MAB problem. The goal of Algorithm X is to design an algorithm to maximize the expected value of total benefits.

[0081] For example, the optimal value expectation of the total benefit can be defined as:

[0082]

[0083] is the action with the largest expected return at the experimental time t. The problem of maximizing the expected value of total return can be equivalent to the difference R between the expected value of the optimal total return and the expected value of the total return. A (T) Minimization problem:

[0084]

[0085] In T experiments, by making the difference R A (T) is minimized and the optimal frequency or RAT is predicted for each user.

[0086] In some embodiments, the technical architecture of the present disclosure may include a data source layer, a frequency RAT prediction layer, and a device layer.

[0087] For example, the data source layer is used for data collection, data preprocessing, data set construction, etc. The data set may include service time period, service type (such as eMBB, URLLC, etc.), UE mobility, location type (such as CBD, residential area and school), frequency or RAT, etc.

[0088] For example, the frequency RAT prediction layer is used to predict the frequency or RAT of the terminal based on the data set constructed by the data source layer, using the reinforcement learning method. The reinforcement learning method is that the agent (Agent, referring to the frequency RAT prediction platform) learns in a "trial and error" manner. The reward obtained by interacting with the environment guides the behavior, and the goal is to enable the agent to obtain the maximum reward, so as to improve the action plan to adapt to the environment.

[0089] In the MAB (Multi-Armed Bandit) model, a complete state sequence consists of a behavior and an immediate reward (revenue), which is independent of the state. The present disclosure models the frequency or RAT prediction problem as a MAB problem, takes the data set constructed by the data source layer as input, and maps the frequency or RAT selection to an Action. At the same time, the closed-loop feedback between the service experience and the frequency or RAT selection is considered, and a machine learning algorithm is applied to predict the optimal frequency or RAT for each user.

[0090] For example, the device layer is used to use the output results of the frequency or RAT prediction layer to assist the RAN in selecting frequency or RAT, thereby maximizing the use of network resources, meeting consumers' different service requirements for new services and traditional services, and improving users' long-term service experience.

[0091] In some embodiments, in the frequency RAT prediction layer, the step of generating the frequency or RAT prediction result may include steps 1-4.

[0092] In step 1, data preprocessing is performed, including data cleaning, feature construction, and data set construction.

[0093] For example, the data set features include service time period, service type (such as eMBB, URLLC, etc.), terminal mobility, location type (such as CBD, residential area and school), frequency or RAT.

[0094] For example, you can take business time period p, business type c s , terminal mobility m, location type c l , the terminal's frequency or RATc g Construct a new d-dimensional feature vector:

[0095] x=(p,c s ,m,c l ,c g )

[0096] In some embodiments, the service time period represents the division of 24 hours a day. For example, the division results are three time periods: 00-07 in the morning, 08-19 in the daytime, and 20-23 at night. The service type, terminal mobility, location type, frequency or RAT respectively represent the service type used by the terminal in a certain time period, the mobile range of the terminal, the location type, and the frequency or RAT.

[0097] In step 2, the feature vector obtained in step 1 is used as the input of the reinforcement learning model to generate frequency or RAT prediction results.

[0098] In some embodiments, for each action a (a∈A), a feature vector x corresponding to action a is generated. t,a Based on the data observed in the previous experiment, select an Actiona according to algorithmX t ∈A(the eigenvector is ) and get a benefit Represents the user's service experience. Input to algorithmX to update the Action selection strategy.

[0099] In step 3, for the prediction result of the frequency or RAT, the prediction effect evaluation index (ie, user experience) is fed back to the frequency or RAT prediction model for updating the frequency or RAT prediction.

[0100] In step 4, the frequency or RAT result predicted by the reinforcement learning model is transmitted to the RAN to assist the RAN in selecting the frequency or RAT. In this way, network resources are utilized to the maximum extent, and consumers' different service requirements for new services and traditional services are met, thus improving users' long-term service experience.

[0101] In the above embodiment, the relationship between service type, terminal mobility, location type, frequency or RAT and terminal service experience in different service time periods is jointly considered, and the selection of frequency or RAT is mapped to Action; the frequency or RAT prediction problem is modeled as a MAB problem using an enhanced algorithm to achieve frequency or RAT prediction to assist RAN in resource management.

[0102] The influence of user service experience on the frequency or RAT prediction results is considered, and the best user service experience is guaranteed while predicting the frequency or RAT.

[0103] Taking into account the long-term effect of the frequency or RAT prediction model, an enhanced algorithm is used to predict the frequency or RAT to assist RAN in selecting the frequency or RAT, maximize the use of network resources, meet consumers' different service demands for new and traditional services, and improve users' long-term service experience.

[0104] Figure 4 A block diagram showing some embodiments of the apparatus for predicting access to a network disclosed in the present invention.

[0105] like Figure 4 As shown, the prediction device 4 accessing the network includes a generation unit 41 , a modeling unit 42 , and a prediction unit 43 .

[0106] The generating unit 41 takes the selection of each access network of the user to be accessed as a behavior, and generates a feature vector of each behavior according to the terminal information, service information and related information of each access network of the user to be accessed.

[0107] In some embodiments, the terminal information includes at least one of the terminal's mobility and the type of location; the service information includes at least one of the service time period and the service type; and the relevant information of the access network includes the access frequency or RAT.

[0108] The modeling unit 42 takes the service experience of the user to be connected as the benefit and models the prediction problem of accessing the network as a reinforcement learning model.

[0109] The prediction unit 43 inputs each feature vector into the reinforcement learning model to determine the prediction result of the access network.

[0110] In some embodiments, the prediction unit 43 calculates the mean of the revenue of each behavior up to the current test moment based on each feature vector. The access network corresponding to the behavior with the largest mean revenue is selected as the candidate access network at the current test moment. The prediction result is determined based on the total revenue of the candidate access networks at each test moment and the preset expected revenue. The total revenue is the sum of the mean of the revenues of the candidate access networks.

[0111] In some embodiments, the prediction unit 43 obtains the historical mean of the revenue of each behavior and the number of times it is selected as a candidate access network up to the previous test time. According to each feature vector, the estimated value of the revenue of each behavior at the current test time is determined. According to the historical mean, the number of times, and the estimated value, the mean of the revenue of each behavior is calculated.

[0112] In some embodiments, the prediction unit 43 uses the prediction result as a variable, constructs an objective function according to the difference between the expectation of the total revenue and the expectation of the expected revenue, and solves the objective function to determine the prediction result under the condition of minimizing the objective function.

[0113] In some embodiments, the prediction device 4 further includes a sending unit 44, configured to send the prediction result to the RAN so that the RAN selects an access network for the user to be accessed.

[0114] Figure 5 A block diagram showing some other embodiments of the apparatus for predicting access to a network according to the present disclosure.

[0115] like Figure 5 As shown, the access network prediction device 5 of this embodiment includes: a memory 51 and a processor 52 coupled to the memory 51, and the processor 52 is configured to execute the access network prediction method in any embodiment of the present disclosure based on the instructions stored in the memory 51.

[0116] The memory 51 may include, for example, a system memory, a fixed non-volatile storage medium, etc. The system memory may store, for example, an operating system, an application program, a boot loader, a database, and other programs.

[0117] Figure 6 A block diagram showing some further embodiments of the apparatus for predicting access to a network according to the present disclosure.

[0118] like Figure 6 As shown, the access network prediction device 6 of this embodiment includes: a memory 610 and a processor 620 coupled to the memory 610, and the processor 620 is configured to execute the access network prediction method in any one of the aforementioned embodiments based on the instructions stored in the memory 610.

[0119] The memory 610 may include, for example, a system memory, a fixed non-volatile storage medium, etc. The system memory may store, for example, an operating system, an application program, a boot loader, and other programs.

[0120] The prediction device 6 connected to the network may also include an input / output interface 630, a network interface 640, a storage interface 650, etc. These interfaces 630, 640, 650, the memory 610, and the processor 620 may be connected, for example, via a bus 660. The input / output interface 630 provides a connection interface for input / output devices such as a display, a mouse, a keyboard, a touch screen, a microphone, and a speaker. The network interface 640 provides a connection interface for various networked devices. The storage interface 650 provides a connection interface for external storage devices such as an SD card and a USB flash drive.

[0121] Those skilled in the art will appreciate that the embodiments of the present disclosure may be provided as methods, systems, or computer program products. Therefore, the present disclosure may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present disclosure may take the form of a computer program product implemented on one or more computer-usable non-transient storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0122] So far, the access network prediction method and access network prediction device and non-volatile computer-readable storage medium according to the present disclosure have been described in detail. In order to avoid obscuring the concept of the present disclosure, some details known in the art are not described. Based on the above description, those skilled in the art can fully understand how to implement the technical solution disclosed herein.

[0123] The method and system of the present disclosure may be implemented in many ways. For example, the method and system of the present disclosure may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above order of steps for the method is for illustration only, and the steps of the method of the present disclosure are not limited to the order specifically described above, unless otherwise specifically stated. In addition, in some embodiments, the present disclosure may also be implemented as a program recorded in a recording medium, which includes machine-readable instructions for implementing the method according to the present disclosure. Therefore, the present disclosure also covers a recording medium storing a program for executing the method according to the present disclosure.

[0124] Although some specific embodiments of the present disclosure have been described in detail by way of example, it should be understood by those skilled in the art that the above examples are for illustration only and are not intended to limit the scope of the present disclosure. It should be understood by those skilled in the art that the above embodiments may be modified without departing from the scope and spirit of the present disclosure. The scope of the present disclosure is defined by the appended claims.

Claims

1. A method for predicting access to a network, comprising: Taking the selection of each access network of the user to be accessed as a behavior, generating a feature vector of each behavior according to the terminal information, service information of the user to be accessed and the related information of each access network; Taking the service experience of the user to be connected as the benefit, the prediction problem of accessing the network is modeled as a reinforcement learning model; Inputting each feature vector into the reinforcement learning model to determine a prediction result of accessing the network; The step of inputting each feature vector into the reinforcement learning model to determine a prediction result of accessing the network includes: According to each eigenvector, calculate the mean value of the benefit of each behavior up to the current test moment; Selecting the access network corresponding to the behavior with the largest mean benefit as the candidate access network at the current test moment; The prediction result is determined with the constraint condition of minimizing the difference between the total revenue of the candidate access networks at each test moment and the preset expected revenue, and the total revenue is the sum of the means of the revenues of the candidate access networks.

2. The prediction method according to claim 1, wherein: The calculation of the mean value of the benefit of each behavior up to the current test time based on each feature vector includes: Obtaining the historical average of the benefits of each of the behaviors and the number of times each behavior is selected as a candidate access network up to the last test moment; Determine the estimated value of the benefit of each of the behaviors at the current test moment according to each eigenvector; The mean of the benefits of each of the behaviors is calculated based on the historical mean, the number of times, and the estimated value.

3. The prediction method according to claim 1, wherein: The determining of the prediction result according to the total revenue of the candidate access network at each test time and the preset expected revenue includes: Taking the prediction result as a variable, constructing an objective function according to the difference between the expectation of the total benefit and the expectation of the expected benefit; The objective function is solved to determine the prediction result under the condition that the objective function is minimized.

4. The prediction method according to any one of claims 1 to 3, wherein: The terminal information includes at least one of the mobility of the terminal and the type of the location of the terminal; The service information includes at least one of a service time period and a service type; The access network related information includes access frequency or radio access technology RAT.

5. The prediction method according to any one of claims 1 to 3, further comprising: The prediction result is sent to a radio access network RAN ​​so that the RAN selects an access network for the user to access.

6. A prediction device for accessing a network, comprising: A generating unit, configured to take the selection of each access network of the user to be accessed as a behavior, and generate a feature vector of each behavior according to the terminal information, service information of the user to be accessed and the related information of each access network; A modeling unit, configured to model the prediction problem of access network as a reinforcement learning model with the service experience of the user to be accessed as the benefit; A prediction unit, used to input each feature vector into the reinforcement learning model to determine a prediction result of an access network; Among them, the prediction unit calculates the mean of the benefits of each of the behaviors up to the current test moment based on each feature vector, selects the access network corresponding to the behavior with the largest mean benefit as the candidate access network at the current test moment, and determines the prediction result with the constraint of minimizing the difference between the total benefit of the candidate access networks at each test moment and the preset expected benefit, wherein the total benefit is the sum of the means of the benefits of the candidate access networks.

7. The prediction device according to claim 6, wherein: The prediction unit obtains the historical mean of the benefit of each of the behaviors and the number of times it is selected as a candidate access network up to the previous test moment, determines the estimated value of the benefit of each of the behaviors at the current test moment based on each feature vector, and calculates the mean of the benefit of each of the behaviors based on the historical mean, the number of times, and the estimated value.

8. The prediction device according to claim 6, wherein: The prediction unit uses the prediction result as a variable, constructs an objective function according to the difference between the expectation of the total benefit and the expectation of the expected benefit, and solves the objective function to determine the prediction result under the condition of minimizing the objective function.

9. The prediction device according to any one of claims 6 to 8, wherein: The terminal information includes at least one of the mobility of the terminal and the type of the location of the terminal; The service information includes at least one of a service time period and a service type; The access network related information includes access frequency or radio access technology RAT.

10. The prediction device according to any one of claims 6 to 8, further comprising: The sending unit is used to send the prediction result to a radio access network RAN, so that the RAN selects an access network for the user to access.

11. A prediction device for accessing a network, comprising: Memory; and A processor coupled to the memory, wherein the processor is configured to execute the method for predicting an access network according to any one of claims 1 to 5 based on instructions stored in the memory.

12. A non-volatile computer-readable storage medium having a computer program stored thereon, wherein when the program is executed by a processor, the method for predicting an access network according to any one of claims 1 to 5 is implemented.