A terminal roaming guidance method and device, electronic equipment and storage medium

Abstract

This application discloses a terminal roaming guidance method, apparatus, electronic device, and storage medium, relating to the field of network communication technology. The method includes: acquiring historical roaming data of a first target roaming strategy and determining a roaming evaluation result for a target terminal based on the historical roaming data; the first target roaming strategy being the roaming strategy assigned to the target terminal when it enters an AP network environment; if the roaming evaluation result does not meet the expected requirements, adjusting the strategy parameters in the first target roaming strategy until they meet the expected requirements, thereby obtaining a second target roaming strategy for the target terminal; and guiding the target terminal to roam to a target AP in the AP network environment according to the second target roaming strategy, thereby improving the applicability of the terminal device during roaming.

Description

Technical Field

[0001] This application relates to the field of network communication technology, and in particular to a terminal roaming guidance method, device, electronic device, and storage medium. Background Technology

[0002] In wireless communication networks, terminal devices can maintain network communication continuously within the signal coverage area of ​​a wireless access point (AP) device. However, since the signal coverage area of ​​a single AP device is limited, terminal devices often need to move from one AP's service area to another when they are mobile.

[0003] To prevent network communication interruptions when terminal devices switch between different access points (APs), wireless roaming technology has been introduced. Wireless roaming refers to the process where, when a terminal device moves to the boundary area between the coverage areas of two APs, it associates with the new AP and disconnects from the original AP, maintaining an uninterrupted network connection throughout this process. This wireless roaming method must be initiated actively by the terminal device.

[0004] However, some terminal devices with poor roaming initiative (such as low-end mobile phones and tablets) will remain attached to their original access point (AP), even if the terminal device moves to a location with a strong signal and the distance from the original AP is large. These types of terminal devices are called sticky terminal devices.

[0005] For these types of sticky terminal devices, relying on the device's own roaming is insufficient. In this context, intelligent roaming has emerged as a solution. It guides the terminal device to roam in a way that the device understands, making it more intelligent than active roaming.

[0006] Currently, intelligent roaming guides terminal devices to roam, primarily by configuring a threshold on the access point (AP) to trigger roaming. When the AP detects that a terminal device's parameters (e.g., signal-to-noise ratio or communication rate) are below this threshold, it proactively sends a disconnect message to the terminal device, causing it to connect to an AP with better signal and faster communication speed. However, different types of terminal devices, and even different devices of the same type, have varying network communication capabilities. Therefore, the threshold for triggering roaming differs for different types of terminal devices, and even for different devices of the same type. If a threshold is used to guide roaming, the threshold setting is crucial. A fixed threshold is relatively simple to determine, but the roaming effect is poor. A variable threshold is more complex to determine, thus limiting the applicability of this threshold-based roaming method. Summary of the Invention

[0007] This application provides a terminal roaming guidance method to improve the applicability of terminal devices during roaming.

[0008] Firstly, a terminal roaming guidance method is provided, including:

[0009] The system acquires historical roaming data of the target terminal under a first target roaming policy and determines the roaming evaluation result of the target terminal based on the historical roaming data. The first target roaming policy is the roaming policy assigned to the target terminal when it enters the AP network environment. If the roaming evaluation result does not meet the expected requirements, the policy parameters in the first target roaming policy are adjusted until they meet the expected requirements, resulting in a second target roaming policy for the target terminal. Based on the second target roaming policy, the target terminal is guided to roam to a target AP in the AP network environment.

[0010] Optionally, the AP network environment includes N APs and a terminal profile database, wherein the terminal profile database includes at least one roaming policy for each terminal in the N APs, and N is an integer greater than 0; before obtaining the historical roaming data of the target terminal under the first target roaming policy, the method further includes:

[0011] Identify the terminal type of the target terminal. If a first terminal matching the target terminal type is found in the terminal profile database, the roaming policy of the first AP currently associated with the first terminal among the N APs is used as the first target roaming policy of the target terminal. If no first terminal matching the target terminal type is found in the terminal profile database, the initial roaming policy of the target terminal is determined based on the mutual scan information of the N APs in the AP network environment and the roaming information of the target terminal, and the initial roaming policy of the target terminal is used as the first target roaming policy of the target terminal.

[0012] In this embodiment of the application, based on the constructed terminal profile database, newly added target terminals can reuse the roaming strategies of similar terminals, thereby improving the speed of roaming.

[0013] Optionally, the method further includes:

[0014] Obtain network information of N APs in the AP network environment, where N is an integer greater than 0; determine the network environment characteristics of the AP network environment based on the network information of the N APs; construct an AP network profile of the AP network environment based on the network environment characteristics of the AP network environment, and store it in the network profile database.

[0015] In this embodiment of the application, by constructing the above-mentioned network profile library, various AP network environments can be summarized and distinguished. When constructing a new AP network environment in the future, the roaming strategy of similar AP network environments can be reused based on the network profile library. Therefore, the process of re-formulating the roaming strategy can be omitted, and the roaming experience is optimized.

[0016] Optionally, based on the network information of the N APs, the network environment characteristics of the AP network environment are determined, including:

[0017] The network environment characteristics of the AP network environment are determined based on the coverage information in the network information of the N APs; or based on the distribution information in the network information of the N APs; or based on the jitter drop point areas in the network information of the N APs.

[0018] In this embodiment, network environment features in the AP network environment can be extracted from multiple dimensions, making the subsequently constructed network profile database more accurate.

[0019] Optionally, the method further includes:

[0020] Obtain mutual scan information of M APs in the new AP network environment, where M is an integer greater than 0; sort the M APs by importance based on their mutual scan information; construct a mutual scan matrix for the new AP network environment based on the mutual scan signal strength values ​​of the M APs in the importance ranking; generate a new AP network profile for the new AP network environment based on the mutual scan matrix; if the new AP network profile of the new AP network environment meets the similarity requirement with the AP network profile of the AP network environment in the network profile library, then reuse the terminal profile library of the AP network environment for the new AP network environment.

[0021] In this embodiment of the application, when constructing an AP network environment, a fast cold start can be achieved based on the network profile library.

[0022] Optionally, adjusting the strategy parameters in the first target roaming strategy until they meet the desired requirements includes:

[0023] Based on the target terminal's first roaming state, after executing the selected first roaming action, the expected reward and the second roaming state at the next moment are obtained; according to the Q function, the second roaming action to be executed by the target terminal in the second roaming state is determined; according to the second roaming state, the second roaming action, and the expected reward, the Q function is updated until the updated Q function meets the expected requirements; wherein, the Q function satisfies the following expression: Q(s,a)=Q(s,a)+α(r+γQ(s) ′ a ′ )-Q(s,a)); where s is the first roaming state, a is the first roaming action, r is the expected reward, α is the set learning rate, γ is the set discount rate, s' is the second roaming state, and a' is the second roaming action.

[0024] Secondly, a terminal roaming guidance device is provided, comprising:

[0025] An evaluation module is used to acquire historical roaming data of the target terminal under a first target roaming policy, and determine the roaming evaluation result of the target terminal based on the historical roaming data; wherein, the first target roaming policy is the roaming policy assigned to the target terminal when it enters the AP network environment; an adjustment module is used to adjust the policy parameters in the first target roaming policy until they meet the expected requirements if the roaming evaluation result does not meet the expected requirements, thereby obtaining a second target roaming policy for the target terminal; a guidance module is used to guide the target terminal to roam to a target AP in the AP network environment according to the second target roaming policy.

[0026] Optionally, the AP network environment includes N APs and a terminal profile database, the terminal profile database including at least one roaming policy for each terminal in the N APs, where N is an integer greater than 0; the device further includes an allocation module; the allocation module is specifically used for:

[0027] Identify the terminal type of the target terminal; if a first terminal matching the terminal type is found in the terminal profile database, the roaming policy of the first AP currently associated with the first terminal among the N APs is used as the first target roaming policy of the target terminal; if no first terminal matching the terminal type is found in the terminal profile database, the initial roaming policy of the target terminal is determined based on the mutual scan information of the N APs in the AP network environment and the roaming information of the target terminal, and the initial roaming policy of the target terminal is used as the first target roaming policy of the target terminal.

[0028] Optionally, the device further includes a network image construction module; the network image construction module is used to acquire network information of N APs in the AP network environment, where N is an integer greater than 0; determine the network environment characteristics of the AP network environment based on the network information of the N APs; construct an AP network profile of the AP network environment based on the network environment characteristics of the AP network environment, and store it in a network profile library.

[0029] Optionally, the network image construction module is specifically used for:

[0030] The network environment characteristics of the AP network environment are determined based on the coverage information in the network information of the N APs; or based on the distribution information in the network information of the N APs; or based on the jitter drop point areas in the network information of the N APs.

[0031] Optionally, the network image construction module is further configured to: acquire mutual scan information of M APs in the new AP network environment, where M is an integer greater than 0; sort the M APs by importance based on their mutual scan information; construct a mutual scan matrix for the new AP network environment based on the mutual scan signal strength values ​​of the M APs in the importance ranking; generate a new AP network profile for the new AP network environment based on the mutual scan matrix; and if the new AP network profile of the new AP network environment meets the similarity requirement with the AP network profile of the AP network environment in the network profile library, then reuse the terminal profile library of the AP network environment for the new AP network environment.

[0032] Thirdly, an electronic device is provided, comprising:

[0033] A memory for storing computer programs; a processor for executing the computer programs stored in the memory to implement the method steps described in any one of the first aspects.

[0034] Fourthly, a computer-readable storage medium is provided, wherein a computer program is stored therein, and when executed by a processor, the computer program implements the steps of the method described in any one of the first aspects.

[0035] In this embodiment, the historical roaming data of the target terminal under the first target roaming strategy is analyzed. The first target roaming strategy is the roaming strategy assigned to the target terminal when it enters the AP network environment. It is determined whether the target terminal's roaming using the first target roaming strategy meets the expected requirements. If it does not meet the requirements, the strategy parameters in the first target roaming strategy are optimized before guiding the target terminal to roam. Therefore, compared with using a threshold method to guide terminal device roaming, this method takes into account the differences between different terminal devices. It can adaptively adjust the roaming strategy adopted by different terminal devices during the roaming process to maximize roaming benefits, thereby improving the applicability of terminal device roaming.

[0036] For the various aspects of the second to fourth aspects mentioned above, and the technical effects that each aspect may achieve, please refer to the above description of the technical effects that can be achieved for the first aspect or the various possible solutions in the first aspect, which will not be repeated here. Attached Figure Description

[0037] Figure 1 This is a schematic diagram illustrating the application scenarios applicable to the embodiments of this application;

[0038] Figure 2 A flowchart illustrating a terminal roaming guidance method provided in this application embodiment;

[0039] Figure 3 This is a schematic diagram of the structure of a terminal roaming guidance device provided in an embodiment of this application;

[0040] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0041] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The specific operational methods in the method embodiments can also be applied to the device embodiments or system embodiments. It should be noted that in the description of this application, "multiple" is understood as "at least two". "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. A connected to B can represent: A and B directly connected, and A and B connected through C. Furthermore, in the description of this application, terms such as "first" and "second" are used only for distinguishing the purpose of description and should not be construed as indicating or implying relative importance or order.

[0042] To facilitate understanding by those skilled in the art, the technical terms involved in the embodiments of this application will first be explained.

[0043] (1) Cold start refers to the ability of a terminal to reuse the roaming strategy of the same type of terminal, and the ability of a new AP network environment to reuse the roaming strategy of terminals associated with similar network environments.

[0044] (2) The SARSA algorithm (English: State Action Reward State Action, abbreviation: SARSA) is an algorithm for learning Markov decision process policies. It is commonly used in the fields of machine learning and reinforcement learning. It updates the current action (policy) by updating the value function (Q function), and then generates a new state and reward by using a new action (policy), which in turn updates the value function. This process continues iteratively until the value function and policy converge to meet the expected requirements.

[0045] (3) Greedy algorithms, also known as greedy algorithms, refer to algorithms that always make the best choice at the moment when solving a problem. In other words, they do not consider the overall optimal solution and obtain a locally optimal solution in some sense.

[0046] (4) Received Signal Strength Indicator (RSSI): Generally used to describe the strength of wireless signals. It refers to the signal strength during wireless reception and is related to the transmission power of the wireless module, the design of the RF front end, and the gain of the antenna.

[0047] The following is a brief introduction to the application scenarios to which the technical solutions of the embodiments of this application are applicable. It should be noted that the application scenarios described below are only for illustrating the embodiments of this application and are not intended to limit the scope. In specific implementation, the technical solutions provided by the embodiments of this application can be flexibly applied according to actual needs.

[0048] Figure 1 This is a schematic diagram of the application scenario to which this application applies. As shown in the figure, the scenario includes a target terminal 101 and an AP terminal 102; wherein the AP terminal 102 includes 3 APs. It should be noted that the number of APs involved in the AP terminal 102 is not limited. Figure 1 Only 3 APs are described.

[0049] AP 102 forms an AP network environment 103 with three APs. When a target terminal 101 enters this AP network environment 103, AP 102 can assign a matching first target roaming policy to the target terminal 101. If, during the roaming process, it is determined that the target terminal 101's roaming based on the first target roaming policy does not meet the expected requirements, the policy parameters of the first target roaming policy can be optimized in real time until they meet the expected requirements. Then, based on the optimized second target roaming policy, the target terminal 101 is guided to roam to the target AP in the AP network environment 103 (e.g., ...). Figure 1 AP1 in the context of network services, such as channels (resources) and Wi-Fi signals, can provide network services to the target terminal 101 that is connected.

[0050] The target terminal 101 can be an electronic device such as a mobile phone, smart wearable device, tablet computer, or laptop computer. In addition, the target terminal 101 can also have a related client installed. The client can be software, such as an application (APP), browser, short video software, or web page, mini program, etc.

[0051] In this embodiment, a network profile library can be pre-built. This library can be used to store AP network profiles for various AP network environments, and to enable rapid cold starts when building new AP network environments by performing similarity comparisons based on this library. Specifically, this can involve obtaining network information of N APs in the AP network environment (e.g., ...). Figure 1 The network information of AP1, AP2, and AP3 (in the provided text); based on the network information of these three APs, determine the AP network environment (e.g., ...). Figure 1 The network environment characteristics of the AP network environment 103 are analyzed; based on the network environment characteristics of these three APs, an AP network profile of the AP network environment is constructed and stored in the network profile library.

[0052] Optionally, the network environment characteristics of the above N APs can be determined from the following aspects:

[0053] Aspect 1: Based on the coverage information from N AP network information sets, determine the network environment characteristics of the AP network. Through coverage information, the network environment of the AP network can be marked (e.g., Figure 1 In the AP network environment 103, there are points with insufficient signal coverage, so as to avoid affecting the roaming effect due to insufficient signal coverage during roaming.

[0054] Aspect 2: Based on the distribution information of the N AP network data, determine the network environment characteristics of the AP network. The distribution information allows recording the specific location of each AP in the network environment, facilitating improved roaming accuracy during subsequent roaming operations.

[0055] Aspect 3: Based on the jitter point areas in the N AP network information, determine the network environment characteristics of the AP network. By identifying the jitter point areas, points in the AP network environment where signal jitter exists can be marked and addressed accordingly, preventing roaming performance from being affected by signal jitter during subsequent roaming.

[0056] Aspect 4: Based on the mutual scan information from the N AP network information, determine the network environment characteristics of the AP network environment. Through mutual scan information, the identifier, signal strength, etc. of each AP in the AP network environment can be recorded, thus facilitating the differentiation of each AP in the AP network environment.

[0057] It should be noted that the construction of AP network profiles for various AP network environments can be achieved through one or more of the above aspects. There are no restrictions on the specific combination methods. Furthermore, aspects 1 to 4 mentioned above are only examples. In actual roaming, AP network profiles can also be constructed based on other characteristics.

[0058] Optionally, when building a new AP network environment, a fast cold start method based on the network profile database can be used, specifically:

[0059] Obtain the mutual scan information of M APs in the new AP network environment, where M is an integer greater than 0; rank the M APs by importance based on their mutual scan information; construct the mutual scan matrix of the new AP network environment based on the mutual scan signal strength values ​​of each of the M APs in the importance ranking; generate a new AP network profile of the new AP network environment based on the mutual scan matrix; if the new AP network profile of the new AP network environment matches the AP network environment in the network profile database (e.g., ...), ... Figure 1If the AP network profile of AP network environment 103 in the network image meets the similarity requirement, the terminal profile library of AP network environment 103 will be reused for the new AP network environment to achieve a fast cold start; if the new AP network profile of the new AP network environment does not meet the similarity requirement with any AP network profile in the AP network environment in the network profile library, the new AP network profile will be added to the network profile library.

[0060] Optionally, the importance of the M APs can be ranked based on their mutual scan signal strength values, with the APs having higher mutual scan signal strength values ​​being more important; or the importance can be ranked based on their respective frequencies, with the APs having higher frequencies being more important.

[0061] Furthermore, generating a new AP network profile based on the mutual scan matrix of the new AP network environment can be achieved by: extracting network environment features of the new AP network environment using a feature training model based on the mutual scan matrix of the new AP network environment, and generating a new AP network profile based on the network environment features of the new AP network environment.

[0062] By constructing the aforementioned network profile database, various AP network environments can be summarized and distinguished. When constructing a new AP network environment in the future, roaming strategies for similar AP network environments can be reused based on this network profile database. Therefore, the process of re-formulating roaming strategies can be omitted, which can improve the roaming efficiency of terminals and optimize the roaming experience.

[0063] In this embodiment of the application, a terminal profile library may also be configured for the constructed AP network environment (for example, for...). Figure 1 The AP terminal 102 is configured with its own terminal profile library, which includes at least one roaming policy for each terminal in N APs. For example, Class A terminals include roaming policies for AP1, AP2, and AP3 respectively, Class B terminals include roaming policies for AP1, AP2, and AP3 respectively, and Class C terminals also include roaming policies for AP1, AP2, and AP3 respectively.

[0064] In some embodiments, the roaming strategies of various terminals in N APs can be distinguished by terminal profiles. A terminal profile may carry information such as the terminal's network communication capabilities (e.g., RSSI), terminal name, terminal identifier, terminal type, and the roaming strategy adopted by the terminal when roaming in each of the N APs. This terminal profile database can assist APs in guiding terminal roaming. For example, when a target terminal comes online, if a matching terminal profile exists in the terminal profile database, roaming guidance can be provided to the target terminal based on the matched terminal profile.

[0065] It should be noted that the specific forms of the aforementioned network profile database and terminal profile database are not limited in this embodiment of the application.

[0066] To further illustrate the technical solutions provided in the embodiments of this application, a detailed description is provided below in conjunction with the accompanying drawings and specific implementation methods. Although the embodiments of this application provide method operation steps as shown in the following embodiments or drawings, the method may include more or fewer operation steps based on conventional or non-inventive methods. In steps where there is no logically necessary causal relationship, the execution order of these steps is not limited to the execution order provided in the embodiments of this application. In actual processing or when the device executes the method, it may be executed in the order shown in the embodiments or drawings, or in combination.

[0067] Figure 2 This is a flowchart illustrating a terminal roaming guidance method provided in an embodiment of this application. This process can be executed by a terminal roaming guidance device, which can be implemented in software, hardware, or a combination of both. Figure 2 As shown, the process includes the following steps:

[0068] 201: Obtain the historical roaming data of the target terminal under the first target roaming strategy, and determine the roaming evaluation result of the target terminal based on the historical roaming data.

[0069] The first target roaming strategy is the target terminal (such as...) Figure 1 The roaming policy assigned to the target terminal 101 when it enters the AP network environment.

[0070] Optionally, the terminal type of the target terminal can be identified, and a first target roaming strategy can be assigned to the target terminal based on the terminal type of the target terminal and the terminal profile database configured above.

[0071] Optionally, the terminal type of the target terminal can be identified by recognizing the media access control (MAC) address; the terminal type of the target terminal can also be identified by the hypertext transfer protocol (HTTP) initiated by the target terminal; or the terminal type of the target terminal can be identified by the attribute information of the target terminal (e.g., user information, internet access behavior information, internet access device information, etc.); the embodiments of this application are not limited here.

[0072] Optionally, based on the terminal type of the target terminal and the terminal profile database configured above, a first target roaming strategy is assigned to the target terminal, which mainly includes the following two cases:

[0073] Scenario 1: If, based on the terminal type of the target terminal, a first terminal matching that type is found in the terminal profile database, then the roaming policy of the first AP currently associated with that first terminal among N APs is used as the first target roaming policy for the target terminal. This allows newly joined target terminals to reuse the roaming policy of terminals of the same type (referred to as cold start), improving the speed of roaming. Based on the above... Figure 1 For example, if the terminal type of the target terminal matches that of a Class A terminal, the roaming policies of the Class A terminal in AP1, AP2, and AP3 can be obtained from the terminal profile database. When it is determined that the first AP currently associated with the Class A terminal is AP1, the roaming policy of the Class A terminal in AP1 can be used as the first target roaming policy of the target terminal, thereby achieving a fast cold start.

[0074] Scenario 2: If, based on the terminal type of the target terminal, no first terminal matching the terminal type is found in the terminal profile database associated with each of the N APs, then the initial roaming strategy of the target terminal is determined based on the mutual scan information of the N APs in the AP network environment and the roaming information of the target terminal (e.g., signal strength before and after roaming, roaming time, channel utilization, uplink bytes, downlink bytes, packet loss rate, physical rate, etc.), and the initial roaming strategy of the target terminal is taken as the first target roaming strategy of the target terminal.

[0075] Optionally, the system may acquire historical roaming data of the target terminal over a period of time under the first target roaming policy; or it may acquire sufficient historical roaming data of the target terminal under the first target roaming policy; or it may acquire all historical roaming data of the target terminal under the first target roaming policy. This embodiment of the application does not impose any limitations on these options.

[0076] Optionally, the roaming evaluation result of the target terminal can be determined based on the historical roaming data. Specifically, this can be achieved by performing a roaming evaluation based on at least one historical roaming data point of the target terminal under the first target roaming strategy, such as signal strength, uplink physical rate, downlink physical rate, uplink byte count, downlink byte count, and packet loss rate, to obtain the roaming evaluation result of the target terminal. Here, uplink physical rate refers to the rate at which the target terminal transmits data, and downlink physical rate refers to the rate at which the target terminal receives data; uplink byte count refers to the number of bytes transmitted by the target terminal per unit time, and downlink byte count refers to the number of bytes received by the target terminal per unit time.

[0077] 202: If the roaming evaluation result does not meet the expected requirements, adjust the strategy parameters in the first target roaming strategy until they meet the expected requirements, and obtain the second target roaming strategy for the target terminal.

[0078] Optionally, the roaming assessment result can be obtained based on one type of historical roaming data or on multiple types of historical roaming data. This application embodiment does not impose any limitations on this.

[0079] For example, taking the aforementioned signal strength as an example, if the signal strength is assessed to be lower than the set signal strength threshold, it indicates that the target terminal does not meet the set expected requirements using the first target roaming strategy. In this case, the strategy parameters in the first target roaming strategy are adjusted until they meet the expected requirements, thus obtaining the second target roaming strategy for the target terminal. Otherwise, it indicates that the target terminal can meet the expected requirements using the first target roaming strategy. Based on the first target roaming strategy, the target terminal is guided to roam to the target AP in the AP network environment.

[0080] Taking the aforementioned uplink and downlink physical rates as examples, if the uplink physical rate is assessed as being below the rate threshold and the downlink physical rate is also assessed as being below the rate limit, it indicates that the target terminal does not meet the set expected requirements using the first target roaming strategy. In this case, the strategy parameters in the first target roaming strategy are adjusted until they meet the expected requirements, thus obtaining the second target roaming strategy for the target terminal. Otherwise, it indicates that the target terminal can meet the expected requirements using the first target roaming strategy. Based on the first target roaming strategy, the target terminal is guided to roam to the target AP in the AP network environment.

[0081] Taking the aforementioned signal strength, packet loss rate, uplink physical rate, and downlink physical rate as examples, they are compared with their respective set thresholds. If any one of them does not exceed the set threshold, it indicates that the target terminal does not meet the set expected requirements using the first target roaming strategy. In this case, the strategy parameters in the first target roaming strategy are adjusted until they meet the expected requirements, thus obtaining the second target roaming strategy for the target terminal. Otherwise, it indicates that the target terminal can meet the expected requirements using the first target roaming strategy. Based on the first target roaming strategy, the target terminal is guided to roam to the target AP in the AP network environment.

[0082] Optionally, the strategy parameters in the first target roaming strategy can be adjusted until the desired requirement is met. This can be achieved using the SARSA algorithm, specifically: based on the target terminal's current first roaming state, after executing a selected first roaming action, the expected reward and the second roaming state at the next moment are obtained; according to the Q function, the second roaming action to be executed by the target terminal in the second roaming state is determined. For example, in a greedy algorithm configured according to the Q function, a second roaming action is selected and executed in the second roaming state; based on the second roaming state, the second roaming action, and the expected reward, the Q function is updated until the updated Q function meets the desired requirement. This desired requirement can refer to the optimal roaming time (e.g., optimal signal strength and physical speed) for the target terminal to roam to the target AP, maximizing the benefit when the target terminal roams to the target AP, using the adopted strategy parameters (such as roaming action).

[0083] The Q function satisfies the following expression: Q(s,a)=Q(s,a)+α(r+γQ(s) ′ a ′ )-Q(s,a)); s is the first roaming state, a is the first roaming action, r is the expected reward, α is the set learning rate, γ is the set discount rate, s' is the second roaming state, and a' is the second roaming action.

[0084] It should be noted that, in addition to the SARSA algorithm mentioned above, other similar reinforcement learning algorithms can also be used to adjust the policy parameters of the roaming strategy, and this application embodiment does not impose any restrictions on this.

[0085] 203: According to the second target roaming strategy, guide the target terminal to roam to the target AP in the AP network environment.

[0086] Optionally, after guiding the target terminal to roam to the target AP in the AP network environment according to the second target roaming policy, the second target roaming policy can also be updated to the terminal profile database so that when the target terminal re-enters the AP network environment, it can be directly guided to roam according to the policy to achieve fast cold start and improve the roaming experience.

[0087] In this embodiment, the historical roaming data of the target terminal under the first target roaming strategy is analyzed. The first target roaming strategy is the roaming strategy assigned to the target terminal when it enters the AP network environment. It is determined whether the target terminal's roaming using the first target roaming strategy meets the expected requirements. If it does not meet the requirements, the strategy parameters in the first target roaming strategy are optimized before guiding the target terminal to roam. Therefore, compared with using a threshold method to guide terminal device roaming, this method takes into account the differences between different terminal devices. It can adaptively adjust the roaming strategy adopted by different terminal devices during the roaming process to maximize roaming benefits, thereby improving the applicability of terminal device roaming.

[0088] Based on the same technical concept, this application also provides a terminal roaming guidance device, which can implement the terminal roaming guidance method flow described above in this application.

[0089] Figure 3 This is a schematic diagram of the structure of a terminal roaming guidance device provided in an embodiment of this application. Figure 3 As shown, the device includes: an evaluation module 301, an adjustment module 302, and a guidance module 303; furthermore, the device may also include an allocation module 304 and a network image construction module 305.

[0090] The evaluation module 301 is used to acquire the historical roaming data of the target terminal under the first target roaming strategy, and determine the roaming evaluation result of the target terminal based on the historical roaming data;

[0091] The adjustment module 302 is used to adjust the strategy parameters in the first target roaming strategy until they meet the expected requirements if the roaming evaluation result does not meet the expected requirements, thereby obtaining the second target roaming strategy for the target terminal.

[0092] The guidance module 303 is used to guide the target terminal to roam to the target AP in the AP network environment according to the second target roaming policy.

[0093] Optionally, the adjustment module 302 is specifically used for:

[0094] Based on the target terminal's current first roaming state, after executing the selected first roaming action, the expected reward and the next roaming state are obtained; according to the Q function, the second roaming action to be executed by the target terminal in the second roaming state is determined; according to the second roaming state, the second roaming action, and the expected reward, the Q function is updated until the updated Q function meets the expected requirements; wherein, the Q function satisfies the following expression: Q(s,a)=Q(s,a)+α(r+γQ(s) ′ a′ )-Q(s,a)); where s is the first roaming state, a is the first roaming action, r is the expected reward, α is the set learning rate, γ is the set discount rate, s' is the second roaming state, and a' is the second roaming action.

[0095] Optionally, the AP network environment includes N APs and a terminal profile database, wherein the terminal profile database includes at least one roaming policy for each terminal in the N APs, and N is an integer greater than 0; the allocation module 304 is specifically used for:

[0096] Identify the terminal type of the target terminal; if a first terminal matching the terminal type is found in the terminal profile database, the roaming policy of the first AP currently associated with the first terminal among the N APs is used as the first target roaming policy of the target terminal; if no first terminal matching the terminal type is found in the terminal profile database, the initial roaming policy of the target terminal is determined based on the mutual scan information of the N APs in the AP network environment and the roaming information of the target terminal, and the initial roaming policy of the target terminal is used as the first target roaming policy of the target terminal.

[0097] Optionally, the network image construction module 305 is used to obtain network information of N APs in the AP network environment, where N is an integer greater than 0; determine the network environment characteristics of the AP network environment based on the network information of the N APs; construct an AP network profile of the AP network environment based on the network environment characteristics of the AP network environment, and store it in the network profile library.

[0098] Optionally, the network image building module 305 is specifically used for:

[0099] The network environment characteristics of the AP network environment are determined based on the coverage information in the network information of the N APs; or based on the distribution information in the network information of the N APs; or based on the jitter drop point areas in the network information of the N APs.

[0100] Optionally, the network image construction module 305 is further configured to: acquire mutual scan information of M APs in the new AP network environment, where M is an integer greater than 0; sort the M APs by importance based on their mutual scan information; construct a mutual scan matrix for the new AP network environment based on the mutual scan signal strength values ​​of the M APs in the importance ranking; generate a new AP network profile for the new AP network environment based on the mutual scan matrix; and if the new AP network profile for the new AP network environment meets the similarity requirement with the AP network profile for the AP network environment in the network profile library, then reuse the terminal profile library for the AP network environment for the new AP network environment.

[0101] It should be noted that the apparatus provided in this application embodiment can implement all the method steps in the above method embodiment and achieve the same technical effect. Therefore, the parts and beneficial effects that are the same as those in the method embodiment will not be described in detail here.

[0102] Based on the same technical concept, this application also provides an electronic device that can realize the function of the aforementioned terminal roaming guidance device.

[0103] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0104] At least one processor 401 and a memory 402 connected to at least one processor 401. In this embodiment, the specific connection medium between the processor 401 and the memory 402 is not limited. Figure 4 The example shown is the connection between processor 401 and memory 402 via bus 400. Bus 400 is... Figure 4 The connections between other components are indicated by thick lines and are for illustrative purposes only, not as limiting information. The 400 bus can be divided into address bus, data bus, control bus, etc., for ease of representation. Figure 4 The term is represented by a single thick line, but this does not imply that there is only one bus or one type of bus. Alternatively, processor 401 can also be called a controller; there is no restriction on the name.

[0105] In this embodiment, memory 402 stores instructions executable by at least one processor 401. By executing the instructions stored in memory 402, at least one processor 401 can execute a terminal roaming guidance method described above. Processor 401 can implement... Figure 3 The functions of each module in the device shown.

[0106] The processor 401 is the control center of the device. It can connect to various parts of the control device through various interfaces and lines. By running or executing instructions stored in memory 402 and calling data stored in memory 402, the processor can perform various functions and process data, thereby monitoring the device as a whole.

[0107] In one possible design, processor 401 may include one or more processing units. Processor 401 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may also not be integrated into processor 401. In some embodiments, processor 401 and memory 402 may be implemented on the same chip; in some embodiments, they may also be implemented separately on separate chips.

[0108] Processor 401 can be a general-purpose processor, such as a central processing unit (CPU), digital signal processor, application-specific integrated circuit, field-programmable gate array or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component, capable of implementing or executing the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of a terminal roaming guidance method disclosed in the embodiments of this application can be directly manifested as execution by a hardware processor, or execution by a combination of hardware and software modules within the processor.

[0109] Memory 402, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules. Memory 402 may include at least one type of storage medium, such as flash memory, hard disk, multimedia card, card-type memory, random access memory (RAM), static random access memory (SRAM), programmable read-only memory (PROM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), magnetic storage, magnetic disk, optical disk, etc. Memory 402 can be any other medium capable of carrying or storing desired program code in the form of instructions or data structures that can be accessed by a computer, but is not limited thereto. In the embodiments of this application, memory 402 can also be a circuit or any other device capable of implementing storage functions for storing program instructions and / or data.

[0110] By designing and programming the processor 401, the code corresponding to the terminal roaming guidance method described in the foregoing embodiments can be embedded into the chip, thereby enabling the chip to execute the code during operation. Figure 2 The illustrated embodiment provides a terminal roaming guidance method. How to design and program the processor 401 is a technique well-known to those skilled in the art and will not be described further here.

[0111] It should be noted that the communication electronic device provided in this application embodiment can implement all the method steps implemented in the above method embodiment and can achieve the same technical effect. Here, the parts that are the same as those in the method embodiment and the beneficial effects will not be described in detail.

[0112] This application also provides a computer-readable storage medium storing computer-executable instructions for causing a computer to execute a terminal roaming guidance method described in the above embodiments.

[0113] This application also provides a computer program product, which, when invoked by a computer, causes the computer to execute a terminal roaming guidance method described in the above embodiments.

[0114] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0115] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0116] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0117] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

Claims

1. A terminal roaming guidance method, characterized in that, include: The system acquires historical roaming data of the target terminal under a first target roaming policy and determines the roaming evaluation result of the target terminal based on the historical roaming data; wherein, the first target roaming policy is the roaming policy assigned to the target terminal when it enters the wireless access point (AP) network environment; If the roaming evaluation result does not meet the expected requirements, the strategy parameters in the first target roaming strategy are adjusted until the expected requirements are met to obtain the second target roaming strategy for the target terminal; wherein, the expected requirements are used to indicate that the adopted strategy parameters are the optimal roaming time for the target terminal to roam to the target AP; According to the second target roaming strategy, the target terminal is guided to roam to the target AP in the AP network environment.

2. The method as described in claim 1, characterized in that, The AP network environment includes N APs and a terminal profile database. The terminal profile database includes at least one roaming policy for each terminal in the N APs, where N is an integer greater than 0. Before acquiring the historical roaming data of the target terminal under the first target roaming strategy, the method further includes: Identify the terminal type to which the target terminal belongs; If a first terminal matching the terminal type is found in the terminal profile database according to the terminal type to which the target terminal belongs, then the roaming strategy of the first AP currently associated with the first terminal among the N APs is taken as the first target roaming strategy of the target terminal. If no first terminal matching the terminal type is found in the terminal profile database according to the terminal type to which the target terminal belongs, then the initial roaming strategy of the target terminal is determined according to the mutual scan information of the N APs in the AP network environment and the roaming information of the target terminal, and the initial roaming strategy of the target terminal is used as the first target roaming strategy of the target terminal.

3. The method as described in claim 1, characterized in that, The method further includes: Obtain network information for N APs in the AP network environment, where N is an integer greater than 0; Based on the network information of the N APs, determine the network environment characteristics of the AP network environment; Based on the network environment characteristics of the AP network environment, an AP network profile of the AP network environment is constructed and stored in the network profile database.

4. The method as described in claim 3, characterized in that, The step of determining the network environment characteristics of the AP network environment based on the network information of the N APs includes: Based on the coverage information in the network information of the N APs, determine the network environment characteristics of the AP network environment; or Based on the distribution information in the network information of the N APs, determine the network environment characteristics of the AP network environment; or Based on the jitter reduction point regions in the network information of the N APs, the network environment characteristics of the AP network environment are determined.

5. The method as described in claim 3, characterized in that, The method further includes: Obtain mutual scan information of M APs in the new AP network environment, where M is an integer greater than 0; sort the M APs by importance based on the mutual scan information of the M APs; Based on the mutual scan signal strength values ​​of the M APs in the importance ranking, construct the mutual scan matrix of the new AP network environment; Based on the mutual scan matrix of the new AP network environment, a new AP network profile of the new AP network environment is generated; If the new AP network profile of the new AP network environment meets the similarity requirement with the AP network profile of the AP network environment in the network profile library, then the terminal profile library in the AP network environment will be reused for the new AP network environment.

6. The method according to any one of claims 1 to 4, characterized in that, Adjusting the strategy parameters in the first target roaming strategy until they meet the desired requirements includes: Based on the target terminal's current first roaming state, after executing the selected first roaming action, the expected reward and the second roaming state at the next moment are obtained; Based on the Q function, the second roaming action performed by the target terminal in the second roaming state is determined; Based on the second roaming state, the second roaming action, and the expected reward, update the Q function until the updated Q function meets the expected requirements; The Q function satisfies the following expression: ; where s is the first roaming state, a is the first roaming action, and r is the expected reward. For the set learning rate, the For the set discount rate, the This is the second roaming state, the This is the second roaming action.

7. A terminal roaming guidance device, characterized in that, include: An evaluation module is used to acquire historical roaming data of the target terminal under a first target roaming policy, and determine the roaming evaluation result of the target terminal based on the historical roaming data; wherein, the first target roaming policy is the roaming policy assigned to the target terminal when it enters the wireless access point (AP) network environment; An adjustment module is used to adjust the strategy parameters in the first target roaming strategy until the expected requirements are met if the roaming evaluation result does not meet the expected requirements, thereby obtaining a second target roaming strategy for the target terminal; wherein, the expected requirements are used to indicate that the adopted strategy parameters are the optimal roaming time for the target terminal to roam to the target AP; The guidance module is used to guide the target terminal to roam to the target AP in the AP network environment according to the second target roaming policy.

8. The apparatus as claimed in claim 7, characterized in that, The AP network environment includes N APs and a terminal profile database, wherein the terminal profile database includes at least one roaming policy for each terminal in the N APs, and N is an integer greater than 0; the device further includes an allocation module; the allocation module is specifically used for: Identify the terminal type to which the target terminal belongs; If a first terminal matching the terminal type is found in the terminal profile database according to the terminal type to which the target terminal belongs, then the roaming strategy of the first AP currently associated with the first terminal among the N APs is taken as the first target roaming strategy of the target terminal. If no first terminal matching the terminal type is found in the terminal profile database according to the terminal type to which the target terminal belongs, then the initial roaming strategy of the target terminal is determined according to the mutual scan information of the N APs in the AP network environment and the roaming information of the target terminal, and the initial roaming strategy of the target terminal is used as the first target roaming strategy of the target terminal.

9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, when executing a computer program stored in the memory, implements the method of any one of claims 1-6.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method of any one of claims 1-6.

Images