Intelligent management method of portable WIFI based on reinforcement learning and portable WIFI

Abstract

The invention relates to the technical field of reinforcement learning, WIFI (Wireless Fidelity), Internet access, related services and the like, and provides an intelligent management method of a portable WIFI based on reinforcement learning and the portable WIFI. The state representation data of the carry-on WIFI comprises a flow use record of the carry-on WIFI of the user, the continuous use duration of the user for connecting the carry-on WIFI each time and the signal intensity of the carry-on WIFI of the user equipment, and inputting the state representation data of the carry-on WIFI into the trained reinforcement learning model, the data transmission rate adaptive to the portable WIFI of the current user is obtained through analysis of the reinforcement learning model, the flow use state of the flow package of the portable WIFI of the current user is intelligently adjusted according to the data transmission rate, and the flow package adaptive to the portable WIFI of the current user is intelligently recommended according to the flow use state. Therefore, the data transmission rate and the flow recommendation of the portable WIFI are optimized, and the product use experience of the user is improved.

Description

Technical Field

[0001] The present invention relates to the technical fields of reinforcement learning, WIFI, Internet access and related services, and in particular to an intelligent management method and system for portable WIFI based on reinforcement learning. Background Art

[0002] As an Internet access device, portable WIFI can connect the connected device of the portable WIFI signal to the Internet through the communication network of the communication operator to achieve data transmission. Before obtaining the communication permission to access the Internet, the user of the connected device of the portable WIFI signal needs to log in to the portable WIFI management system of the portable WIFI operator to purchase the traffic package. Different traffic packages correspond to different traffic and prices. After the traffic package takes effect, in order to stimulate traffic usage, the portable WIFI management system configures the data transmission rate according to the traffic surplus. The larger the traffic surplus, the faster the data transmission rate. However, the portable WIFI management system simply configures the data transmission rate according to the traffic surplus, and fails to fully consider the real-time needs of users and changes in the network environment, resulting in insufficient data transmission rate or waste of resources in some cases. In addition, the recommendation mechanism of the traffic package is relatively simple, and fails to make personalized recommendations based on the user's usage habits and needs, affecting the user experience and the traffic management efficiency of the operator.

[0003] In summary, the existing portable WIFI management technology has technical problems such as insufficient data transmission rate or waste of resources, and failure to make personalized recommendations based on user usage habits and needs. Summary of the invention

[0004] In view of the deficiencies in the above-mentioned prior art, the present invention provides an intelligent management method and system for portable WIFI based on reinforcement learning, so as to optimize the data transmission rate and traffic recommendation of the portable WIFI and enhance the user's product experience.

[0005] In a first aspect, the present invention provides an intelligent management method for portable WIFI based on reinforcement learning, comprising: Monitor the status data of the current user's portable WIFI, wherein the status data of the portable WIFI includes the traffic usage record of the user's portable WIFI, the continuous usage time of each time the user connects to the portable WIFI, and the signal strength of the portable WIFI of the user's device; Inputting the state representation data of the portable WIFI into the trained reinforcement learning model, and obtaining the data transmission rate of the portable WIFI adapted to the current user through analysis by the reinforcement learning model; According to the data transmission rate, the traffic usage status of the traffic package of the current user's portable WIFI is intelligently adjusted, and the traffic package of the portable WIFI suitable for the current user is intelligently recommended according to the traffic usage status.

[0006] In a second aspect, the present invention provides a portable WIFI, which is managed using the above-mentioned intelligent management method of portable WIFI based on reinforcement learning.

[0007] Compared with the prior art, the present invention has the following beneficial effects: The invention provides a portable WIFI intelligent management method based on reinforcement learning and the portable WIFI. The portable WIFI status representation data of the current user is monitored, wherein the portable WIFI status representation data includes the flow usage record of the portable WIFI of the user, the continuous use time of each connection of the portable WIFI by the user, and the signal strength of the portable WIFI of the user device. The portable WIFI status representation data is input into a trained reinforcement learning model, and the data transmission rate of the portable WIFI adapted to the current user is obtained through the reinforcement learning model analysis. According to the data transmission rate, the flow usage status of the flow package of the portable WIFI of the current user is intelligently adjusted, and according to the flow usage status, the flow package of the portable WIFI adapted to the current user is intelligently recommended, thereby optimizing the data transmission rate and flow recommendation of the portable WIFI, and improving the user's product use experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of the present invention. The exemplary embodiments of the present invention and their description are used to explain the present invention and do not constitute an improper limitation of the present invention. Some specific embodiments of the present invention will be described in detail in an exemplary and non-restrictive manner with reference to the drawings. The same reference numerals in the drawings indicate the same or similar components or parts. It should be understood by those skilled in the art that these drawings are not necessarily drawn to scale. In the drawings: Figure 1 It is a flow chart of an intelligent management method of portable WIFI based on reinforcement learning according to an embodiment of the present invention; Figure 2 This is a schematic diagram of a system architecture for counting the continuous usage time of each connection of a user to a portable WIFI according to an embodiment of the present invention; Figure 3 A schematic diagram of a system architecture for counting the signal strength of a portable WIFI of a user device according to an embodiment of the present invention; Figure 4 It is a schematic diagram of the architecture of a server according to an embodiment of the present invention. DETAILED DESCRIPTION

[0009] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiment of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiment of the present invention. Obviously, the described embodiment is only an embodiment of a part of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work should fall within the scope of protection of the present invention.

[0010] Embodiment 1 See also Figure 1-Figure 4 This embodiment provides a portable WIFI intelligent management method based on reinforcement learning, including step S101, step S102 and step S103. Among them, step S101, step S102 and step S103 can be run on the server to implement the portable WIFI intelligent management method based on reinforcement learning, by monitoring the status representation data of the portable WIFI of the current user, the status representation data of the portable WIFI includes the flow usage record of the portable WIFI of the user, the continuous use time of each connection of the portable WIFI by the user, and the signal strength of the portable WIFI of the user's device, the state representation data of the portable WIFI is input into the trained reinforcement learning model, and the data transmission rate of the portable WIFI adapted to the current user is obtained through the reinforcement learning model analysis, and the flow usage status of the flow package of the portable WIFI of the current user is intelligently adjusted according to the data transmission rate, and the flow package adapted to the portable WIFI of the current user is intelligently recommended according to the flow usage status, thereby optimizing the data transmission rate and flow recommendation of the portable WIFI, and improving the user's product experience.

[0011] See also Figure 4 , Figure 4 It should be noted that the server, as the execution subject of all or part of the steps in the intelligent management method of portable WIFI based on reinforcement learning, can execute steps S101, S102 and S103 in this embodiment, and can also run part or all of the steps of the method involved below.

[0012] from Figure 4It can be known that the server can be an information processing device including a memory, a processor and a network interface that are interconnected through a system bus. It can be understood by those skilled in the art that the server here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to a microprocessor, an application-specific integrated circuit (ASIC), a programmable gate array (FPGA), a digital processor (DSP), an embedded device, etc. The server can interact with the user through a touch panel or a voice-controlled device. The memory includes at least one type of readable storage medium, and the readable storage medium includes a flash memory, a hard disk, a multimedia card, a card-type memory (for example, SD or DX memory, etc.), a random access memory (RAM), a static random access memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a disk, an optical disk, etc. In some embodiments, the memory can be an internal storage unit of the server, such as a hard disk or memory of the server. In other embodiments, the memory may also be an external storage device of the server, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the server. Of course, the memory may also include both the internal storage unit of the server and its external storage device.

[0013] See also Figure 1-Figure 4 The intelligent management method of portable WIFI based on reinforcement learning provided in this embodiment includes the following steps: S101, monitoring the status data of the portable WIFI of the current user, wherein the status data of the portable WIFI includes the traffic usage record of the portable WIFI of the user, the continuous usage time of each connection of the portable WIFI by the user, and the signal strength of the portable WIFI of the user's device; S102, inputting the state representation data of the portable WIFI into a trained reinforcement learning model, and obtaining a data transmission rate of the portable WIFI adapted to the current user through analysis by the reinforcement learning model; S103. Intelligently adjust the traffic usage status of the traffic package of the portable WIFI of the current user according to the data transmission rate, and intelligently recommend a traffic package suitable for the portable WIFI of the current user according to the traffic usage status.

[0014] It should be noted that in step S101, the status representation data of the current user's portable WIFI is monitored, including the user's portable WIFI traffic usage record, the continuous usage time of each connection of the user to the portable WIFI, and the signal strength of the portable WIFI of the user's device, so as to obtain the user's usage behavior and current network environment information, reflecting the user's real-time needs and changes in network conditions. In step S102, the state representation data of the current user's portable WIFI monitored is analyzed using a reinforcement learning model to predict and determine the optimal data transmission rate. The reinforcement learning model can learn the user's usage habits and the dynamic changes of the network environment through continuous training and feedback, and adjust the data transmission rate in real time, thereby avoiding the simple rate configuration based on the traffic surplus, improving resource utilization efficiency, and reducing the situation of insufficient or wasteful rate. In step S103, according to the data transmission rate, the usage status of the traffic package is intelligently adjusted, and the adapted traffic package is intelligently recommended to ensure that the user obtains the optimal traffic package, which not only meets its data transmission needs, but also avoids the waste of resources. At the same time, the personalized recommendation mechanism can improve the user experience and improve the traffic management efficiency of the operator.

[0015] In some preferred embodiments, when training the reinforcement learning model, it includes: obtaining statistical data representing the state of the portable WIFI, the state representing data including the flow usage record of the user's portable WIFI, the continuous use time of each connection of the user to the portable WIFI, and the signal strength of the portable WIFI of the user's device; integrating the obtained state representation data of the portable WIFI into a state vector, and defining the adjustment action of the data transmission rate of the portable WIFI corresponding to the state vector; defining a reward function, the reward function evaluates the adjustment of the data transmission rate based on user satisfaction and resource utilization, and the reward function is used to guide the reinforcement learning model to optimize the adjustment strategy of the data transmission rate during the training process; selecting a proximal strategy optimization algorithm as a reinforcement learning algorithm, and training the reinforcement learning algorithm through the state vector, the adjustment action of the data transmission rate and the reward function to obtain a reinforcement learning model that can analyze the adaptive data transmission rate of the current user's portable WIFI. It should be noted that the flow usage record and the use time directly reflect the user's Internet needs and habits. These data can help the model understand the changes in user needs in different time periods and different usage scenarios. Signal strength affects data transmission rate and stability. Understanding the signal strength can help the model adjust its strategy when the network conditions are poor, avoiding excessive rate drops or waste of resources. These state data can provide the model with sufficient information basis, enabling it to make intelligent decisions based on actual usage, rather than relying on static or simple rules. In addition, in this embodiment, multi-dimensional data (traffic usage, usage time, signal strength) are integrated into a unified state vector, so that the model can consider multiple factors at the same time to make decisions. In the form of a state vector, the model can comprehensively consider user behavior and network environment to avoid decision deviations caused by a single factor. In addition, the adjustment action of the data transmission rate of the portable WIFI corresponding to the state vector is defined, so that the model can flexibly adjust the rate under different states, adapt to the real-time needs of users and network changes, improve overall management efficiency, and make the decision space of the model clear, which is convenient for training and optimization strategies. In addition, in this embodiment, a reward function is defined to evaluate the adjustment of the data transmission rate based on user satisfaction and resource utilization. The composition of the reward function includes user satisfaction and resource utilization. User satisfaction can reflect user experience, such as whether the data transmission rate meets user needs, and avoid dissatisfaction caused by insufficient rate. Resource utilization can measure the efficiency of the operator's resource use and avoid resource waste, such as unnecessary bandwidth occupation caused by excessively high rate configuration. The reward function takes into account both user satisfaction and resource utilization to ensure that the model does not waste the operator's resources while improving the user experience. Through the reward function, the model can clearly define the optimization goal and learn how to balance user needs and resource constraints in different states, thereby formulating reasonable strategies.Moreover, the reward function can adjust the strategy according to real-time feedback, so that the model can be continuously optimized in a constantly changing environment. In addition, in this embodiment, the Proximal Policy Optimization (PPO) algorithm is selected as the reinforcement learning algorithm. PPO is a policy gradient method with good sample efficiency and stability, and can perform effective policy optimization in complex high-dimensional spaces. It should also be noted that in this embodiment, through the state vector, the adjustment action of the data transmission rate and the training of the reward function, the model can understand and predict the user's usage pattern and changes in the network environment. This understanding ability enables the model to flexibly adjust the data transmission rate in different scenarios to meet user needs while optimizing resource utilization.

[0016] In some further embodiments, when defining the adjustment action of the data transmission rate of the portable WIFI corresponding to the state vector, it includes: according to a pre-set strategy, the state vector is mapped to a set of predefined data transmission rate adjustment levels and an adjustment level is selected. It should be noted that by mapping the state vector to the predefined adjustment level, the number of possible actions can be reduced, making the decision-making process of the reinforcement learning model more concise, which helps to speed up the training speed and improve the learning efficiency of the model. Moreover, the predefined adjustment level can ensure the efficient use of resources according to the allocation strategy of different traffic packages and network resources, and avoid waste or shortage of resources caused by improper rate adjustment. Exemplarily, it is assumed that the network load is high at a certain moment, and multiple users are connected to the portable WIFI at the same time. User A is in a high-traffic usage state (for example, high-definition video streaming is in progress), and its state vector is mapped to a high-speed level, and a high data transmission rate continues to be maintained to meet the demand. User B is in a medium-traffic usage state, and its state vector is mapped to a medium-speed level to ensure a reasonable data transmission rate while saving network resources. User C is in a low-traffic usage state, and its state vector is mapped to a low-speed level to reduce resource occupation and ensure that more users can obtain a stable connection. In this way, the predefined adjustment levels can not only simplify the decision-making process of the model, but also flexibly and efficiently allocate data transmission rates according to the needs of different users and the actual situation of network resources, avoid resource waste or insufficient rate, and improve the overall network management efficiency and user experience.

[0017] In some further embodiments, when defining the reward function, the user satisfaction is determined by measuring the degree to which the portable WIFI meets the user's data transmission needs at the current data transmission rate; the resource utilization rate is determined by evaluating the resource utilization efficiency of the communication network resources at the current data transmission rate; the user experience score is calculated according to the user satisfaction; the resource efficiency score is calculated according to the resource utilization rate; the user experience score and the resource efficiency score are weighted and summarized to obtain a comprehensive reward value; the comprehensive reward value is used to guide the reinforcement learning model to optimize the adjustment strategy of the data transmission rate during the training process to maximize the comprehensive reward value. It should be noted that by evaluating whether the portable WIFI meets the user's data transmission needs at the current data transmission rate, the user experience is ensured to be unaffected. High user satisfaction means that the user's demand for network services is fully met, avoiding user dissatisfaction caused by insufficient speed. By evaluating the efficiency of the use of communication network resources at the current data transmission rate, it is ensured that network resources are efficiently utilized, avoiding resource waste caused by too high a rate configuration, or insufficient resources caused by too low a rate configuration. Based on the quantitative results of user satisfaction, the quality of user experience is evaluated separately, which can clarify the user's satisfaction at the current rate. Based on the quantitative results of resource utilization, separately evaluating the efficiency of resource use can help operators understand the impact of the current configuration on resource management. By weighted aggregation of user experience scores and resource efficiency scores, it is possible to balance user satisfaction and resource utilization efficiency during the optimization process, allowing weights to be adjusted according to actual needs to adapt to different operating strategies and business goals. The comprehensive reward value, as a feedback signal for the reinforcement learning model, can guide the model to simultaneously optimize user experience and resource management during the training process, thereby achieving a balance between the two goals. In this embodiment, by using the comprehensive reward value as the optimization target, the reinforcement learning model can learn to select the optimal rate adjustment action under different states to maximize the comprehensive evaluation of user satisfaction and resource utilization efficiency.

[0018] In some preferred embodiments, when counting the traffic usage records of the user's portable WIFI, the traffic usage records of the user's portable WIFI are counted by integrating a traffic monitoring module in the firmware of the portable WIFI; or, when counting the traffic usage records of the user's portable WIFI, the traffic usage records of the user's portable WIFI are counted through the network interface module of the portable WIFI. It should be noted that by integrating the traffic monitoring module at the firmware level, the underlying data transmission information of the device can be directly accessed to reduce data loss or errors. The monitoring module at the firmware level can capture and record traffic usage in real time, ensure the timeliness and accuracy of the data, and help to adjust the management strategy in time. In addition, through the network interface module, the data packet information passing through the network interface can be directly obtained to achieve efficient traffic statistics. The network interface module has efficient data processing capabilities, can complete traffic statistics tasks with low latency, and improve the response speed of the overall system.

[0019] See also Figure 1-Figure 4 In some preferred embodiments, the continuous use time of each time the user connects to the portable WIFI is counted, including: when the user device is connected to or disconnected from the portable WIFI, a connection event or a disconnection event is triggered; the connection event and the disconnection event are monitored, and when the connection event and the disconnection event are captured, the current precise time is obtained, and the timestamp corresponding to the connection event and the timestamp corresponding to the disconnection event are generated; according to the connection event and the timestamp corresponding to the connection event, the disconnection event and the timestamp corresponding to the disconnection event, the difference between the time when the user device is connected to and disconnected from the portable WIFI each time is calculated, and the continuous use time of each time the user connects to the portable WIFI is obtained. Specifically, when the user device is connected to or disconnected from the portable WIFI, a connection event or a disconnection event is triggered through the network interface module of the portable WIFI. Specifically, the connection event and the disconnection event triggered by the network interface module are monitored by the event listener integrated in the portable WIFI firmware, and when the event listener captures the connection event and the disconnection event, the system time function is called to obtain the current precise time, and the timestamp corresponding to the connection event and the timestamp corresponding to the disconnection event are generated. Specifically, the session management module of the portable WIFI receives the connection event and its corresponding timestamp transmitted by the event listener, and receives the disconnection event and its corresponding timestamp transmitted by the event listener, so as to calculate the difference between the connection time and the disconnection time of each user device and the portable WIFI, and obtain the continuous usage time of each user connection to the portable WIFI.

[0020] It should be noted that, in this embodiment, when the user device is connected or disconnected from the portable WIFI, the network interface module automatically triggers the corresponding event. These events are captured in real time through the event listener integrated in the firmware. When an event occurs, the system time function is called to generate a corresponding timestamp. By triggering the event immediately when connected or disconnected and recording the timestamp, the calculation of the continuous use time is ensured to be highly real-time and accurate, which helps to accurately reflect the actual use behavior of the user and avoid inaccurate data due to delays or errors. In addition, when an event occurs, the event listener immediately records the event type (connection or disconnection) and its exact time. The captured event and timestamp are passed to the session management module for subsequent time difference calculation. By separating event monitoring from session management, the modularity and maintainability of the portable WIFI system are enhanced. The event listener focuses on capturing and recording events, while the session management module is responsible for data processing and calculation. Accurate continuous use time data can help operators gain an in-depth understanding of users' usage habits and behavior patterns, and support subsequent traffic management and package recommendations. By understanding the actual usage time of users, operators can more effectively allocate network resources, optimize bandwidth usage, and improve overall network efficiency. Statistics on continuous usage time help monitor service quality and promptly identify and resolve issues that may affect user experience, such as frequent disconnections or unstable network conditions.

[0021] See also Figure 1-Figure 4 In some preferred embodiments, when counting the signal strength of the portable WIFI of the user device, it includes: collecting the wireless signal strength data provided by the network operator's cellular base station in real time through the wireless receiving module of the portable WIFI device to obtain the signal strength of the portable WIFI of the user device. It should be noted that the wireless signal strength is an important indicator for measuring the quality of network connection. By collecting signal strength data in real time, it is possible to accurately understand the changes in the current network environment, such as signal weakening or enhancement, network congestion, etc. The network environment is dynamically changing, and real-time monitoring of signal strength can help the model learn to respond to environmental changes in a timely manner, adjust data transmission strategies, and ensure that users always get the best network experience.

[0022] In some preferred embodiments, when the flow usage status of the flow package of the portable WIFI of the current user is intelligently adjusted according to the data transmission rate, it includes: receiving the data transmission rate of the portable WIFI adapted to the current user obtained by the reinforcement learning model analysis, and adjusting the data transmission parameters of the portable WIFI of the current user based on the received data transmission rate, wherein the data transmission parameters include the upload rate and the download rate; real-time monitoring and recording the flow usage of the flow package of the portable WIFI of the current user, wherein the flow usage includes the used flow and the remaining flow; according to the real-time monitoring and recording of the flow usage and the adjusted data transmission rate, dynamically updating the use status of the flow package. It should be noted that the network environment and user needs are dynamically changing. For example, the user's usage mode is different in different time periods, and the network load will also fluctuate accordingly. In this embodiment, based on the adapted data transmission rate output by the reinforcement learning model, the upload rate and the download rate are dynamically adjusted, which can optimize the data transmission in real time and ensure that the best network performance and user experience can be provided in various environments. In addition, the upload rate and the download rate are adjusted to match the user's current data transmission needs and the network resource utilization efficiency, which not only improves the user experience, but also avoids resource waste, and achieves a balance between the two. In addition, by monitoring the user's used and remaining traffic in real time, the user's traffic consumption can be accurately understood. Based on the monitored traffic usage and the adjusted data transmission rate, the usage status of the traffic package can be dynamically updated to ensure the reasonable allocation of resources.

[0023] In some further embodiments, when intelligently recommending a traffic package that is suitable for the portable WIFI of the current user according to the traffic usage status, it includes: analyzing the traffic usage status, the traffic usage status includes the used traffic, the remaining traffic, and the user's usage pattern, the user's usage pattern includes the usage frequency during peak hours and the application type (such as video streaming, online games, file downloads, etc.); intelligently recommending a traffic package that is suitable for the portable WIFI of the current user according to the used traffic, the remaining traffic, and the user's usage pattern. It should be noted that by analyzing the used traffic and the remaining traffic, the user's traffic consumption can be accurately grasped, and it can be identified whether the user is close to the traffic limit, whether the package needs to be upgraded or the usage strategy needs to be adjusted. In addition, the usage frequency during peak hours and the application type (such as video streaming, online games, file downloads, etc.) reflect the specific needs and usage habits of the user. Different usage patterns have different requirements for traffic packages. For example, video streaming requires a higher download rate and a larger traffic package, while online games pay more attention to low latency and stability. In this embodiment, based on the user's specific usage and habits, a suitable traffic package is recommended, which can ensure that the user obtains a better network experience in different usage scenarios, avoid the user's interruption of use due to insufficient traffic, or pay unnecessary fees due to excess traffic, and improve overall user satisfaction. In addition, understanding the user's traffic usage and usage pattern helps operators optimize the allocation of network resources. For example, during peak hours, the transmission rate and traffic allocation can be dynamically adjusted according to the user's actual needs to ensure that the network remains efficient under high load. By monitoring the remaining traffic and user usage habits, unreasonable traffic usage can be identified and adjusted to avoid waste of resources.

[0024] In some further embodiments, when intelligently recommending a data package of a portable WIFI suitable for the current user based on the used data, the remaining data and the user's usage pattern, the method includes: predicting the user's future data demand based on the used data, the remaining data and the user's usage pattern; matching the predicted future data demand of the user with a plurality of preset data package options, so as to obtain data packages of portable WIFI with a high to low matching degree and recommend them to the current user. It should be noted that by analyzing the used data, the remaining data and the usage pattern (such as the usage frequency and application type during peak hours) of the user, the data demand of the user in the future can be accurately predicted. This prediction helps to identify the possible insufficient or excessive data of the user in advance, ensuring that the user can obtain sufficient data support when needed, while avoiding waste of resources. By matching the predicted future data demand with a plurality of preset data package options, the user can obtain the data package that best suits him / her, avoid frequent package upgrades due to insufficient data or unnecessary fees due to excessive data, and optimize the cost-effectiveness of the user. Sorting the traffic package options from high to low according to suitability and generating multiple recommended plans can provide users with flexible choices. This not only improves the accuracy of recommendations, but also gives users more decision-making power to meet the personalized needs of different users.

[0025] Embodiment 2 See also Figure 1-Figure 4 The present embodiment provides a portable WIFI, and the portable WIFI is managed by the intelligent management method of the portable WIFI based on reinforcement learning described in any of the above embodiments. By monitoring the status representation data of the portable WIFI of the current user, the status representation data of the portable WIFI includes the traffic usage record of the portable WIFI of the user, the continuous usage time of each connection of the portable WIFI by the user, and the signal strength of the portable WIFI of the user device, the status representation data of the portable WIFI is input into the trained reinforcement learning model, and the data transmission rate of the portable WIFI adapted to the current user is obtained through the reinforcement learning model analysis. According to the data transmission rate, the traffic usage status of the traffic package of the portable WIFI of the current user is intelligently adjusted, and according to the traffic usage status, the traffic package adapted to the portable WIFI of the current user is intelligently recommended, thereby optimizing the data transmission rate and traffic recommendation of the portable WIFI, and improving the user's product usage experience.

[0026] It should be pointed out that the above embodiments are only preferred specific implementation modes of the present invention, and the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by any technician familiar with the technical field within the technical scope disclosed by the present invention should be covered within the protection scope of the present invention. The protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. An intelligent management method for portable WIFI based on reinforcement learning, characterized in that: include: Monitor the status data of the current user's portable WIFI, wherein the status data of the portable WIFI includes the traffic usage record of the user's portable WIFI, the continuous usage time of each connection of the user to the portable WIFI, and the signal strength of the portable WIFI of the user's device; Inputting the state representation data of the portable WIFI into the trained reinforcement learning model, and obtaining the data transmission rate of the portable WIFI adapted to the current user through analysis by the reinforcement learning model; According to the data transmission rate, the traffic usage status of the traffic package of the current user's portable WIFI is intelligently adjusted, and according to the traffic usage status, a traffic package suitable for the current user's portable WIFI is intelligently recommended.

2. The intelligent management method of portable WIFI based on reinforcement learning as claimed in claim 1, characterized in that: When training the reinforcement learning model, it includes: Obtaining statistical data representing the status of the portable WIFI, wherein the data representing the status includes the flow usage record of the portable WIFI of the user, the continuous usage time of each connection of the portable WIFI by the user, and the signal strength of the portable WIFI of the user's device; Integrate the acquired state representation data of the portable WIFI into a state vector, and define an adjustment action of the data transmission rate of the portable WIFI corresponding to the state vector; defining a reward function, wherein the reward function evaluates the adjustment of the data transmission rate based on user satisfaction and resource utilization, and the reward function is used to guide the reinforcement learning model to optimize the adjustment strategy of the data transmission rate during the training process; A proximal policy optimization algorithm is selected as a reinforcement learning algorithm, and the reinforcement learning algorithm is trained through the state vector, the data transmission rate adjustment action and the reward function to obtain a reinforcement learning model that can analyze the adaptive data transmission rate of the current user's portable WIFI.

3. The intelligent management method of portable WIFI based on reinforcement learning as claimed in claim 2, characterized in that: When counting the traffic usage records of the user's portable WIFI, the traffic usage records of the user's portable WIFI are counted by integrating a traffic monitoring module in the firmware of the portable WIFI; or, when counting the traffic usage records of the user's portable WIFI, the traffic usage records of the user's portable WIFI are counted through the network interface module of the portable WIFI.

4. The intelligent management method of portable WIFI based on reinforcement learning as claimed in claim 2, characterized in that: Statistics on the continuous use time of each time the user connects to the portable WIFI include: When the user device is connected to or disconnected from the portable WIFI, a connection event or a disconnection event is triggered; Listen to the connection event and the disconnection event, and when the connection event and the disconnection event are captured, obtain the current precise time, and generate a timestamp corresponding to the connection event and a timestamp corresponding to the disconnection event; According to the connection event and the timestamp corresponding to the connection event, the disconnection event and the timestamp corresponding to the disconnection event, the difference between the time when the user device is connected to and disconnected from the portable WIFI each time is calculated to obtain the continuous usage time of the user's connection to the portable WIFI each time.

5. The intelligent management method of portable WIFI based on reinforcement learning as claimed in claim 4, characterized in that: When the user device is connected to or disconnected from the portable WIFI, a connection event or a disconnection event is triggered through the network interface module of the portable WIFI.

6. The intelligent management method of portable WIFI based on reinforcement learning as claimed in claim 5, characterized in that: The connection event and disconnection event triggered by the network interface module are monitored by the event listener integrated in the portable WIFI firmware. When the event listener captures the connection event and disconnection event, the system time function is called to obtain the current precise time, and the timestamp corresponding to the connection event and the timestamp corresponding to the disconnection event are generated.

7. The intelligent management method of portable WIFI based on reinforcement learning as claimed in claim 6, characterized in that: The session management module of the portable WIFI receives the connection event and its corresponding timestamp transmitted by the event listener, and receives the disconnection event and its corresponding timestamp transmitted by the event listener, so as to calculate the difference between the connection time and the disconnection time of each user device and the portable WIFI, and obtain the continuous use time of each user connection to the portable WIFI.

8. The intelligent management method of portable WIFI based on reinforcement learning as claimed in claim 2, characterized in that: When counting the signal strength of the portable WIFI of the user device, it includes: collecting the wireless signal strength data provided by the network operator's cellular base station in real time through the wireless receiving module of the portable WIFI device to obtain the signal strength of the portable WIFI of the user device.

9. The intelligent management method for portable WIFI based on reinforcement learning according to any one of claims 1 to 8, characterized in that: According to the data transmission rate, intelligently adjusting the traffic usage status of the current user's portable WIFI traffic package includes: Receiving the data transmission rate of the portable WIFI adapted to the current user obtained by the reinforcement learning model analysis, and adjusting the data transmission parameters of the portable WIFI of the current user based on the received data transmission rate, wherein the data transmission parameters include an upload rate and a download rate; Real-time monitoring and recording of the traffic usage of the current user's portable WIFI traffic package, including the used traffic and remaining traffic; The usage status of the traffic package is dynamically updated based on the traffic usage and the adjusted data transmission rate that are monitored and recorded in real time.

10. A portable WIFI, characterized in that: The portable WIFI is managed using the intelligent management method for portable WIFI based on reinforcement learning as described in any one of claims 1 to 9.

Images