A LTE and WiFi switching method using deep learning network

Abstract

The present invention belongs to the technical field of Internet of Vehicles, and specifically relates to a method for switching between LTE and WiFi using a deep learning network. By collecting the strength, delay and packet loss rate data of LTE and WiFi signals in real time, combined with GPS and IMU, the geographical location and motion state of the device are judged, and the deep reinforcement learning algorithm is used to select the best network switching strategy according to the current state, execute the switching operation, ensure senseless switching, store the network status and switching history, and periodically update the DQN model. This technical solution combines the vehicle navigation system and IMU, and the system can identify specific scenarios in advance and predict network switching. Through the DQN algorithm, the system has self-learning capabilities and can adapt to different environments. It uses dual network concurrent connections to ensure that users do not perceive the switching process, and gives priority to using the WiFi network to save LTE traffic and avoid instability caused by signal fluctuations.

Description

Technical Field

[0001] The present invention belongs to the technical field of vehicle networking, and in particular relates to a LTE and WiFi switching method using a deep learning network. Background Art

[0002] In recent years, the demand for networking of in-vehicle devices has continued to increase. Car owners hope to enjoy high-quality network connections in their vehicles, especially in navigation, streaming media playback and remote monitoring scenarios. Existing in-vehicle network technologies mostly use LTE or Wi-Fi connections, both of which have their own limitations:

[0003] Although LTE networks have wide coverage, signals are easily attenuated or even interrupted in underground parking garages, closed buildings or remote areas. In addition, the cost of using LTE networks is high, especially in traffic-intensive applications, which may incur high fees.

[0004] The coverage of WiFi signals is limited, usually confined to homes or companies, and it is impossible to always maintain a connection in mobile scenarios. In addition, some WiFi networks are subject to access control, and the connection establishment process requires authentication time, which affects the switching experience.

[0005] At the same time, some devices also default to using LTE or WiFi networks instead of dynamically judging the signal quality, causing these devices to remain on the original network when the signal is poor, affecting the user experience. In order to overcome the above problems, signal threshold switching technology has also been mentioned, but all current network switching solutions rely on a single signal threshold judgment. When the signal threshold fluctuates up and down, frequent switching may be triggered, resulting in network jitter and interruption. In addition, due to the lack of scene perception capabilities, existing technical solutions cannot intelligently identify the environment and needs of the device (such as parking status and driving status), and cannot make adaptive adjustments.

[0006] On the other hand, all existing technologies use the method of first disconnecting the current network and then connecting to the target network to perform network switching. During the network switching process, data transmission interruption is inevitable. Moreover, the existing network switching technologies only switch after the current network signal is lower than the threshold. It is impossible to predict the network signal and prepare for network switching in advance to ensure smooth network switching and minimize the impact on data transmission. Summary of the invention

[0007] The object of the present invention is to provide a LTE and WiFi switching method using a deep learning network to solve the problems existing in the above-mentioned background technology.

[0008] To achieve the above objectives, this application is implemented through the following technical solutions:

[0009] A method for switching between LTE and WiFi using a deep learning network, comprising the following steps:

[0010] S1, the scene recognition module determines the geographical location and motion status of the device and determines the signal monitoring interval according to the set conditions;

[0011] S2, the signal monitoring module collects the strength, delay and packet loss rate of LTE and WiFi signals, and predicts the future current network signal strength and target network signal strength based on historical data, and judges that if the future current network signal strength will be lower than the set threshold, then enter step S3, if the future current network signal strength is above the set threshold, maintain the current network connection;

[0012] S3, judging the target network signal strength, if the future target network signal strength is within the set range, or the time difference between the future target network signal strength being lower than the set threshold and the future current network signal strength being lower than the set threshold is greater than the set time, then proceeding to step S4;

[0013] S4. Before the current network signal strength reaches a set threshold, the current network is switched to the target network through a network switching management mechanism according to a set network switching strategy.

[0014] Furthermore, in step S1, the scene recognition module combines the vehicle navigation system and the IMU unit to determine the geographic location and motion status of the device.

[0015] Furthermore, the motion state includes a parking state, a state where the vehicle speed is within a set threshold, and a state where the vehicle speed is greater than a set threshold.

[0016] Furthermore, in step S3, if the time difference between the future target network signal strength being lower than the set threshold and the future current network signal strength being lower than the set threshold is less than the set time, then step S5 is entered;

[0017] S5. Before the current network signal strength drops to a set threshold, connect to the target network to maintain the dual network connection state.

[0018] Furthermore, in step S4, the network switching management mechanism includes the following steps:

[0019] S41, pre-connection, according to the set network switching strategy and switching window mechanism, connecting to the target network before the current network signal strength drops to the set threshold;

[0020] S42, maintain dual network signal connection to prevent data loss;

[0021] S43. When the target network connection is stable, disconnect the original network connection.

[0022] Furthermore, the switching window mechanism adopts a switching buffer window Tswitch method, and the formula is:

[0023] Wherein, λ is the switching frequency control coefficient, and Var(RSSI) is the variance of the signal strength.

[0024] Furthermore, in step S4, the network switching strategy is implemented by establishing an MDP model:

[0025] Building the MDP model includes the following steps:

[0026] 1) State space S:

[0027] S = {s1, s2, ..., sn} represents the current state of the vehicle-mounted device, including: current signal strength: RSSI current, target signal strength: RSSI target, vehicle speed v: stationary or driving state, device location: coordinates obtained by the vehicle-mounted navigation system;

[0028] 2) Action space A:

[0029] A = {switch to current network, switch to target network, keep current network};

[0030] 3) Reward function R(s,a):

[0031] The expression is as follows: R(s,a) = W1×stability reward + W2×traffic saving-W3×frequent switching penalty, where W1, W2, and W3 are the weights of adjusting each factor;

[0032] Furthermore, the MDP model is confirmed by DQN, which is based on the learned Bellman equation, with the formula:

[0033] Among them, α is the learning rate, a' represents an action selected from all possible actions in the next state s', s' represents the next state the system enters after executing action a in the current state s, r is the reward, which is used to control the update speed of the Q value, and γ is the discount factor, which determines the impact of future rewards.

[0034] Furthermore, DQN uses a deep neural network as an approximate function, and its loss function is defined as:

[0035] Among them, θ is the parameter of the current network, θ - are the parameters of the target network.

[0036] Furthermore, it also includes data storage structure and model training, including:

[0037] The data storage structure includes historical network data and experience replay cache;

[0038] The model training and updating is to sample data from the experience replay buffer at set intervals to train the model, and use the above loss function to update the DQN network parameters.

[0039] The beneficial effects of the present invention are:

[0040] This technical solution combines the vehicle navigation system and IMU. The system can identify specific scenarios in advance and predict network switching. Through the DQN algorithm, the system has self-learning capabilities and can adapt to different environments. It uses dual network concurrent connections to ensure that users are unaware of the switching process, and gives priority to using WiFi networks to save LTE traffic and avoid instability caused by signal fluctuations. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 Logic diagram of the LTE and WiFi switching method using a deep learning network in the present invention. DETAILED DESCRIPTION

[0042] The technical solution of the present invention is described in detail below through examples. The following examples are merely exemplary and can only be used to explain and illustrate the technical solution of the present invention, but cannot be construed as limiting the technical solution of the present invention.

[0043] The technical solution of this application is that the system realizes the intelligent switching of vehicle-mounted equipment between LTE and WiFi networks through the coordinated cooperation of software and hardware. The deep Q network (DQN) algorithm is used for network selection, which collects real-time signals, monitors device status, intelligently analyzes the environment, and seamlessly switches networks to improve user experience and system performance.

[0044] See also Figure 1 , the present application provides a method for switching between LTE and WiFi using a deep learning network, comprising the following steps:

[0045] S1. The scene recognition module determines the geographical location and motion state of the device, and determines the signal monitoring interval according to the set conditions; the scene recognition module of the present application combines the vehicle navigation system (such as GPS) and the inertial measurement unit IMU to determine the geographical location and motion state of the device and determine the signal monitoring interval. For example: if the vehicle is in a stopped state, whether it is an LTE network or a WiFi network, its signal transmission is relatively stable. At this time, a long interval time can be selected for monitoring. When the vehicle is in motion, such as under the set speed condition, the network connection will be unstable, so the monitoring interval time should be shortened. Here is an example, such as when the speed is ≥60Km / h, monitoring is performed every 5s. When the vehicle speed is <60Km / h or stopped, monitoring is performed every 30s. The above is only an example, and those skilled in the art can set the interval time and set conditions according to actual needs.

[0046] S2, the signal monitoring module collects the strength, delay and packet loss rate of LTE and WiFi signals, and predicts the future current network signal strength and target network signal strength in combination with historical data, and judges that if the future current network signal strength will be lower than the set threshold, then enter step S3, if the future current network signal strength is above the set threshold, maintain the current network connection. It should be noted that if the current network is an LTE network, considering the cost, if an available WiFi signal is monitored, even if the current LTE network is stable, the network can be switched to a WiFi signal to avoid additional costs due to traffic.

[0047] In the present application, the current network signal can be a non-LTE network and a WiFi network, or other usable network. Of course, the current network signal can also be an LTE signal or a WiFi signal, that is, the technical solution of the present application can also be used to implement switching from one network to another.

[0048] S3. Determine the target network signal strength. If the future target network signal strength is within the set range, or the time difference between the future target network signal strength being lower than the set threshold and the future current network signal strength being lower than the set threshold is greater than the set time, proceed to step S4.

[0049] In this step, if the time difference between the future target network signal strength being lower than the set threshold and the future current network signal strength being lower than the set threshold is less than the set time, then the process proceeds to step S5;

[0050] Step S5 is to connect to the target network before the current network signal strength drops to the set threshold, maintain the dual network connection state, but do not switch the network, only to avoid data interruption. Although the target network strength can now meet the needs of signal switching, after the target network is switched to the current network, its network has large signal fluctuations or the signal strength will drop below the threshold in the future. In addition, the time from the current signal strength to below the threshold is short, lower than the set time. If a signal switch is performed, the signal will be switched again in a short time, resulting in frequent signal switching. By connecting to the network but not switching, the data stream transmission can be maintained without interruption for a short period of time, thereby ensuring the data transmission problem when the current network signal strength is low and the signal strength of the target network is also low.

[0051] S4. Before the current network signal strength reaches a set threshold, the current network is switched to the target network through a network switching management mechanism according to a set network switching strategy.

[0052] In the present application, under actual circumstances, through the monitoring of the signal monitoring module, the current network has the following situations. The first situation is that the current network is stable in the current scenario, and there is no need to switch the network at this time. However, if the current network is a charged grid and the target network includes a free network or a network with a lower fee, network switching can also be performed; the second situation is that the grid signal strength of the current network has dropped below the set threshold during the monitoring interval, and network switching is required immediately. This technical solution is actually a technology developed to avoid such a situation; the third situation is that the current network signal is above the set threshold, but through the prediction after monitoring, the network signal is declining or drops below the set threshold at some time in the future. At this time, network switching is required within the set time.

[0053] The network switching management mechanism includes the following steps:

[0054] S41, pre-connection, according to the set network switching strategy and switching window mechanism, connect to the target network before the current network signal strength drops to the set threshold. This network switching method is essentially different from the current network switching method. The current network switching method is to first disconnect the current network and then connect to the target network. However, no matter how short the switching time is, data flow transmission will be interrupted.

[0055] S42, maintain dual network signal connection to prevent data loss. At this time, although the signal strength of the current network is low, it still ensures a certain amount of data flow transmission. At the same time, it is connected to the target network, and the data flow of the target network is transmitted to the device at the same time, thereby avoiding data loss.

[0056] S43. When the target network connection is stable, disconnect the original network connection.

[0057] The switching window mechanism adopts the switching buffer window Tswitch method, and the specific formula is:

[0058] Wherein, λ is the switching frequency control coefficient, and Var(RSSI) is the variance of the signal strength.

[0059] In this application, in order to select the optimal network switching strategy in complex scenarios, the network switching strategy is determined by establishing an MDP model and DQN:

[0060] The MDP model is used to predict the future signal strength changes of the current network, and the future signal strength changes of the target network are also predicted synchronously. First, the timing of network switching is determined, and secondly, the target network is prevented from switching to a target network with frequent signal strength changes within a set time in the future, which may lead to network instability after switching.

[0061] In this application, establishing the MDP model includes the following steps:

[0062] 1) State space S:

[0063] S = {s1, s2, ..., sn} represents the current state of the vehicle-mounted device, including: LTE signal strength: RSSILTE, WiFi signal strength: RSSIWiFi, vehicle speed v: stationary or driving state, device location: coordinates obtained by GPS. Similarly, if the current network is other networks, it is also determined according to the above conditions.

[0064] 2) Action space A:

[0065] A={switch to LTE, switch to WiFi, keep current network}.

[0066] 3) Reward function R(s,a):

[0067] The expression is as follows: R(s,a) = W1×stability reward + W2×traffic saving-W3×frequent switching penalty, where W1, W2, and W3 are weights for adjusting various factors. For the reward function, those skilled in the art can change the corresponding parameters according to the actual situation.

[0068] DQN is based on the learned Bellman equation, which is used to estimate the value of state-action pairs. The formula is:

[0069] Among them, α is the learning rate, a' represents the action we choose from all possible actions in the next state s'. This action is used to calculate the maximum Q value, that is, to select the next optimal action to obtain the maximum expected reward, s' represents the next state the system enters after executing action a in the current state s, that is, s' is the new state of the environment feedback after executing action a, r is the reward, which is used to control the Q value update speed, and γ is the discount factor, which determines the impact of future rewards. The specific Q value update speed can be the same as the acquisition interval of the signal monitoring module, or it can be less than the acquisition interval of the signal monitoring module.

[0070] DQN uses deep neural networks as approximate functions to solve optimization problems in large-scale state spaces. Its loss function is defined as:

[0071] Among them, θ is the parameter of the current network, θ - are the parameters of the target network.

[0072] This application also includes a network switching probability model, which uses a Softmax function to calculate the probability of switching to a certain network. Assuming that the signal strengths of WiFi and LTE are RSSIWiFi and RSSILTE respectively, the probability of switching to WiFi PWiFiP_{WiFi}PWiFi is:

[0073] Among them, β controls the impact of signal strength on the switching probability. When RSSIWiFi is high, PWiFi→1, and the system tends to choose the WiFi network.

[0074] Calculation example of network switching probability model:

[0075] Assume that the current device is in state s1, WiFi signal RSSIWiFi = -50dBm, LTE signal RSSILTE = -70dBm. Calculate the probability of switching to WiFi:

[0076]

[0077] If β=0.1, then:

[0078] Therefore, the system tends to switch to the WiFi network.

[0079] This application also includes data storage structure and model training, including:

[0080] Historical network data is used to record signal strength, network status and location information. The historical network data in this application is based on the storage structure, multiple monitoring data before this signal strength monitoring, and the specific storage time of historical data can be set by technicians in this field as needed. It can also include monitoring data when the vehicle passed the same location before. The specific situation is determined according to the storage situation of the storage module, such as the storage time of 1 year, 6 months, 3 months, 1 month, 15 days, 10 days, 5 days, 3 days, 1 day, 6h, 3h, 1h, 30min, 15min or the middle time of the above time, etc.

[0081] Replay Buffer: stores state-action-reward triplets.

[0082] Model training and updating are:

[0083] The system samples data from the Replay Buffer at regular intervals to train the model. The model here is the training of the MDP prediction model to improve the prediction accuracy of the MDP prediction model.

[0084] In this application, the following loss function is used to update the DQN network parameters:

[0085] In addition, by periodically synchronizing the parameters θ of the target network - ←θ, ensuring the stability of the training process.

[0086] This technical solution adopts the deep reinforcement learning (DQN) algorithm and modular design to realize the intelligent switching of vehicle-mounted equipment between LTE and WiFi networks. Through the Softmax probability model and seamless switching mechanism, the system ensures the stability of the network and the improvement of user experience. The formula-derived strategy ensures that the device selects the best network in different scenarios, improves networking performance, and reduces operating costs.

[0087] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for switching between LTE and WiFi using a deep learning network, characterized in that: The following steps are involved: S1, the scene recognition module determines the geographical location and motion status of the device and determines the signal monitoring interval according to the set conditions; S2, the signal monitoring module collects the strength, delay and packet loss rate of LTE and WiFi signals, and predicts the future current network signal strength and target network signal strength based on historical data, and judges that if the future current network signal strength will be lower than the set threshold, then enter step S3, if the future current network signal strength is above the set threshold, maintain the current network connection; S3, judging the target network signal strength, if the future target network signal strength is within the set range, or the time difference between the future target network signal strength being lower than the set threshold and the future current network signal strength being lower than the set threshold is greater than the set time, then proceed to step S4; and if the time difference between the future target network signal strength being lower than the set threshold and the future current network signal strength being lower than the set threshold is less than the set time, then proceed to step S5; S4. Before the current network signal strength reaches a set threshold, the current network is switched to the target network through a network switching management mechanism according to a set network switching strategy; The network switching management mechanism includes the following steps: S41, pre-connection, according to the set network switching strategy and switching window mechanism, connecting to the target network before the current network signal strength drops to the set threshold; S42, maintain dual network signal connection to prevent data loss; S43, when the target network connection is stable, disconnect the original network connection; S5. Before the current network signal strength drops to a set threshold, connect to the target network to maintain the dual network connection state.

2. The LTE and WiFi switching method using a deep learning network according to claim 1, characterized in that: In step S1, the scene recognition module combines the vehicle navigation system and the IMU unit to determine the geographical location and motion status of the device.

3. The LTE and WiFi switching method using a deep learning network according to claim 2, characterized in that: The motion state includes the parking state, the state where the vehicle speed is within the set threshold, and the state where the vehicle speed is greater than the set threshold.

4. The LTE and WiFi switching method using a deep learning network according to claim 1, characterized in that: The switching window mechanism adopts the switching buffer window Tswitch method, and the formula is: Wherein, λ is the switching frequency control coefficient, and Var(RSSI) is the variance of the signal strength.

5. The LTE and WiFi switching method using a deep learning network according to claim 1, characterized in that: In step S4, the network switching strategy is implemented by establishing an MDP model: Building an MDP model includes the following steps: 1) State space S: S = {s1, s2, ..., sn} represents the current state of the vehicle-mounted device, including: current signal strength: RSSI current, target signal strength: RSSI target, vehicle speed v: stationary or driving state, device location: coordinates obtained by the vehicle-mounted navigation system; 2) Action space A: A = {switch to current network, switch to target network, keep current network}; 3) Reward function R(s,a): The expression is as follows: R(s,a) = W1×stability reward+W2×traffic saving-W3×frequent switching penalty, where W1, W2, and W3 are the weights for adjusting each factor.

6. The LTE and WiFi switching method using a deep learning network according to claim 5, characterized in that: The MDP model is determined by DQN, which is based on the learned Bellman equation, and the formula is: Among them, α is the learning rate, a' represents an action selected from all possible actions in the next state s', s' represents the next state the system enters after executing action a in the current state s, r is the reward, which is used to control the update speed of the Q value, and γ is the discount factor, which determines the impact of future rewards.

7. The LTE and WiFi switching method using a deep learning network according to claim 6, characterized in that: DQN uses a deep neural network as an approximate function, and its loss function is defined as: Among them, θ is the parameter of the current network, θ - are the parameters of the target network.

8. The LTE and WiFi switching method using a deep learning network according to claim 7, characterized in that: It also includes data storage structure and model training, including: The data storage structure includes historical network data and experience replay cache; The model training and updating is to sample data from the experience replay buffer at set intervals to train the model, and use the above loss function to update the DQN network parameters.

Images