Vehicle-mounted softSIM (Subscriber Identity Module) switching method based on Markov decision

Abstract

The invention belongs to the technical field of Internet of Vehicles, and particularly relates to a Markov decision-based vehicle-mounted softSIM switching method, which comprises the following steps of: monitoring the signal strength corresponding to each softSIM in real time, and when the signal strength of the current softSIM is lower than a set threshold T, switching the softSIM to the current softSIM; monitoring the real-time signal strength, historical data and geographic position of each softSIM except the current softSIM by using an MDP prediction model, predicting the signal change of each softSIM in the future, and determining a switching target softSIM; and executing Ping detection, establishing network connection with the target softSIM in advance before switching, and completing softSIM switching by adopting a switching time delay Tswitch. According to the technical scheme, the future network state can be predicted according to real-time data and historical signal performance, the switching opportunity is intelligently selected, and the network stability of the vehicle in the moving process is ensured.

Description

Technical Field

[0001] The present invention belongs to the technical field of vehicle networking, and in particular relates to a vehicle-mounted softSIM switching method based on Markov decision making. Background Art

[0002] With the development of the Internet of Things and Internet of Vehicles technologies, the demand for networking of vehicle-mounted devices is increasing. Traditional vehicle-mounted devices rely on physical SIM cards or eSIMs for networking. However, when the vehicle moves at high speed or enters complex terrain (such as mountainous areas, tunnels, underground parking lots, etc.), the network coverage of a single operator often cannot guarantee stable signals. To solve this problem, the existing SIM switching technologies mainly include the following categories:

[0003] 1. Manual switching or priority switching:

[0004] Users configure a preferred SIM card based on experience, or the system sets a fixed priority between multiple SIM cards, and only tries to switch to the secondary SIM card when the signal of the preferred SIM card becomes poor. This method lacks intelligence and is prone to triggering switching after network disconnection, resulting in untimely switching.

[0005] 2. Signal strength threshold switching:

[0006] The system detects the signal strength of the SIM card in real time and switches to other operators' networks once it falls below a certain threshold. This method will lead to frequent switching and cannot effectively handle false switching caused by short-term signal fluctuations.

[0007] The prior art has at least the following defects:

[0008] 1) Switching lag: The system often triggers switching when the signal decays or disconnects, and cannot predict the future signal status.

[0009] 2) Frequent switching: Fluctuations in signal strength can easily lead to frequent switching, increasing power consumption and reducing user experience.

[0010] 3) Fixed priority rigidity: The preset priority strategy lacks flexibility and performs poorly in complex scenarios.

[0011] 4) High power consumption: Frequent signal detection will significantly increase power consumption, especially in the case of high-frequency detection. Summary of the invention

[0012] The purpose of the present invention is to provide a vehicle-mounted softSIM switching method based on Markov decision making, which can predict the future network status according to real-time data and historical signal performance, and intelligently select the switching timing to ensure the network stability of the vehicle on the move, so as to solve the problems existing in the above-mentioned background technology.

[0013] To achieve the above objectives, this application is implemented through the following technical solutions:

[0014] A vehicle-mounted softSIM switching method based on Markov decision making comprises the following steps:

[0015] S1. Set an interval to monitor the signal strength of each softSIM. If the signal strength of the set softSIM is lower than the set threshold T, it is marked as a weak signal area.

[0016] S2, using the MDP prediction model to predict the future signal changes of each softSIM using the real-time signal strength, historical data and geographic location of each softSIM monitored in step S1, and determine more than two switching target softSIMs;

[0017] S3, performing Ping detection on the two or more switching target softSIMs determined in step S2, and sorting them according to the detection results;

[0018] S4. Select the target softSIM corresponding to the best Ping detection result in step S3 and switch.

[0019] Furthermore, in step S1, the signal strength corresponding to each softSIM is monitored at an interval determined according to signal fluctuation or vehicle speed.

[0020] Furthermore, a dynamic frequency adjustment strategy is adopted to increase the detection frequency when the signal fluctuates or the vehicle speed is greater than a threshold, and to reduce the detection frequency when the signal fluctuation is less than or equal to the threshold or the vehicle stops.

[0021] Furthermore, in step S1, the signal strength RSSI value formula corresponding to the softSIM is:

[0022] Let Si(t)S_i(t)Si(t) denote the signal strength of the iii-th softSIM at time ttt, then: , where: n is the path loss factor, d and d0 are the current distance and reference distance respectively, P0 is the reference power, is random noise.

[0023] Furthermore, the MDP prediction model in step S2 includes the following steps:

[0024] S21. State space modeling:

[0025] The state space SSS contains the current signal strength, historical fluctuation data and geographic location information of each softSIM. Let the state sts_tst represent the network status at time ttt:

[0026] , where Location t is the current location of the vehicle;

[0027] S22, Action Space Modeling:

[0028] The actions in action space AAA are "keep the current softSIM" or "switch to another softSIM", and the actions are defined as: , where ai represents the switch to the i-th softSIM;

[0029] S23. Reward function R(s,a)R(s, a)R(s,a):

[0030] The principle of the reward function is: if the network is stable and the delay is lower than the set value, a positive reward is given; if the switch causes a network disconnection or the delay is higher than the set value, a negative reward is given;

[0031] S24, state transition probability:

[0032] Use historical data and geographic location to estimate the state transition probability P(s′|s,a),

[0033] ,The state transition probability represents the signal change trend after the current softSIM is selected;

[0034] S25. Strategy Optimization:

[0035] Use the value iteration algorithm to calculate the optimal action in each state and make predictions.

[0036] Furthermore, the reward function in step S23 is defined as:

[0037] Among them, N, B, and D are all real numbers.

[0038] Furthermore, the value iteration algorithm in step S25 is implemented by the Bellman equation, and the formula is:

[0039] , where γ is a discount factor used to balance current rewards and future rewards.

[0040] Furthermore, in step S3, Ping detection determines whether the network is available by collecting delay and packet loss rate, specifically:

[0041] Let PLi be the packet loss rate of the i-th softSIM and RTTi be its average ping delay. If the following conditions are met, the network is considered available:

[0042] PL i < M and RTT i< K, where M is the percentage and K is the time in ms.

[0043] Furthermore, in step S4, a network connection is established with the target softSIM before switching, the data flow is not interrupted during switching, and the softSIM switching is completed using a switching delay Tswitch.

[0044] Furthermore, in step S4, the switching delay Tswitch is determined by the following factors:

[0045] T switch =T detection + T Ping + T HANDOFF , where T detection is the detection time, T Ping T is the Ping pre-detection time. HANDOFF is the actual switching time.

[0046] The beneficial effects of the present invention are:

[0047] This technical solution is based on the Markov decision process and Ping pre-detection switching strategy, which can predict the future network status based on real-time data and historical signal performance, and intelligently select the switching time to ensure the network stability of the vehicle while on the move. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 The figure is a logic diagram of the vehicle-mounted softSIM switching method based on Markov decision making of the present invention. DETAILED DESCRIPTION

[0049] The technical solution of the present invention is described in detail below through examples. The following examples are merely exemplary and can only be used to explain and illustrate the technical solution of the present invention, but cannot be construed as limiting the technical solution of the present invention.

[0050] This technical solution requires dynamic deployment of multiple operator softSIMs, pre-installing multiple softSIMs in the device to cover the networks of different operators, and the system selects the best softSIM to use according to the actual situation. Pre-installing multiple softSIMs in the device is already a prior art and will not be described in detail here. Those skilled in the art can select and implement it according to actual needs.

[0051] The system using the present technical solution generally includes a signal monitoring module, an MDP prediction module, a Ping pre-detection module, a power consumption management module and a network switching module.

[0052] See also Figure 1As shown, the present application provides a vehicle-mounted softSIM switching method based on Markov decision making (MDP), comprising the following steps:

[0053] S1. Real-time detection of the signal strength of all softSIMs in the device, and recording of signal history data. The sampling interval is specifically set, such as 5s, or other required time. In this embodiment, it refers to real-time detection of all softSIMs pre-installed in the vehicle-mounted device, including both the currently used softSIMs and the remaining unused softSIMs. For the setting of the sampling interval, the present application also includes a corresponding determination based on the subsequent vehicle state. Specifically, when the vehicle is stationary, the softSIM in the passing environment is relatively stable, so the sampling interval is relatively long. When the vehicle is in a driving state, especially when the vehicle speed exceeds certain set values, the softSIM will become unstable. At this time, the sampling interval needs to be shortened.

[0054] Monitor the signal strength corresponding to each softSIM, such as RSSI value, the formula is:

[0055] Let Si(t)S_i(t)Si(t) denote the signal strength of the iii-th softSIM at time ttt, then: , where: n is the path loss factor, d and d0 are the current distance and reference distance respectively, P0 is the reference power, is random noise.

[0056] In the vehicle-mounted device, through pre-verification, the lowest limit of softSIM strength corresponding to the signal stability of the relevant device is determined. When it is lower than this limit, the signal of the device is unstable or the signal connection is interrupted. When monitoring the signal strength RSSI value of all softSIMs in the vehicle-mounted device, first determine the signal strength of the currently used softSIM.

[0057] When the signal strength RSSIi(t) of the set softSIM is lower than the threshold T, it is marked as a weak signal area, and the system will try to predict and switch. In other embodiments of the present application, under normal monitoring conditions, although the current softSIM signal strength RSSI value is not lower than the threshold T, when compared with the previous historical data, a continuous attenuation of the signal strength RSSI value can be obtained. The continuous attenuation here is not necessarily an arithmetic attenuation, but it maintains the attenuation trend and continues in this way. The current softSIM signal strength will drop to or below the threshold T. At this time, the system can make predictions and pre-switching preparations, but the actual switching is not performed at this time, and in the actual switching, the switching is not started when the softSIM signal strength is lower than the threshold T, but the switching is started at a set time before the threshold T or at the threshold T.

[0058] At the same time, for other softSIMs that are the target softSIMs to be switched, corresponding signal strength predictions are also performed to avoid the problem that the signal strength of the target softSIM attenuates too quickly after the target softSIM is switched, resulting in frequent signal switching. Instead, predictions are made based on the target softSIM to be switched, ensuring that the signal strength of the softSIM at the time of switching is the best, and the predicted signal strength at a set time in the future is also the best, or the comprehensive evaluation result of the two is the best.

[0059] S2. By using the MDP prediction model, the real-time signal strength, historical data and geographical location of each softSIM monitored in step S1 are used to predict the signal change of each softSIM in the future, and determine more than two switching target softSIMs. This step is to monitor the signal strength values ​​of all the softSIMs to be switched, sort them according to the signal strength values, and then predict the changes of the signal strength values ​​of all the softSIMs to be switched at the future set time, and then select the target softSIM for switching in combination with the switching timing of the current softSIM. Of course, there is also a situation where the signal strength of the predicted target softSIM is good, but the signal strength of the target softSIM changes significantly during the switching, and does not meet the switching conditions. Therefore, all the softSIMs to be switched need to be comprehensively sorted according to the above conditions. When the preferred softSIM is not suitable for switching, the following ones are replaced in turn. In addition, in dynamic monitoring, the preferred softSIM to be switched also changes dynamically.

[0060] In this application, the modeling of the MDP prediction model includes the following steps:

[0061] S21. State space modeling:

[0062] The state space SSS includes the current signal strength, historical fluctuation data and geographic location information of each softSIM. The historical fluctuation data here refers to the multiple monitoring data of the softSIM before the current signal strength monitoring. The storage time of the historical data can be set by those skilled in the art as needed. It can also include the monitoring data when the vehicle passed the same location before. The specific situation is determined according to the storage situation of the storage module, such as the storage time of 1 year, 6 months, 3 months, 1 month, 15 days, 10 days, 5 days, 3 days, 1 day or the middle time of the above time, etc.

[0063] Assume that state sts_tst represents the network state at time ttt:

[0064] , where Location t is the current location of the vehicle, and the current location of the vehicle is given by the real-time data of the on-board navigation system.

[0065] S22, Action Space Modeling:

[0066] The actions in action space AAA are "keep the current softSIM" or "switch to another softSIM", and the actions are defined as: , where ai represents switching to the i-th softSIM. In this step, the i-th softSIM is the relatively optimal softSIM obtained through the above prediction and comprehensive sorting.

[0067] S23. Reward function R(s,a)R(s, a)R(s,a):

[0068] The principle of the reward function is: if the network is stable and the delay is lower than the set value, a positive reward is given; if the switch causes a network disconnection or the delay is higher than the set value, a negative reward is given. The reward function of this application is given when predicting all softSIMs to be switched, not the current softSIM.

[0069] The reward function is defined as:

[0070] Among them, N, B, and D are all real numbers.

[0071] In this embodiment, In other embodiments of the present application, N, B, and D are set accordingly according to specific circumstances or actual needs.

[0072] S24, state transition probability:

[0073] The state transition probability P(s′|s,a) is estimated using historical data and geographic location. The historical data refers to step S21. The geographic location is given by the vehicle navigation system. The specific navigation system used, such as BDS, GPS or GLONASS, is determined by the actual situation.

[0074] , the state transition probability represents the signal change trend after the current softSIM is selected. In this application, if the current softSIM signal strength is above the threshold T and the signal switching does not need to be performed immediately, the state transition probability when the signal switching is required is predicted to be performed when the current softSIM signal strength drops to the threshold T or a set value above the threshold T, so as to ensure that the target softSIM for signal switching is at least the optimal choice.

[0075] S25. Strategy Optimization:

[0076] The value iteration algorithm is used to calculate the optimal action in each state and make predictions. This step is to perform calculations in sequence according to the set monitoring interval time to ensure the continuity of monitoring and the stability of the signal after switching.

[0077] The value iteration algorithm uses the Bellman equation, the formula is:

[0078] , where γ is a discount factor used to balance current rewards and future rewards.

[0079] S3. Ping detection is performed on the two or more switching target softSIMs determined in step S2, and they are sorted according to the detection results. When it is predicted that the current network signal may drop, the network status of other softSIMs is pre-pinged to ensure their availability. This process is a supplement to the MDP prediction model, and is used to avoid the situation where the target softSIM to be switched determined by the MDP prediction model meets the current conditions, but when the signal is switched, the signal of the target softSIM changes and no longer meets the requirements of being the switching target softSIM, so as to timely transform the target softSIM to ensure the stability of the signal after switching.

[0080] Each Ping test collects latency and packet loss to determine whether the network is available. Specifically, let PLi be the packet loss rate of the i-th softSIM and RTTi be its average Ping latency. If the following conditions are met, the network is considered available:

[0081] .

[0082] Ping detection strategy:

[0083] When the signal is predicted to decline, Ping detection of the target softSIM is performed in advance to ensure that the network is available after the handover.

[0084] The present application also includes a dynamic frequency adjustment strategy, which increases the detection frequency when the signal fluctuates or the vehicle speed is greater than a threshold, and reduces the detection frequency when the signal fluctuation is less than or equal to the threshold or the vehicle stops.

[0085] Code example:

[0086] if (Speed ​​> 60 km / h or Signal_Variation > Threshold) {

[0087] Detection_Interval = 5 seconds;

[0088] } else {

[0089] Detection_Interval = 30 seconds;

[0090] This code is only an example and does not mean that only the parameters corresponding to this code are applicable. Those skilled in the art can make corresponding adjustments according to actual needs, which does not affect the implementation of the technical solution of this application.

[0091] S4. Select the target softSIM corresponding to the best Ping detection result in step S3 for switching. Specifically, establish a network connection with the target softSIM in advance before switching, do not interrupt the data flow during switching, and use the switching delay Tswitch to complete the softSIM switching. This switching method is different from the signal switching of the existing technology. The current network signal switching is to cut off the current network signal and then connect to the target network. This switching method, no matter how short the switching time is, will inevitably cause data transmission interruption. The switching delay method of the present application is to first establish a network connection with the target softSIM. At this time, the data flow of the current softSIM is not interrupted. At the same time, the data flow of the target softSIM has begun to be transmitted. When the data flow transmission of the target softSIM is stable, the current softSIM network data flow is cut off.

[0092] The switching delay Tswitch is determined by the following factors:

[0093] T switch =T detection + T Ping + T HANDOFF , where T detection is the detection time, T PingT is the Ping pre-detection time. HANDOFF is the actual switching time.

[0094] This technical solution uses the MDP model to predict future signal changes using real-time signal strength, historical data, and geographic location to achieve intelligent switching. It detects the availability of the target network before switching to avoid false switching caused by short-term signal fluctuations. It uses AI to optimize the detection frequency, reduce unnecessary detection operations, and reduce power consumption while ensuring network stability.

[0095] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A vehicle-mounted softSIM switching method based on Markov decision making, It is characterized in that The following steps are involved: S1. Set an interval to monitor the signal strength of each softSIM. If the signal strength of the set softSIM is lower than the set threshold T, it is marked as a weak signal area. S2, using the MDP prediction model to predict the future signal changes of each softSIM using the real-time signal strength, historical data and geographic location of each softSIM monitored in step S1, and determine more than two switching target softSIMs; S3, performing Ping detection on the two or more switching target softSIMs determined in step S2, and sorting them according to the detection results; S4. Select the target softSIM corresponding to the best Ping detection result in step S3 and switch.

2. The vehicle-mounted softSIM switching method based on Markov decision making according to claim 1, It is characterized in that In step S1, the signal strength corresponding to each softSIM is monitored at intervals determined according to signal fluctuations or vehicle speed.

3. The vehicle-mounted softSIM switching method based on Markov decision making according to claim 2, It is characterized in that A dynamic frequency adjustment strategy is adopted to increase the detection frequency when the signal fluctuates or the vehicle speed is greater than the threshold, and to reduce the detection frequency when the signal fluctuation is less than or equal to the threshold or when the vehicle stops.

4. The vehicle-mounted softSIM switching method based on Markov decision making according to claim 1, It is characterized in that In step S1, the signal strength RSSI value formula corresponding to softSIM is: Let Si(t)S_i(t)Si(t) denote the signal strength of the iii-th softSIM at time ttt, then: , where: n is the path loss factor, d and d0 are the current distance and reference distance respectively, P0 is the reference power, is random noise.

5. The vehicle-mounted softSIM switching method based on Markov decision making according to claim 1, It is characterized in that The MDP prediction model in step S2 includes the following steps: S21. State space modeling: The state space SSS contains the current signal strength, historical fluctuation data and geographic location information of each softSIM. Let the state sts_tst represent the network status at time ttt: , where Location t is the current location of the vehicle; S22, Action Space Modeling: The actions in action space AAA are "keep the current softSIM" or "switch to another softSIM", and the actions are defined as: , where ai represents the switch to the i-th softSIM; S23. Reward function R(s,a)R(s, a)R(s,a): The principle of the reward function is: if the network is stable and the delay is lower than the set value, a positive reward is given; if the switch causes a network disconnection or the delay is higher than the set value, a negative reward is given; S24, state transition probability: Use historical data and geographic location to estimate the state transition probability P(s′|s,a), ,The state transition probability represents the signal change trend after the current softSIM is selected; S25. Strategy Optimization: Use the value iteration algorithm to calculate the optimal action in each state and make predictions.

6. The vehicle-mounted softSIM switching method based on Markov decision making according to claim 5, It is characterized in that The reward function in step S23 is defined as: Among them, N, B, and D are all real numbers.

7. The vehicle-mounted softSIM switching method based on Markov decision making according to claim 5, It is characterized in that The value iteration algorithm in step S25 uses the Bellman equation, and the formula is: , where γ is a discount factor used to balance current rewards and future rewards.

8. The vehicle-mounted softSIM switching method based on Markov decision making according to claim 1, It is characterized in that In step S3, Ping detection determines whether the network is available by collecting delay and packet loss rate, specifically: Let PLi be the packet loss rate of the i-th softSIM and RTTi be its average ping delay. If the following conditions are met, the network is considered available: PL i < M and RTT i < K, where M is the percentage and K is the time in ms.

9. The vehicle-mounted softSIM switching method based on Markov decision making according to claim 1, It is characterized in that In step S4, a network connection is established with the target softSIM before switching, the data flow is not interrupted during switching, and the softSIM switching is completed using a switching delay Tswitch.

10. The vehicle-mounted softSIM switching method based on Markov decision making according to claim 9, It is characterized in that In step S4, the switching delay Tswitch is determined by the following factors: T switch =T detection + T Ping + T HANDOFF , where T detection is the detection time, T Ping is the Ping pre-detection time, T HANDOFF is the actual switching time.

Images