Network switching method based on satellite-ground convergence network and related equipment

Abstract

One or more embodiments of the invention provide a network switching method based on a satellite-ground convergence network and related equipment. The method comprises the following steps: according to user information at the current moment, obtaining predicted received signal strength of a network adjacent to a user and a predicted operation state of the user relative to a corresponding base station; and obtaining an optimal network switching scheme through a deep reinforcement learning model according to the predicted received signal strength and the predicted operation state. According to the technical scheme, the switching offline rate can be effectively reduced, the switching frequency is reduced, the system benefit is effectively improved, and the method is suitable for a high-speed motion scene.

Description

Technical Field

[0001] One or more embodiments of the present application relate to the field of satellite-ground fusion network technology, and more particularly, to a network switching method and related equipment based on a satellite-ground fusion network. Background Art

[0002] With the rapid development of space networks represented by satellite communications, space networks and ground networks have formed two independent communication networks. With the gradual integration of services and the continuous expansion of deployment scenarios, ground networks and space networks have strong complementarity. Facing the urgent needs of the future intelligent connection of all things and global wide-area coverage, in order to achieve more efficient resource sharing, building a satellite-ground fusion network that combines space networks with ground networks has become an important development trend of B5G and 6G communication networks.

[0003] The satellite-ground integrated network refers to a three-dimensional mobile communication network composed of high-orbit satellite networks, medium- and low-orbit satellite networks, airside networks, ground cellular networks, etc., which forms seamless three-dimensional coverage around the world, realizes broadband mobile communications without blind spots, and meets different deployment scenarios and diversified business needs.

[0004] At present, there is an urgent need to propose a high-efficiency and high-yield satellite-ground fusion switching algorithm. Summary of the invention

[0005] In view of this, an object of one or more embodiments of the present application is to propose a method to solve the problems raised by the background technology.

[0006] Based on the above purpose, one or more embodiments of the present application provide a network switching method based on a satellite-ground fusion network, including:

[0007] According to the user information at the current moment, the predicted received signal strength of the user's adjacent network and the predicted operating state of the user relative to the corresponding base station are obtained;

[0008] According to the predicted received signal strength and the predicted operating state, an optimal network switching solution is obtained through a deep reinforcement learning model.

[0009] Optionally, obtaining an optimal network switching solution through a deep reinforcement learning model according to the predicted received signal strength and the predicted operating state includes:

[0010] Determining a candidate switching network according to the predicted received signal strength and the predicted operating state;

[0011] According to the candidate switching networks, an optimal network switching solution is obtained through iterative calculation.

[0012] Optionally, determining a candidate switching network according to the predicted received signal strength and the predicted operating state includes:

[0013] According to the predicted received signal strength and the predicted operating state, the candidate network set t = [number, rss, w] is generated, where number represents the candidate network number, rss represents the predicted received signal strength level of the user's adjacent network, rss∈{1, 2, 3, 4, 5}, w represents the predicted operating state of the user relative to the corresponding base station, w∈{-4, -3, -2, -1, 1, 2, 3, 4}, the absolute value represents the speed, and the positive and negative values ​​represent the operating direction relative to the base station.

[0014] Optionally, obtaining an optimal network switching solution through iterative calculation according to the candidate switching networks includes:

[0015] Taking the candidate switching network as an initial state;

[0016] The following operations are iteratively performed according to the initial state until the reward value converges:

[0017] According to the current round state, executing the current round action, the action representing the selection of the candidate switching network;

[0018] Calculate the reward value R of this round according to the action π (t,b);

[0019] R π (t,b)=s(t,b)+β∑ t′ Q t，t′ (b)W π (t′);

[0020] Where s(t,b) represents the reward function for executing action b in state t, β represents the discount factor, β∈[0,1], Q t，t′ (b) represents the state transition probability, W π (t′) represents the total expected discounted return of the system in state t′;

[0021] According to the state transition probability Q t，t′ (b) Update the next round state t′.

[0022] Optionally, the profit value s(t,b) is calculated by the following formula:

[0023] s(t, b) = α rss V rss (rss b )+α w V w (w b )-D(b);

[0024] Among them, V rss and Vw They represent the utility functions for predicting received signal strength and predicting operating status, respectively. rss min represents the minimum received signal strength required for the user to maintain a normal connection, μ1 represents the normalization parameter, μ1∈[0,1], rss b represents the predicted received signal strength of the user under action b, w b represents the predicted speed of the user relative to the corresponding base station under action b, w represents the absolute speed of the user, W U represents the speed threshold, α rss and α w represents the weight of the utility function attribute, D represents the cost function of action b, 1 represents the ground network cost, represents satellite network cost, Ground Network represents ground network, and Satellite Network represents satellite network.

[0025] Optionally, the predicted received signal strength is calculated using the following formula:

[0026]

[0027] Among them, A lianshan represents a constant, λ lishan represents the average received signal strength, η represents the propagation constant, (y′ u , z′ u ) represents the user's predicted location information, (y′ lishan , z′ lishan ) represents the location information of the base station.

[0028] Optionally, the predicted operating state is calculated by the following formula:

[0029]

[0030] in, represents the user's basic predicted motion state, Φ represents the state transition matrix, L u+1 represents the Kalman gain, a u+1 =In u +w u , w u ~O(0,S),w u Parameter representing the measurement noise, n u represents the user's current motion state, I represents the measurement matrix connecting the measurement value and the state value, and S represents the covariance matrix.

[0031] Based on the same inventive concept, one or more embodiments of the present application further provide a network switching device based on a satellite-ground fusion network, including:

[0032] A prediction module, configured to obtain, based on user information at a current moment, a predicted received signal strength of a neighboring network of the user and a predicted operating state of the user relative to a corresponding base station;

[0033] The decision module is configured to obtain an optimal network switching solution through a deep reinforcement learning model according to the predicted received signal strength and the predicted operating state.

[0034] Based on the same inventive concept, one or more embodiments of the present application also provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, a network switching method based on a satellite-ground fusion network as described in any one of the above is implemented.

[0035] Based on the same inventive concept, one or more embodiments of the present application also provide a non-transitory computer-readable storage medium, which stores computer instructions, and the computer instructions are used to enable the computer to execute any of the above-mentioned network switching methods based on the satellite-ground fusion network.

[0036] From the above, it can be seen that the network switching method based on the satellite-ground fusion network provided by one or more embodiments of the present application obtains the predicted received signal strength of the user's adjacent network and the predicted operating status of the user relative to the corresponding base station based on the user information at the current moment; based on the predicted received signal strength and the predicted operating status, the optimal network switching solution is obtained through a deep reinforcement learning model.

[0037] Through one or more embodiments of the present application, the minimum switching disconnection rate and the number of disconnections can be achieved, which can effectively improve the system revenue, adapt to high-speed motion scenarios, and ensure the continuity of user services.

[0038] The network switching device, electronic device and computer-readable storage medium based on a satellite-ground fusion network provided in the present application can implement the steps of the above-mentioned network switching method based on a satellite-ground fusion network, and therefore also have the beneficial effects of the above-mentioned network switching method based on a satellite-ground fusion network. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate one or more embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only one or more embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0040] Figure 1 A flowchart of a network switching method based on a satellite-ground fusion network according to one or more embodiments of the present application;

[0041] Figure 2 A schematic diagram of the structure of a network switching device based on a satellite-ground fusion network according to one or more embodiments of the present application;

[0042] Figure 3 A schematic diagram of a satellite-ground fusion network structure of one or more embodiments of the present application;

[0043] Figure 4 A schematic diagram of a switching drop rate experiment result according to one or more embodiments of the present application;

[0044] Figure 5 A schematic diagram of the experimental results of the number of switching times when the user rate changes according to one or more embodiments of the present application;

[0045] Figure 6 A schematic diagram of the switching drop rate experiment results under different user arrival rates of one or more embodiments of the present application;

[0046] Figure 7 A schematic diagram of the hardware structure of an electronic device according to one or more embodiments of the present application. DETAILED DESCRIPTION

[0047] In order to make the objectives, technical solutions and advantages of the present disclosure more clearly understood, the present disclosure is further described in detail below in combination with specific embodiments and with reference to the accompanying drawings.

[0048] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in one or more embodiments of the present application should be understood by people with ordinary skills in the field to which the present disclosure belongs. The "first", "second" and similar words used in one or more embodiments of the present application do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "comprise" and similar words mean that the elements or objects appearing before the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0049] As described in the background technology section, the equipment environment status, equipment operation status, users, and power grid management of the power system have the characteristics of time and space distribution. The traditional power system communication method cannot meet the communication needs of the power industry in full coverage and harsh environments. Satellite networks can solve the above problems. Satellite network online monitoring technology can replace or assist patrol personnel in on-site inspections, improve operating efficiency and operation quality, and thus improve power supply reliability. In terms of emergency communications, satellite networks can also provide reliable communication services for the rapid recovery of the power industry. However, satellite communications have the problem of greatly reduced communication quality in cities with high-rise buildings. At this time, it is necessary to use ground networks to improve the service quality of users in buildings, thereby making up for the above-mentioned defects of satellite networks. In addition, ground networks also have the advantages of low latency, low communication cost, and fast speed. Therefore, it is necessary to combine the advantages of satellite networks and ground networks to ensure the communication needs of the power industry anytime and anywhere.

[0050] The satellite-ground fusion network consists of a ground network and a satellite network, where the satellite network can be composed of one or more multi-beam satellites and a ground communication system with a large coverage area. Both the ground network and the satellite network use satellite frequency bands. The satellite-ground fusion network can fully utilize the advantages of the satellite network's wide-area coverage, and can also give play to the advantages of the ground network's low communication latency, high data rate, and low communication cost, thereby meeting the actual application needs of the power industry. In the overlapping coverage area of ​​the ground network and the satellite network, the terminal equipment can communicate with both the satellite network and the ground network, meeting the communication needs of ubiquitous access. The switching brought by the satellite-ground fusion network mainly includes: switching between low-Earth orbit satellites, switching between cells in the ground network, switching between networks, etc.

[0051] With the development of satellite-ground fusion network technology, proposing a high-efficiency and high-yield satellite-ground fusion switching algorithm has become an urgent problem to be solved.

[0052] Therefore, the present application proposes a network switching method based on a satellite-ground fusion network. The network switching method of the satellite-ground fusion network of the present application can be applied to Figure 3 The satellite-ground fusion network architecture of the grid scenario shown in the figure. The satellite-ground fusion network includes some large ground base stations that use large gateway feeder link antennas and radio frequency equipment. The satellite-ground fusion network also includes core switching and routing equipment, which can provide air interface access points based on the Internet Protocol (IP) for the power industry and users, and then establish a connection from the satellite network to the ground network. Mobile satellites have powerful signal reception and transmission capabilities and can easily establish connections with small terminals. In the space communication link, the spot beam provided by the high-power antenna can reconfigure the signal to a high-power mode. This creates convenient conditions for increasing the degree of frequency reuse. The terminal power grid and its users can achieve seamless switching between the ground network and the satellite network. Therefore, in the satellite-ground fusion network, whether the terminal is in the packet domain or the circuit domain, it can efficiently transmit text, images, voice and other data to the target terminal, which provides great convenience for power grid inspection and data transmission. It can be seen that the core features of satellite-ground fusion network technology are: the ground network and satellite network reuse the communication frequency band, and the air interface standards and formats are also compatible, which will not increase the complexity of the terminal; the hardware, size and software standards of the RF module of the handheld device ensure compatibility with the existing ground base station network terminal, and the terminal size in the satellite network is consistent with the device, or even exactly the same; because the handheld device reuses the same frequency band, the complexity of the baseband processing module design and maintenance is reduced, making smooth switching between satellite networks possible; the satellite network has no strict restrictions on the spatial signal modulation method, and the ground wireless communication technology can operate in this system, and the satellite network will not have compatibility problems due to the update and iteration of the ground network technology. Based on the above characteristics, the satellite-ground fusion network can achieve high bandwidth, large capacity, high speed and comprehensive network coverage, and can play an important role in harsh conditions such as remote mountainous areas, emergency power communications and natural disasters.

[0053] refer to Figure 1 The network switching method based on the satellite-ground fusion network of one or more embodiments of the present application includes the following steps:

[0054] Step S101: According to the user information at the current moment, the predicted received signal strength of the neighboring network of the user and the predicted operating state of the user relative to the corresponding base station are obtained.

[0055] Since satellite networks have the characteristics of large changes in user movement speed and extended link time, the user's predicted operating status and the user's predicted received signal strength are first considered in this step. Subsequently, the network switching plan will be calculated based on the above predicted received signal strength and predicted operating status.

[0056] The predicted operating state includes the predicted operating position and the predicted operating speed.

[0057] In some embodiments, the predicted running state of the user may be calculated by a Kalman filter algorithm. One application of the Kalman filter algorithm is to use an existing motion information sequence to predict the motion state of the user at the next moment, so as to track the user's position.

[0058] It can be understood that the user's motion equation in two-dimensional space can be regarded as:

[0059] n u+1 =Φn u +x u , x u ~O(0, R);

[0060] a u+1 =In u +w u , w u ~O(0,S);

[0061] Among them, n u ([y u , z u , w y，u , w z，u ] T ) represents the motion state vector of the user at time u, y u and z u represents the user's location information in two-dimensional space, w y，u and w z，u represents the measurement noise parameter related to the position information, Φ represents the state transition matrix from the current time u to the next time u+1, I is the measurement matrix connecting the measurement value and the state value, x u and w u They represent the parameters related to process noise and measurement noise respectively, and the corresponding covariance matrices are R and S respectively.

[0062] Based on the above principles, user tracking using the Kalman filter algorithm can be divided into two steps: user motion state prediction and posterior error covariance update.

[0063] The formula for calculating the user's predicted motion state using the Kalman filter algorithm is:

[0064]

[0065] in, represents the user's basic predicted motion state, Φ represents the state transition matrix, L u+1 represents the Kalman gain, S represents the covariance matrix, a u+1 =In u +w u , w u ~O(0,S),w u Parameter representing the measurement noise, n u represents the user's current motion state, and I represents the measurement matrix connecting the measurement value and the state value.

[0066] After obtaining the predicted motion state value, the posterior error covariance matrix can be further calculated:

[0067]

[0068] J represents the posterior error covariance matrix.

[0069] In addition to predicting the user's operating status, it is also necessary to predict the user's received signal strength at the next moment as a condition for switching networks.

[0070] It can be understood that the user received signal strength at time u can be expressed as: Y u =Q u +A u Among them, Q u A represents the average power, and its value is determined by the path loss, which is related to the strength of the transmitted signal and the distance between the transmitter and the receiver. u Represents shadow attenuation, and its value is determined by obstacles on the propagation path, such as buildings, number, etc. (the signal strength received by the user is also related to small-scale fading. This application assumes that small-scale fading is eliminated by filtering at the receiving end).

[0071] In this way, the predicted received signal strength of the user in the ground network is calculated by the following formula:

[0072]

[0073] Among them, A lishan represents the shadow fading, expressed as a predicted constant, λ lishan represents the average received signal strength, η represents the propagation constant, (y′ u , z′ u ) represents the location information of user u at time u, (y′ lishan , z′ lishan ) represents the distance between the user and the ground base station.

[0074] According to the characteristics of satellite channels, the fading process can be regarded as a transition between a good channel state and a bad channel state. This transition can be described by a first-order Markov two-state channel model, and the statistical probability density function of each state obeys the Loo distribution. According to the transition probability matrix, the Markov model generates the corresponding state sequence. Each state transition only considers the current state information and the transition matrix.

[0075] Therefore, the predicted received signal strength of the user in the satellite network is calculated by the following formula:

[0076] Y LEO =λ LEO +A LEO ;

[0077] Among them, λ LEO Represents the average received power of the user. Since the distance between the user and the satellite is basically unchanged, the average received power can be expressed by a preset constant. A LEO Indicates the user's received power and represents the situation where the channel is affected by shadow fading.

[0078] Step S102: According to the above-mentioned predicted received signal strength and the above-mentioned predicted operating state, an optimal network switching solution is obtained through a deep reinforcement learning model.

[0079] Deep Reinforcement Learning (DRL) is a combination of reinforcement learning (RL) and deep learning (DL). It optimizes reinforcement learning algorithms through deep learning techniques such as neural networks. DRL is a machine learning method. Reinforcement learning refers to learning an agent in an environment. In the process of continuous interaction with the environment, the agent continuously optimizes its strategy through rewards so that the agent can obtain the maximum cumulative reward in the environment.

[0080] In the technical solution of the present application, the situation of the user's adjacent network in the above-mentioned satellite-ground fusion network can be regarded as a state, and the choice of network switching can be regarded as an action. Through the feedback of the deep reinforcement learning model on the action, the next round of state and action can be continuously iterated and updated to obtain the best strategy.

[0081] In this application, it can be first assumed that the initial state is the adjacent network that the user can select at the sampling time, and the adjacent network includes a satellite network and a ground network. For example, T can be used to represent the state set, t = [number, rss, w] represents the state, where number represents the candidate network number, rss represents the predicted received signal strength level of the user's adjacent network, rss∈{1, 2, 3, 4, 5}, w represents the predicted operating state of the user relative to the corresponding base station, w∈{-4, -3, -2, -1, 1, 2, 3, 4}, the absolute value represents the speed, and the positive and negative represent the running direction relative to the base station.

[0082] The action can be represented by b = {1, ..., O}, which means that the user selects the adjacent network as the switching network. The action space can be represented by y.

[0083] In the process of implementing this application, the applicant found that the total expected discounted rate of return of the system can be expressed as: Among them, F represents the expected value, β u ∈[0,1] represents the discount factor, s(t u , y(t u )) represents the reward function value of the state-action pair.

[0084] The applicant found that the optimal decision set π can be obtained through iterative learning of the deep reinforcement learning model * (t)∈B, according to the optimal decision set, the above problem can be transformed into the optimal criterion problem solved by Bayemann: Among them, s(t,b) represents the reward value of executing action b in state t, β∈[0,1] represents the discount factor, and Q t,t′ (b) represents the state transition probability from state t to state t′ when action b is executed, W * (t′) represents the optimal solution of the total expected discounted return rate under state t′.

[0085] It should be noted that when using deep reinforcement learning algorithms, there is no need to determine s(t,b) and Q t,t′ The specific distribution of (b) can get the optimal solution. At this time, the above formula can be converted into the solution of the reward value: R π (t, b) = s(t, b) + β∑ t′ Q t，t′ (b)W π (t′).

[0086] The applicant found that under the condition of the optimal network switching decision set, the above formula can be optimized as follows: R * (t, b) = s(t, b) + β∑ t′ Q t,t′ (b)Q *(t′),

[0087] After multiple iterations of the deep reinforcement learning model, the reward value at time u+1 has the following relationship with the reward value at time u:

[0088] Among them, δ represents the learning rate, When the number of iterations tends to infinity, δ tends to 0, R u (t,b) converges to R * (t,b). That is, through repeated iterative calculations, when the reward value tends to converge, the optimal network switching solution can be obtained.

[0089] In some embodiments, the profit value s(t, b) is calculated by the following formula: s(t, b) = α rss V rss (rss b )+α w V w (w b )-D(b). Among them, V rss and V w They represent the utility functions for predicting received signal strength and predicting operating status, respectively. rss min represents the minimum received signal strength required for the above user to maintain normal connection, μ1 represents the normalization parameter, μ1∈[0,1], rss b represents the predicted received signal strength of the user under action b, w b represents the predicted speed of the user relative to the corresponding base station under action b, w represents the absolute speed of the user, W U represents the speed threshold, μ2, μ3 and μ4 represent the utility function coefficients, α rss and α w represents the weight of the utility function attribute, D represents the cost function of action b, 1 represents the ground network cost, Indicates the satellite network cost, GroundNetwork indicates the ground network, and Satellit e Network means satellite network.

[0090] In the technical solution of the present application, actions can be obtained through exploration or execution. Here, exploration means that the agent obtains more information by interacting with the environment, while execution means using the currently known information to make the agent perform best. Only one of the two can be selected at the same time.

[0091] When the exploration process is selected, the agent will randomly switch to a candidate network. When the execution process is selected, the agent will choose the network with the highest reward value, that is, choose the best action based on the existing switching decision set. Regardless of which process is selected, the reward value will be updated and stored.

[0092] In order to verify the technical effect of the technical solution of this application, the applicant also conducted the following experiments.

[0093] In order to verify the performance of the proposed network switching algorithm, the applicant conducted relevant simulation experiments on OPNET and Matlab software. The specific simulation environment is: a satellite beam with three base station cells overlapping in the beam. The applicant assumes that the satellite beam radius is 1200km, the cell radius is 3km, the satellite link is modeled as a two-state channel model, the satellite elevation angle is 45°, the low hysteresis threshold of the received signal strength is 28dB, and the decision time interval is 0.5 seconds. The average connection time of the user follows an exponential distribution. The applicant normalizes its average value, and the user arrival rate follows a Poisson distribution. The user speed varies between 1 and 180km / h. The speed threshold is set to 70km / h. The evaluation indicators are the switching drop rate and the number of switching times. The comparison algorithms mainly include Comparison Algorithm 1: Single threshold switching decision method based on received signal strength. Comparison Algorithm 2, hysteresis threshold method based on received signal strength.

[0094] Figure 4 The switching drop rate comparison of different network switching algorithms is shown. Compared with the other two comparison methods, the technical solution of the present application achieves better switching performance and the lowest switching drop rate. Specifically, when the user rate is less than 45km / h, the switching drop rate of the present application is close to 0, the switching drop rate of algorithm 1 is 0.17, and the switching drop rate of algorithm 2 is 0.22. This is because Algorithm 1 and Algorithm 2 use the current received signal strength to make switching decisions. However, the present application makes decisions based on the received signal strength at the next moment predicted by the Kalman filter. Obviously, the present application can effectively reduce the switching drop rate. It is objectively verified that the present application has obvious advantages and performance under the condition of user rate changes. With the continuous increase of user rate, the switching drop rate of the above three network switching algorithms will increase to varying degrees. This is because when the user rate is too high, network switching failures are more likely to occur. However, when the user rate reaches 70km / h, the drop rate of the two comparison algorithms is as high as 0.3 or more, which seriously affects the performance of the satellite-ground fusion network. This will also have an important impact on the safety, reliability and stability of power communications in special situations and remote areas. However, the packet loss rate of this application is only 0.1. It can still provide a more solid guarantee for communications in special areas of the power industry.

[0095] Figure 5The number of switching times of different algorithms when the user rate changes is shown. Compared with the other two comparison methods, this application achieves better switching performance. Specifically, when the user rate is 0-180, the number of switching times of this application is less than 5 times. The highest number of switching times of Algorithm 1 and Algorithm 2 exceeds 30 times. This is because this application not only uses the received signal strength prediction method, but also uses the deep reinforcement learning algorithm to make switching decisions. In addition, it also takes into account future benefits, making the switching decision more accurate and reliable. The reward value in the deep reinforcement learning model takes into account the user's speed, which can effectively reduce the number of switching times in high-speed scenarios. When the speed exceeds the threshold, the user's gain in the ground network decreases, the gain in the satellite network increases, and the user is more inclined to switch to the satellite network. Therefore, when the user's speed is greater than 60km / h, the number of switching times no longer increases. This further verifies that this application has fewer switching times when the user rate increases, thereby ensuring network performance. When the user rate continues to increase, the number of switching times of the switching algorithm we proposed is less than 5 times. However, the number of switching times of Algorithm 1 and Algorithm 2 continues to increase, which may cause network switching failure or poor performance. When the user rate exceeds 100, the number of switching between Algorithm 1 and Algorithm 2 exceeds 20. At this point, the performance of the satellite-ground fusion network has been affected and cannot meet the actual needs of the power industry for communication reliability, security, and stability.

[0096] Figure 6 The comparison of switching drop rates of different network switching algorithms under different user arrival rates is shown, and the present application achieves the lowest switching drop rate. Specifically, the switching drop rate of the present application is less than 0.10. The switching drop rates of Algorithm 1 and Algorithm 2 are both higher than 0.15. This is because Algorithm 1 and Algorithm 2 do not use the Kalman filter method to predict the received signal strength, and cannot optimize the network switching timing based on the prediction results. However, the present application can make decisions based on the signal strength received at the next moment, which can effectively reduce the switching drop rate. This further verifies that the present application has obvious advantages and performance, and provides technical support for reliable and secure communications in the power industry.

[0097] In summary, the technical solution of the present invention can effectively reduce the switching drop rate, reduce the number of switching times, effectively improve the system efficiency, and adapt to high-speed motion scenarios. It allows users to make correct decisions and ensure business continuity.

[0098] It can be understood that the method can be executed by any device, equipment, platform, or device cluster with computing and processing capabilities.

[0099] It should be noted that the method of one or more embodiments of the present application can be performed by a single device, such as a computer or server. The method of this embodiment can also be applied to a distributed scenario and completed by multiple devices cooperating with each other. In the case of such a distributed scenario, one of the multiple devices can only perform one or more steps in the method of one or more embodiments of the present application, and the multiple devices will interact with each other to complete the described method.

[0100] It should be noted that the above describes specific embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0101] Based on the same inventive concept, corresponding to any of the above-mentioned embodiment methods, the present application also provides a network switching device based on a satellite-ground fusion network.

[0102] like Figure 2 As shown, the network switching device based on the satellite-ground fusion network of the present application includes:

[0103] The prediction module 11 is configured to obtain the predicted received signal strength of the user's adjacent network and the predicted operating state of the user relative to the corresponding base station according to the user information at the current moment;

[0104] The decision module 12 is configured to obtain an optimal network switching solution through a deep reinforcement learning model according to the predicted received signal strength and the predicted operating state.

[0105] For the convenience of description, the above devices are described in terms of functions and modules. Of course, when implementing one or more embodiments of the present application, the functions of each module can be implemented in the same or multiple software and / or hardware.

[0106] The apparatus of the above-mentioned embodiment is used to implement the corresponding method in the above-mentioned embodiment, and has the beneficial effects of the corresponding method embodiment, which will not be described in detail here.

[0107] Figure 7A more specific schematic diagram of the hardware structure of an electronic device provided in this embodiment is shown, and the device may include: a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040 are connected to each other through the bus 1050 in the device.

[0108] The processor 1010 can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application.

[0109] The memory 1020 may be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1020 may store an operating system and other application programs. When the technical solution provided in the embodiment of the present application is implemented by software or firmware, the relevant program code is stored in the memory 1020 and is called and executed by the processor 1010.

[0110] The input / output interface 1030 is used to connect the input / output module to realize information input and output. The input / output module can be configured in the device as a component (not shown in the figure), or it can be externally connected to the device to provide corresponding functions. The input device may include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output device may include a display, a speaker, a vibrator, an indicator light, etc.

[0111] The communication interface 1040 is used to connect a communication module (not shown) to realize communication interaction between the device and other devices. The communication module can realize communication through a wired mode (such as USB, network cable, etc.) or a wireless mode (such as mobile network, WIFI, Bluetooth, etc.).

[0112] The bus 1050 includes a path that transmits information between the various components of the device (eg, the processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040).

[0113] It should be noted that, although the above device only shows the processor 1010, the memory 1020, the input / output interface 1030, the communication interface 1040 and the bus 1050, in the specific implementation process, the device may also include other components necessary for normal operation. In addition, it can be understood by those skilled in the art that the above device may also only include the components necessary for implementing the embodiment of the present application, and does not necessarily include all the components shown in the figure.

[0114] The electronic device of the above embodiment is used to implement the corresponding method in the above embodiment, and has the beneficial effects of the corresponding method embodiment, which will not be described in detail here.

[0115] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.

[0116] Those skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present disclosure (including the claims) is limited to these examples. Based on the concept of the present disclosure, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of different aspects of one or more embodiments of the present application as described above, which are not provided in detail for the sake of simplicity.

[0117] In addition, to simplify the description and discussion, and in order not to make one or more embodiments of the present application difficult to understand, the known power / ground connections to the integrated circuit (IC) chip and other components may or may not be shown in the provided drawings. In addition, the device can be shown in the form of a block diagram to avoid making one or more embodiments of the present application difficult to understand, and this also takes into account the fact that the details of the implementation of these block diagram devices are highly dependent on the platform on which one or more embodiments of the present application will be implemented (that is, these details should be fully within the scope of understanding of those skilled in the art). In the case of elaborating specific details (e.g., circuits) to describe exemplary embodiments of the present disclosure, it is obvious to those skilled in the art that one or more embodiments of the present application can be implemented without these specific details or when these specific details are changed. Therefore, these descriptions should be considered illustrative rather than restrictive.

[0118] Although the present disclosure has been described in conjunction with specific embodiments of the present disclosure, many replacements, modifications and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may use the embodiments discussed.

[0119] One or more embodiments of the present application are intended to cover all such substitutions, modifications and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of one or more embodiments of the present application should be included in the scope of protection of this disclosure.

Claims

1. A network switching method based on a satellite-ground fusion network, characterized in that: include: According to the user information at the current moment, the predicted received signal strength of the user's adjacent network and the predicted operating state of the user relative to the corresponding base station are obtained; According to the predicted received signal strength and the predicted operating state, an optimal network switching solution is obtained through a deep reinforcement learning model.

2. The method according to claim 1, characterized in that: The obtaining an optimal network switching solution through a deep reinforcement learning model according to the predicted received signal strength and the predicted operating state includes: Determining a candidate switching network according to the predicted received signal strength and the predicted operating state; According to the candidate switching networks, an optimal network switching solution is obtained through iterative calculation.

3. The method according to claim 2, characterized in that The determining a candidate switching network according to the predicted received signal strength and the predicted operating state includes: According to the predicted received signal strength and the predicted operating state, the candidate network set t = [number, rss, w] is generated, where number represents the candidate network number, rss represents the predicted received signal strength level of the user's adjacent network, rss∈{1,2,3,4,5}, w represents the predicted operating state of the user relative to the corresponding base station, w∈{-4,-3,-2,-1,1,2,3,4}, the absolute value represents the speed, and the positive and negative values ​​represent the operating direction relative to the base station.

4. The method according to claim 2, characterized in that: The step of obtaining an optimal network switching solution through iterative calculation according to the candidate switching networks includes: Taking the candidate switching network as an initial state; The following operations are iteratively performed according to the initial state until the reward value converges: According to the current round state, executing the current round action, the action representing the selection of the candidate switching network; Calculate the reward value R of this round according to the action π (t,b); R π (t,b)=s(t,b)+β∑ t′ Q t,t′ (b)W π (t′); Where s(t,b) represents the reward function for executing action b in state t, β represents the discount factor, β∈[0,1], Q t,t′ (b) represents the state transition probability, W π (t′) represents the total expected discounted return of the system in state t′; According to the state transition probability Q t,t′ (b) Update the next round state t′.

5. The method according to claim 4, characterized in that The profit value s(t,b) is calculated by the following formula: s(t,b)=α rss V rss (rss b )+α w V w (w b )-D(b); Among them, V rss and V w They represent the utility functions for predicting received signal strength and predicting operating status, respectively. rss min represents the minimum received signal strength required for the user to maintain a normal connection, μ1 represents the normalization parameter, μ1∈[0,1], rss b represents the predicted received signal strength of the user under action b, w b represents the predicted speed of the user relative to the corresponding base station under action b, w represents the absolute speed of the user, W U represents the speed threshold, μ2, μ3 and μ4 represent the utility function coefficients, α rss and α w represents the weight of the utility function attribute, D represents the cost function of action b, 1 represents the ground network cost, represents satellite network cost, GroundNetwork represents ground network, and Satellite Network represents satellite network.

6. The method according to claim 1, characterized in that The predicted received signal strength is calculated by the following formula: Among them, A lianshan represents a constant, λ lishan represents the average received signal strength, η represents the propagation constant, (y u ′,z′ u ) represents the user's predicted location information, (y′ lishan ,z′ lishan ) represents the location information of the base station.

7. The method according to claim 1, characterized in that The predicted operating state is calculated by the following formula: in, Indicates the user's basic predicted motion state, Φ represents the state transition matrix, L u+1 represents the Kalman gain, S represents the covariance matrix, a u+1 =In u +w u ,w u ~O(0,S),w u Parameter representing the measurement noise, n u represents the user's current motion state, and I represents the measurement matrix connecting the measurement value and the state value.

8. A network switching device based on a satellite-ground fusion network, characterized in that: include: A prediction module, configured to obtain, based on user information at a current moment, a predicted received signal strength of a neighboring network of the user and a predicted operating state of the user relative to a corresponding base station; The decision module is configured to obtain an optimal network switching solution through a deep reinforcement learning model according to the predicted received signal strength and the predicted operating state.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the method according to any one of claims 1 to 7 is implemented.

10. A non-transitory computer-readable storage medium, characterized in that: The non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to cause the computer to execute any one of claims 1 to 7.

Images