High-speed mobile terminal switching method, device, equipment, medium and program product

Abstract

The invention provides a high-speed mobile terminal switching method and device, equipment, a medium and a program product, and relates to the technical field of communication, and the method comprises the steps: constructing a terminal switching model between a high-speed mobile terminal and a low-orbit satellite, obtaining a tetrad for the interaction between the terminal and the low-orbit satellite, storing the tetrad in an experience pool, the state set of the tetrad comprises coverage information and at least two items of residual service time, an idle channel and a user signal-to-noise ratio, extracting a training sample, performing iterative training on an inter-satellite switching decision-making process through a strategy network and a value network, and obtaining an inter-satellite switching decision-making result; and determining loss function values of the strategy network and the value network according to the reward function and the action value function, obtaining an optimal inter-satellite switching strategy set of the terminal switching model when training of the strategy network and the value network is completed, and performing inter-satellite switching according to the optimal inter-satellite switching strategy set. And multiple target training is considered, and the optimal switching strategy set is obtained, so that the switching flexibility, the success rate and the user service quality can be improved.

Description

Background Art

[0002] Since both low Earth orbit (LEO) satellites and high-speed railways have high-speed mobility, multi-beam switching in satellite-to-ground communication links is more complicated. Frequent inter-satellite switching not only increases network overhead costs, but also places higher requirements on the satellite's computing processing capabilities and management efficiency. There is a risk of interruption of satellite-to-ground communications, which in turn affects the safety of high-speed railway operation and the smooth communication experience of passengers.

[0003] In the related art, the low-orbit satellite switching strategy for high-speed mobile terminals generally only considers a single goal, the switching strategy is not flexible, the inter-satellite switching success rate is low, and the user service quality is relatively low.

[0004] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0005] The present disclosure provides a high-speed mobile terminal switching method, device, equipment, medium and program product, which at least to a certain extent overcome the problems of inflexible switching strategy, low inter-satellite switching success rate and relatively low user service quality in the related art.

[0006] Other features and advantages of the present disclosure will become apparent from the following detailed description, or may be learned in part by the practice of the present disclosure.

[0007] In a first aspect, an embodiment of the present disclosure provides a high-speed mobile terminal switching method, the method comprising:

[0008] Taking the high-speed mobile terminal as an intelligent agent, a terminal switching model between the high-speed mobile terminal and the low-orbit satellite is constructed;

[0009] A quaternion of the interaction between the high-speed mobile terminal and the low-orbit satellite is obtained and stored in an experience pool; the quaternion includes: a state set, an action set, a reward set of the t time slot and a state set of the t+1 time slot; the state set includes at least: coverage information between the low-orbit satellite and the high-speed mobile terminal, and at least two of the remaining service time, idle channels and user signal-to-noise ratio; the action set is a switching action for the high-speed mobile terminal to select a target satellite from a set of candidate satellites for access;

[0010] Extract training samples from the experience pool, iteratively train the switching process through the policy network and the value network, determine the loss function values ​​of the policy network and the value network according to the reward function and the action value function, and obtain the optimal switching strategy set of the terminal switching model when the policy network and the value network training are completed;

[0011] Switching is performed according to the optimal switching strategy set.

[0012] In a possible embodiment, training samples are extracted from the experience pool, the switching process is iteratively trained through the policy network and the value network, and the loss function values ​​of the policy network and the value network are determined according to the reward function and the action value function. When the policy network and the value network are trained, the optimal switching strategy set of the terminal switching model is obtained, including:

[0013] When the number of quadruplets in the experience pool reaches a first preset number, a second preset number of training samples are randomly extracted from the experience pool, and the policy network and the value network are iteratively trained for the switching process, wherein an iterative process at time slot t includes:

[0014] Input the training samples into the policy network and the value network, and determine the value loss function value of the value network and the policy loss function value of the policy network according to the reward function and the action value function;

[0015] According to the value loss function value and the strategy loss function value, the network parameters are adjusted until the value loss function value meets the first preset value and the strategy loss function value meets the second preset value, and the training of the strategy network and the value network is determined to be completed, and the optimal switching strategy set of the terminal switching model is obtained.

[0016] In a possible embodiment, determining a reward function value corresponding to the reward function includes:

[0017] Determine, according to the coverage information of the t time slot, the switching overhead between the high-speed mobile terminal and the satellite covering the high-speed mobile terminal in the t time slot;

[0018] Determining entropy weights of at least two of remaining service time, idle channels, and user signal-to-noise ratio;

[0019] According to the entropy weight, the state set under the t time slot and the switching cost of the t time slot, the reward function value corresponding to the reward function is determined.

[0020] In a possible embodiment, determining entropy value weights of at least two of remaining service time, idle channels, and user signal-to-noise ratio includes:

[0021] Normalizing at least two of the remaining service time, the idle channel, and the user signal-to-noise ratio, and determining a proportion of at least two of the remaining service time, the idle channel, and the user signal-to-noise ratio;

[0022] According to the proportion of at least two of the remaining service time, the idle channel and the user signal-to-noise ratio, the entropy value weights of at least two of the remaining service time, the idle channel and the user signal-to-noise ratio are determined.

[0023] In a possible embodiment, the experience pool includes at least: a regular experience pool and an effective experience pool;

[0024] Extract training samples from the experience pool, including:

[0025] Assign sample extraction weights to the regular experience pool and the effective experience pool;

[0026] When the number of training samples in both the regular experience pool and the effective experience pool reaches a first preset number, a second preset number of training samples are randomly selected from the regular experience pool and the effective experience pool according to the sample extraction weight.

[0027] In a possible embodiment, the method further includes:

[0028] In each iterative training process, the temporal difference error of the value network is used to determine whether the training sample is a valid sample;

[0029] If so, the valid sample replaces the original four-tuple in the valid experience pool and is put into the valid experience pool.

[0030] In a possible embodiment, obtaining an optimal switching strategy set of a terminal switching model includes:

[0031] When the training of the policy network and the value network is completed, the optimal switching strategy set is obtained based on the state set and action set of the four-tuple in the effective experience pool.

[0032] In a possible embodiment, obtaining the remaining service time includes:

[0033] For any low-orbit satellite and a low-orbit satellite high-speed mobile system formed by any high-speed mobile terminal provided with services by any low-orbit satellite, obtain a first velocity vector of any high-speed mobile terminal and a second velocity vector of any low-orbit satellite;

[0034] Determining a composite velocity vector based on the first velocity vector and the second velocity vector;

[0035] Determine the maximum coverage time of any low-orbit satellite to any high-speed mobile terminal based on the composite speed, satellite orbit radius, earth radius, minimum elevation angle, and the shortest distance from the high-speed mobile terminal to any low-orbit satellite footprint trajectory plane;

[0036] Obtaining the entry time of any high-speed mobile terminal into the coverage area of ​​any low-orbit satellite;

[0037] The remaining service time is determined based on the maximum coverage time and the entry time.

[0038] In a second aspect, an embodiment of the present disclosure provides a high-speed mobile terminal switching device, including:

[0039] A construction unit, used to take the high-speed mobile terminal as an intelligent agent and construct a terminal switching model between the high-speed mobile terminal and the low-orbit satellite;

[0040] The acquisition unit is used to acquire a quaternion of the interaction between the high-speed mobile terminal and the low-orbit satellite, and store it in an experience pool; the quaternion includes: a state set, an action set, a reward set of the t time slot, and a state set of the t+1 time slot; the state set includes at least: coverage information between the low-orbit satellite and the high-speed mobile terminal, and at least two items of the remaining service time, the idle channel, and the user signal-to-noise ratio; the action set is a switching action for the high-speed mobile terminal to select a target satellite from a set of candidate satellites for access;

[0041] A training unit is used to extract training samples from the experience pool, iteratively train the switching process through the policy network and the value network, determine the loss function values ​​of the policy network and the value network according to the reward function and the action value function, and obtain the optimal switching strategy set of the terminal switching model when the policy network and the value network training are completed;

[0042] The switching unit is used for switching according to an optimal switching strategy set.

[0043] In a third aspect, an embodiment of the present disclosure provides an electronic device, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute the method in the first aspect above by executing the executable instructions.

[0044] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the method in the first aspect described above is implemented.

[0045] In a fifth aspect, according to another aspect of the present disclosure, a computer program product or a computer program is also provided, the computer program product or the computer program comprising computer instructions, the computer instructions being stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs any of the above methods.

[0046] The embodiments of the present disclosure provide a high-speed mobile terminal switching method, device, equipment, medium and program product, which relate to the field of communication technology. The method includes: taking the high-speed mobile terminal as an intelligent agent, constructing a terminal switching model between the high-speed mobile terminal and the low-orbit satellite; obtaining a quaternary group of the high-speed mobile terminal interacting with the low-orbit satellite, and storing it in an experience pool; the quaternary group includes: a state set, an action set, a reward set and a state set of the t+1 time slot; the state set includes at least: coverage information between the low-orbit satellite and the high-speed mobile terminal, and at least two items of the remaining service time, the idle channel and the user signal-to-noise ratio; the action set is a switching action for the high-speed mobile terminal to select a target satellite from a candidate satellite set for access; extracting training samples from the experience pool, iteratively training the switching process through a policy network and a value network, determining the loss function values ​​of the policy network and the value network according to the reward function and the action value function, and obtaining the best switching strategy set of the terminal switching model when the policy network and the value network are trained; switching is performed according to the best switching strategy set. Considering multiple objectives to train the model, the flexibility of the switching strategy is improved, switching is performed based on the best switching strategy set, and the success rate and user service quality are improved.

[0047] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] The accompanying drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the specification are used to explain the principles of the present disclosure. Obviously, the accompanying drawings described below are only some embodiments of the present disclosure, and for ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without creative work.

[0049] Figure 1 A system architecture diagram according to an embodiment of the present disclosure is shown;

[0050] Figure 2 A flow chart of a high-speed mobile terminal switching method in an embodiment of the present disclosure is shown;

[0051] Figure 3 A flow chart for determining remaining service time in an embodiment of the present disclosure is shown;

[0052] Figure 4 A schematic diagram of obtaining a synthetic velocity vector in an embodiment of the present disclosure is shown;

[0053] Figure 5 A flowchart for determining a reward function value in an embodiment of the present disclosure is shown;

[0054] Figure 6A flow chart of a training process in an embodiment of the present disclosure is shown;

[0055] Figure 7 A flow chart of updating a valid experience pool in an embodiment of the present disclosure is shown;

[0056] Figure 8 A schematic structural diagram of a high-speed mobile terminal switching device in an embodiment of the present disclosure is shown;

[0057] Fig. 9 A schematic structural diagram of an electronic device in an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0058] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that the disclosure will be more comprehensive and complete and to fully convey the concepts of the example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0059] In addition, the accompanying drawings are only schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor devices and / or microcontroller devices.

[0060] Figure 1 A schematic diagram showing a system architecture diagram of a high-speed mobile terminal switching method that can be applied to an embodiment of the present disclosure is shown.

[0061] like Figure 1 As shown, high-speed mobile terminal switching 100 may include a low-orbit satellite 101 and a high-speed mobile terminal 102 .

[0062] The low-orbit satellite 101 and the high-speed mobile terminal 102 are connected via a network. The high-speed mobile terminal 102 may be a terminal device on a high-speed train, or other high-speed mobile terminal devices. Figure 1 Taking the high-speed rail as an example, the terminals on the high-speed rail are regarded as a whole, and the movement speed of the terminals on the high-speed rail is the same.

[0063] The network is used as a medium to provide a communication link between the low-orbit satellite 101 and the high-speed mobile terminal 102, and can be a wired network or a wireless network.

[0064] Optionally, the wireless network or wired network described above uses standard communication technology and / or protocol. The network is usually the Internet, but it can also be any network, including but not limited to a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), a mobile, wired or wireless network, a dedicated network or any combination of a virtual private network). In some embodiments, technologies and / or formats including Hyper Text Mark-up Language (HTML), Extensible Markup Language (XML), etc. are used to represent data exchanged through the network. In addition, conventional encryption technologies such as Secure Socket Layer (SSL), Transport Layer Security (TLS), Virtual Private Network (VPN), Internet Protocol Security (IPsec) can also be used to encrypt all or some links. In other embodiments, customized and / or dedicated data communication technologies can also be used to replace or supplement the above data communication technologies.

[0065] The high-speed mobile terminal 102 may be various electronic devices, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, wearable devices, augmented reality devices, virtual reality devices, etc.

[0066] Optionally, the client of the application installed in different high-speed mobile terminals 102 is the same, or the client of the same type of application based on different operating systems. Based on the different terminal platforms, the specific form of the client of the application can also be different, for example, the application client can be a mobile client, a PC client, etc.

[0067] Those skilled in the art will know that Figure 1 The number of low-orbit satellites 101 and high-speed mobile terminals 102 is only illustrative, and only one is taken as an example. According to actual needs, there may be any number of low-orbit satellites 101 and high-speed mobile terminals 102. The embodiments of the present disclosure are not limited to this.

[0068] The present exemplary implementation is described in detail below with reference to the accompanying drawings and embodiments.

[0069] First, a high-speed mobile terminal switching method is provided in an embodiment of the present disclosure. The method can be executed by any electronic device with computing and processing capabilities, taking a switching control system under a satellite-ground fusion network architecture as an example.

[0070] Figure 2 A flow chart of a high-speed mobile terminal switching method according to an embodiment of the present disclosure is shown. Figure 2 As shown, the high-speed mobile terminal switching method provided in the embodiment of the present disclosure includes the following steps:

[0071] S202: Taking the high-speed mobile terminal as an intelligent agent, a terminal switching model between the high-speed mobile terminal and the low-orbit satellite is constructed.

[0072] S204: Acquire a quaternion of the interaction between the high-speed mobile terminal and the low-orbit satellite, and store it in an experience pool. The quaternion includes: a state set, an action set, a reward set of the t time slot, and a state set of the t+1 time slot. The state set includes at least: coverage information between the low-orbit satellite and the high-speed mobile terminal, and at least two items of the remaining service time, idle channels, and user signal-to-noise ratio. The action set is a switching action for the high-speed mobile terminal to select a target satellite from a set of candidate satellites for access.

[0073] Among them, the four-tuple is expressed as: (s t ,a t ,r t+1 ,s t+1 ), s t represents the state set of time slot t, a t represents the action set of time slot t, r t+1 represents the reward set, s t+1 Represents the state set of time slot t+1.

[0074] S206: Extract training samples from the experience pool, iteratively train the switching process through the policy network and the value network, determine the loss function values ​​of the policy network and the value network according to the reward function and the action value function, and obtain the optimal switching strategy set of the terminal switching model when the policy network and the value network training are completed.

[0075] S208: Perform switching according to the optimal switching strategy set.

[0076] Among them, S202-S206 is a training process, which can obtain the best switching strategy set.

[0077] In a possible embodiment, during the training process, a terminal switching model is constructed based on the interaction between a high-speed mobile terminal and a low-orbit satellite, wherein the decision problem of inter-satellite switching can be modeled as a Markov decision process, and states, actions, and reward functions are defined. The reward function is solved according to the state set in the quadruple.

[0078] In a possible embodiment, an experience pool is constructed to save the quadruple generated by the interaction, and a deep Q-network (DQN) is constructed as the action-value function through a neural network, and the neural network parameters of the online network of the policy network (Actor) and the value network (Critic) and the target network are set to iteratively train the switching process.

[0079] Among them, the network parameters of the Actor's online network are represented as θ, the network parameters of the Actor's target network are represented as θ′, the network parameters of the Critic's online network are represented as w, and the network parameters of the Critic's target network are represented as w′.

[0080] In deep learning, the Actor selects and executes actions based on the current strategy, and the Critic is used to predict the cumulative reward that the agent can obtain after taking a certain action in a certain state.

[0081] Actor and Critic together constitute the Actor-Critic framework in reinforcement learning, where the Actor is responsible for making decisions and executing actions, while the Critic is responsible for evaluating the quality of these actions and providing feedback to the Actor to optimize its strategy.

[0082] The state set takes into account multiple influencing factors, and no longer makes decisions on intersatellite switching based on a single-target strategy, meeting the needs of users in multi-target situations, ensuring the success rate of switching, and improving the quality of user service. Through deep learning of this network architecture, the switching process that considers multiple influencing factors can improve the accuracy of its switching decision, obtain the best switching strategy set, and obtain a flexible and efficient intersatellite switching strategy.

[0083] In a possible embodiment, the state set shows several examples. (1) The state set may include: coverage information, remaining service time, and idle channels. (2) The state set may include: coverage information, remaining service time, and user signal-to-noise ratio. (3) The state set may include: coverage information, remaining service time, idle channels, and user signal-to-noise ratio.

[0084] Furthermore, the state set may also include other parameters. In addition to coverage information, remaining service time, idle channels and user signal-to-noise ratio, other influencing factors may also be added. The state set includes multiple influencing factors that have an impact on how the switching decision selects an action.

[0085] The present disclosure provides a method for determining the parameters included in the state set in the following embodiments. The solution is not limited to the method shown in the following embodiments, and can also be determined in other ways, as shown below:

[0086] Embodiment 1: Coverage information, information on whether a high-speed mobile terminal is covered by a LEO satellite.

[0087] The present disclosure takes N LEO satellites as an example. represents the set of LEO satellites. Taking the high-speed mobile terminal as a terminal on the high-speed rail as an example, and regarding the high-speed rail as a single user, the coverage information set of the high-speed rail covered by the LEO satellite at time slot t can be expressed as: in It means that the high-speed railway is covered by satellite N in time slot t, otherwise there is no coverage.

[0088] Embodiment 2: Remaining service time, the remaining service time of the LEO satellite to the high-speed mobile terminal it is serving.

[0089] High-speed mobile terminals have high-speed mobility, and LEO satellites also have high-speed mobility. During the training of the switching process, it is necessary to consider the speed of the high-speed mobile terminal and the LEO satellite, and to build models based on the high-speed mobility characteristics of the two. In the process of solving the parameters, the high-speed mobility of the two is taken into account to improve the accuracy and credibility of the final optimal switching strategy set.

[0090] Figure 3 A flow chart of determining the remaining service time in an embodiment of the present disclosure is shown. Figure 3 As shown, the following steps are included:

[0091] S302: For any low-orbit satellite and a low-orbit satellite high-speed mobile system formed by any high-speed mobile terminal provided with services by the low-orbit satellite, obtain a first velocity vector of any high-speed mobile terminal and a second velocity vector of any low-orbit satellite.

[0092] S304: Determine a composite velocity vector according to the first velocity vector and the second velocity vector.

[0093] S306: Determine the maximum coverage time of any low-orbit satellite to any high-speed mobile terminal based on the composite speed, satellite orbit radius, earth radius, minimum elevation angle, and the shortest distance from the high-speed mobile terminal to any low-orbit satellite footprint trajectory plane.

[0094] S308: Obtain the entry time of any high-speed mobile terminal into the coverage area of ​​any low-orbit satellite.

[0095] S310: Determine the remaining service time according to the maximum coverage time and the entry time.

[0096] In a possible embodiment, a low-orbit satellite high-speed mobile system is established, the high-speed mobile terminal itself is used as a reference, the high-speed mobile terminal is regarded as a stationary state, and the velocity vector sum of the high-speed mobile terminal and the LEO satellite is superimposed. For any low-orbit satellite in the low-orbit satellite high-speed mobile system, and any high-speed mobile terminal provided with service by any low-orbit satellite, take low-orbit satellite A and high-speed mobile terminal A as an example. Figure 4 A schematic diagram of obtaining a synthetic velocity vector in an embodiment of the present disclosure is shown. Figure 4 shown.

[0097] v1 is the first velocity vector of the high-speed mobile terminal A, v2 is the second velocity vector of the low-orbit satellite A, and v3 is the composite velocity vector. Figure 4 One satellite and one high-speed mobile terminal are given only for illustrative purposes.

[0098] The remaining service time provided by the low-orbit satellite to the high-speed mobile terminal at time slot t can be expressed as Y t , Y t It can be expressed as

[0099] In a possible embodiment, the method for determining the shortest distance from the high-speed mobile terminal to any low-orbit satellite footprint trajectory plane may include: obtaining the position coordinates of any satellite at three different time points, obtaining the equation of the plane where the satellite orbit is located, and then determining the shortest distance according to the distance formula from the point to the plane, with d min express.

[0100] In a possible embodiment, the remaining service time of LEO satellite n can be determined by formula (1).

[0101]

[0102] Among them, t max It represents the maximum coverage time of LEO satellite n, and t0 is the entry time of the high-speed mobile terminal into the coverage area of ​​LEO satellite n.

[0103] In one possible embodiment, the maximum coverage time t max It can be determined by the following formula (2):

[0104]

[0105] Among them, R e is the radius of the Earth, R LEO is the LEO satellite orbit radius, θ min is the minimum elevation angle, and ω is the angular velocity in the LEO satellite high-speed rail system.

[0106] In one possible embodiment, the angular velocity can be determined by formula (3), as follows:

[0107]

[0108] Embodiment 3: Idle channels, the number of idle channels of LEO satellites.

[0109] The idle channel of the LEO satellite at time slot t can be expressed as: High-speed mobile terminals can obtain it based on the periodic broadcasts of satellites.

[0110] Example 4: Signal-to-noise ratio, signal-to-noise ratio between a high-speed mobile terminal and a LEO satellite.

[0111] The signal-to-noise ratio between the high-speed mobile terminal and the LEO satellite at time slot t can be expressed as:

[0112] Among them, at time slot t, the signal-to-noise ratio between the LEO satellite n and the high-speed mobile terminal can be expressed by formula (4), as shown below:

[0113]

[0114] in, is the transmission power of LEO satellite n, The off-axis angle of the transmitting antenna is The transmission gain of LEO satellite n, G h The receiving gain of the high-speed mobile terminal antenna. is the path loss from LEO satellite n to high-speed mobile terminal, σ 2 is the noise power, Indicates whether LEO satellite n covers the high-speed mobile terminal at time slot t.

[0115] Through the above four embodiments, in the embodiments of the present disclosure, the state set of the defined intelligent agent is taken as coverage information, remaining service time, idle channel and user signal-to-noise ratio as an example, and the state set can be expressed as: t ={X t ,Y t ,K t ,δ t}, which can be understood as the environmental status information of the high-speed mobile terminal in time slot t.

[0116] The defined action set can be expressed as: action It indicates the action of a high-speed mobile terminal selecting a target LEO satellite n from a set of candidate satellites to access.

[0117] The defined reward function can be expressed as: Rt .

[0118] Among them, the reward function can be determined in a variety of ways, and a variety of influencing factors can be comprehensively considered to adjust the reward function value, set a positive feedback reward function, encourage actions to switch in, try to ensure that the reward function value is large, and encourage actions when the expected state appears.

[0119] Figure 5 A flow chart of determining the reward function value in an embodiment of the present disclosure is shown, taking time slot t as an example, Figure 5 As shown, the following steps are included:

[0120] S502: Determine the switching overhead between the high-speed mobile terminal and the satellite covering the high-speed mobile terminal in the t time slot according to the coverage information of the t time slot.

[0121] In a possible embodiment, the switching overhead between the high-speed mobile terminal and its coverage satellite at time slot T is when but when but Where C is a positive number.

[0122] S504: Determine entropy weights of at least two of remaining service time, idle channels, and user signal-to-noise ratio.

[0123] In a possible embodiment, the switching strategy that comprehensively considers multiple influencing factors needs to consider the impact of different states on the action, and the corresponding reward function value should also consider multiple influencing factors.

[0124] In the embodiment of the present disclosure, only three influencing factors including remaining service time, idle channel and user signal-to-noise ratio are taken as examples. When other influencing factors are also included in the state set, other parameters can be introduced to jointly determine the entropy weight. For example, when parameters A and B are introduced, when determining the entropy weight, the influence of A and B on the action and the influence on the reward function can be comprehensively considered. When determining the reward function value, A and B are included in the range of determining the reward function value.

[0125] In a possible embodiment, the determination method may include: standardizing at least two of the remaining service time, the idle channel, and the user signal-to-noise ratio to determine the proportion of at least two of the remaining service time, the idle channel, and the user signal-to-noise ratio; and determining the entropy value weights of at least two of the remaining service time, the idle channel, and the user signal-to-noise ratio based on the proportion of at least two of the remaining service time, the idle channel, and the user signal-to-noise ratio.

[0126] Exemplarily, the entropy weights of the remaining service time, idle channel and user signal-to-noise ratio are determined as an example, as follows: the remaining service time, idle channel and user signal-to-noise ratio are standardized, the first proportion, the second proportion and the third proportion of the remaining service time, idle channel and user signal-to-noise ratio are determined, and the entropy weights of the remaining service time, idle channel and user signal-to-noise ratio are determined according to the first proportion, the second proportion and the third proportion; the sum of the entropy weights of the remaining service time, idle channel and user signal-to-noise ratio is 1. The specific determination method is not limited to the determination method given in the example.

[0127] Exemplarily, the proportions of the standardized remaining service time, idle channels, and user signal-to-noise ratio can be calculated by formulas (6), (7), and (8), which are respectively: the first proportion, the second proportion, and the third proportion.

[0128]

[0129] in, Indicates the first proportion, the proportion of remaining service time.

[0130]

[0131] in, Indicates the second proportion, the proportion of idle channels.

[0132]

[0133] in, It represents the second proportion, the proportion of user signal-to-noise ratio.

[0134] After determining the first weight, the second weight, and the third weight, the entropy value weight of time slot t can be determined by formulas (9), (10), and (11).

[0135]

[0136] in, Represents the entropy weight of time slot t corresponding to the remaining service time.

[0137]

[0138] in, Represents the entropy weight of time slot t corresponding to the idle channel.

[0139]

[0140] in, Represents the entropy value weight of time slot t corresponding to the user signal-to-noise ratio.

[0141] In the above manner, entropy weights can be obtained. However, since the entropy weights obtained in different time slots are different, during the preset time period in the training process, from the running slot t=0 to the ending slot T, the average entropy weight of the preset time period is obtained according to the entropy weight of each time slot t. The average entropy weight is used as the multi-attribute weight, that is, the entropy weights β1, β2, and β3 of the remaining service time, idle channel, and user signal-to-noise ratio are obtained, where β1+β2+β3=1.

[0142] Among them, the preset time period is the overall interaction time length between the intelligent agent (high-speed mobile terminal) and the LEO satellite during the training process.

[0143] It should be noted that if, in addition to the three factors of remaining service time, idle channel, and user signal-to-noise ratio, the remaining service time, idle channel, user signal-to-noise ratio, and the entropy weights of A and B are determined, then the sum of the remaining service time, idle channel, user signal-to-noise ratio, and the entropy weights of A and B is 1.

[0144] S506: Determine a reward function value corresponding to the reward function according to the entropy weight, the state set in the t time slot, and the switching cost of the t time slot.

[0145] In a possible embodiment, the reward function value corresponding to the reward function is determined by formula (12).

[0146]

[0147] Among them, r f is a negative number, δ min is the minimum signal-to-noise ratio threshold, k max is the total number of channels, μ is the adjustment coefficient, which is preset and can be flexibly adjusted according to the algorithm effect.

[0148] In one possible embodiment, Figure 6 A flow chart of a training process in an embodiment of the present disclosure is shown. Figure 6 As shown, the following steps are included:

[0149] S602: When the number of quadruplets in the experience pool reaches a first preset number, a second preset number of training samples are randomly extracted from the experience pool.

[0150] S604: Input the training samples into the policy network and the value network, and determine the value loss function value of the value network and the policy loss function value of the policy network according to the reward function and the action value function.

[0151] S606: Determine whether the value loss function value satisfies the first preset value, and whether the strategy loss function value satisfies the second preset value; if not, execute S608; if yes, execute S610.

[0152] S608: Adjust network parameters according to the value loss function value and the strategy loss function value.

[0153] S610: Determine that the training of the strategy network and the value network is completed, and obtain the optimal switching strategy set of the terminal switching model.

[0154] In a possible embodiment, the experience pool may be set to two experience pools, including at least: a regular experience pool and a valid experience pool.

[0155] The method of extracting training samples may include the following method: assigning sample extraction weights to the regular experience pool and the effective experience pool, and when the training samples in the regular experience pool and the effective experience pool reach a first preset number, randomly extracting a second preset number of training samples from the regular experience pool and the effective experience pool according to the sample extraction weights.

[0156] For example, a regular experience pool and an effective experience pool are introduced during the training process, and a capacity of N is emptied in the initial stage. tot The regular experience pool and the effective experience pool are in the training process. When the number of storages reaches N, the quadruple is stored in the regular experience pool and the effective experience pool. tot When N is randomly selected from the common experience pool S (1-ψ) quadruplets are used as training samples, where ψ represents the sample extraction weight. N quadruplets are randomly selected from the valid experience pool. S ·ψ quadruplets are used as training samples, where N S Indicates the second preset number. N tot It can be a first preset number.

[0157] By setting up a dual experience replay pool, the training speed of the neural network can be accelerated and the training efficiency can be improved.

[0158] In a possible embodiment, during the training process, the temporal difference target value and the temporal difference error (TD error) of the Critic network are calculated according to each sample, and the value loss function value is determined to adjust the network parameters.

[0159] Among them, the temporal difference target value is determined according to the action value function value, that is, the Q function value and reward function value corresponding to the Q network.

[0160] According to each extracted training sample, the temporal difference target value of the Critic network is calculated. The temporal difference target value is expressed as z t , can be determined by formula (13) as follows:

[0161] z t =Rt +γ·Q(s t+1 ,a′ t+1 ;w′) (13)

[0162] Among them, γ represents the preset parameter, s t+1 represents the state set at time slot t+1, a′ t+1 represents the t+1 time sequence action output by the target network of the Actor network, w′ represents the network parameters of the target network of the Critic network, Q(s t+1 ,a′ t+1 ; w′) represents the action value function obtained by inputting the state set at time slot t+1 and the action at time sequence t+1 when w′ is the network parameter.

[0163] Among them, a′ t+1 It can be determined by formula (14) as follows:

[0164] a′ t+1 =π(s t+1 ;θ′) (14)

[0165] Among them, π represents the strategy and θ′ represents the network parameters of the target network of the Actor network.

[0166] After determining the target value of the timing difference, the TD error can be determined by formula (15) as follows:

[0167] ξ t =z t -Q(s t ,a t ;w) (15)

[0168] Among them, ξ t represents the TD error, Q(s t ,a t ; w) represents the action value function obtained by inputting the state set at time slot t and the action at time sequence t under the network parameters of the online network of the Critic network.

[0169] After determining the TD error, the TD error can be used to measure the difference between the agent's current predicted future rewards and the rewards in actual experience, and use this difference to guide the update of the Critic network parameters and the optimization of the strategy.

[0170] During the training process, the value loss function value of the value network and the policy loss function value of the policy network are determined and updated if the preset values ​​are not met.

[0171] Among them, based on the learning rates of the Critic network and the Actor network and the network parameters of the online network during the current training, an update is performed to obtain the network parameters of the online network of the Critic network and the Actor network during the next training.

[0172] The network parameters of the online network can be updated using formulas (16) and (17) as follows:

[0173]

[0174] In order to distinguish the network parameters of the online network during the current training and the network parameters during the next training, w1 and θ1 are used to represent the current one, and w2 and θ2 are used to represent the next one.

[0175] α c is the learning rate of the Critic network, α a is the learning rate of the Actor network, i is the index, representing the i-th training sample, and π is the strategy.

[0176] The network parameters of the target network of the Critic network and the Actor network are updated by soft updating, which can be processed by formulas (18) and (19) as follows:

[0177] w′=ζ·w+(1-ζ)w′ (18)

[0178] θ′=ζ·θ+(1-ζ)θ′ (19)

[0179] The network parameters of the online network used to update the network parameters w′ and θ′ of the target network are uniformly represented by w and θ without numbers. ζ represents the soft update factor, which is used to control the update amplitude.

[0180] During the training process, the training is iterated continuously and the network parameters are updated. The network parameters refer to: the network parameters of the target network of the Critic network and the Actor network, and the network parameters of the online network of the Critic network and the Actor network, until the value loss function value meets the first preset value, and the strategy loss function value meets the second preset value, it is determined that the training of the strategy network and the value network is completed, and the optimal switching strategy set of the terminal switching model is obtained.

[0181] In a possible embodiment, during each training process, two experience pools are updated, and during each training process, the regular experience pool may discard the earliest stored quadruplets in sequence. For the effective experience pool, a corresponding update method is set.

[0182] Figure 7 A flow chart of updating the effective experience pool in an embodiment of the present disclosure is shown. Figure 7As shown, the following steps are included:

[0183] S702: During each iterative training process, whether the training sample is a valid sample is determined based on the temporal difference error of the value network.

[0184] S704: If yes, the valid sample is used to replace the original four-tuple in the valid experience pool and put into the valid experience pool.

[0185] For example, the judgment method can use the TD error as a judgment basis, if |ξ t |>D, where D is a positive preset constant value.

[0186] During the training process, the reward function value and action value function value of the quadruple in the valid experience pool may be higher. Selecting valid samples during training can update the neural network parameters to a greater extent, improve the network training speed, and accelerate convergence. However, it is still necessary to randomly sample from the regular experience pool where all samples are stored to reduce the probability of overfitting.

[0187] In a possible embodiment, during the training process, in order to avoid a certain action never being selected, a greedy strategy can be adopted to enable the agent to keep exploring new actions.

[0188] The new action is selected by formula (20) as follows:

[0189]

[0190] Among them, ε represents the probability of randomly selecting an action. With a probability of 1-ε, the action is determined by finding the maximum value of the action value function.

[0191] In a possible embodiment, in the case where an effective experience pool is set, when the training of the policy network and the value network is completed, the optimal switching strategy set is obtained according to the state set and action set of the four-tuple in the effective experience pool.

[0192] Based on the same inventive concept, the present disclosure also provides a high-speed mobile terminal switching device, such as the following embodiment. Since the principle of solving the problem in the device embodiment is similar to that in the above method embodiment, the implementation of the device embodiment can refer to the implementation of the above method embodiment, and the repeated parts will not be repeated.

[0193] Figure 8 A schematic diagram of the structure of a high-speed mobile terminal switching device in an embodiment of the present disclosure is shown. Figure 8As shown, the device 80 includes: a construction unit 801, which is used to take the high-speed mobile terminal as an intelligent agent and construct a terminal switching model between the high-speed mobile terminal and the low-orbit satellite; an acquisition unit 802, which is used to acquire a quaternion of the high-speed mobile terminal interacting with the low-orbit satellite and store it in an experience pool; the quaternion includes: a state set, an action set, a reward set and a state set of the t time slot; the state set at least includes: coverage information between the low-orbit satellite and the high-speed mobile terminal, and at least two items of the remaining service time, the idle channel and the user signal-to-noise ratio; the action set is a switching action for the high-speed mobile terminal to select a target satellite from a candidate satellite set for access; a training unit 803, which is used to extract training samples from the experience pool, iteratively train the switching process through a policy network and a value network, determine the loss function values ​​of the policy network and the value network according to the reward function and the action value function, and obtain the optimal switching strategy set of the terminal switching model when the policy network and the value network are trained; a switching unit 804, which is used to switch according to the optimal switching strategy set.

[0194] Those skilled in the art will appreciate that various aspects of the present disclosure may be implemented as systems, methods or program products. Therefore, various aspects of the present disclosure may be specifically implemented in the following forms, namely: complete hardware implementation, complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software, which may be collectively referred to herein as "circuits", "modules" or "systems".

[0195] Refer to the following Fig. 9 The electronic device 900 according to this embodiment of the present disclosure is described. Fig. 9 The electronic device 900 shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0196] like Fig. 9 As shown, the electronic device 900 is in the form of a general computing device. The components of the electronic device 900 may include but are not limited to: at least one processing unit 910, at least one storage unit 920, and a bus 930 connecting different system components (including the storage unit 920 and the processing unit 910).

[0197] The storage unit stores a program code, which can be executed by the processing unit 910, so that the processing unit 910 performs the steps of various exemplary embodiments of the present disclosure described in the above “Exemplary Method” section of this specification. For example, the processing unit 910 can perform any step in the above method embodiments.

[0198] The storage unit 920 may include a readable medium in the form of a volatile storage unit, such as a random access storage unit (RAM) 9201 and / or a cache storage unit 9202 , and may further include a read-only storage unit (ROM) 9203 .

[0199] The storage unit 920 may also include a program / utility 9204 having a set (at least one) of program modules 9205, such program modules 9205 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0200] Bus 930 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0201] The electronic device 900 may also communicate with one or more external devices 940 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 900, and / or may communicate with any device that enables the electronic device 900 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface 950. Furthermore, the electronic device 900 may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via a network adapter 960. As shown, the network adapter 960 communicates with other modules of the electronic device 900 via a bus 930. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 900, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.

[0202] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the implementation of the present disclosure.

[0203] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer program product or a computer program, which includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the method in the above embodiment.

[0204] In an exemplary embodiment of the present disclosure, a computer-readable storage medium is also provided, which may be a readable signal medium or a readable storage medium. A program product capable of implementing the above method of the present disclosure is stored thereon. In some possible implementations, various aspects of the present disclosure may also be implemented in the form of a program product, which includes a program code, and when the program product is run on a terminal device, the program code is used to cause the terminal device to execute the steps according to various exemplary implementations of the present disclosure described in the above “Exemplary Method” section of this specification.

[0205] More specific examples of computer-readable storage media in the present disclosure may include, but are not limited to, an electrical connection having one or more conductors, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0206] In the present disclosure, a computer readable storage medium may include a data signal propagated in baseband or as part of a carrier wave, wherein a readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A readable signal medium may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0207] Alternatively, the program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the foregoing.

[0208] In a specific implementation, the program code for performing the operations of the present disclosure may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., using an Internet service provider to connect through the Internet).

[0209] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into multiple modules or units to be embodied.

[0210] In addition, although the steps of the method in the present disclosure are described in a specific order in the drawings, this does not require or imply that the steps must be performed in this specific order, or that all the steps shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps, etc.

[0211] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a mobile terminal, or a network device, etc.) to execute the method according to the implementation of the present disclosure.

[0212] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure, which follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the appended claims.

Claims

1. A high-speed mobile terminal switching method, characterized in that: The method comprises: Taking the high-speed mobile terminal as an intelligent agent, constructing a terminal switching model between the high-speed mobile terminal and the low-orbit satellite; Acquire a quaternion of interaction between the high-speed mobile terminal and the low-orbit satellite, and store it in an experience pool; the quaternion includes: a state set, an action set, a reward set of a time slot t, and a state set of a time slot t+1; the state set includes at least: coverage information between the low-orbit satellite and the high-speed mobile terminal, and at least two of remaining service time, idle channels, and user signal-to-noise ratio; the action set is a switching action for the high-speed mobile terminal to select a target satellite from a set of candidate satellites for access; Extracting training samples from the experience pool, iteratively training the switching process through the policy network and the value network, determining the loss function values ​​of the policy network and the value network according to the reward function and the action value function, and obtaining the optimal switching strategy set of the terminal switching model when the training of the policy network and the value network is completed; Switching is performed according to the optimal switching strategy set.

2. The method according to claim 1, characterized in that The step of extracting training samples from the experience pool, iteratively training the switching process through the policy network and the value network, determining the loss function values ​​of the policy network and the value network according to the reward function and the action value function, and obtaining the optimal switching strategy set of the terminal switching model after the training of the policy network and the value network is completed, includes: When the number of quadruplets in the experience pool reaches a first preset number, a second preset number of training samples are randomly selected from the experience pool, and the policy network and the value network are iteratively trained for the switching process, wherein an iterative process at time slot t includes: Input the training sample into the policy network and the value network, and determine the value loss function value of the value network and the policy loss function value of the policy network according to the reward function and the action value function; According to the value loss function value and the strategy loss function value, the network parameters are adjusted until the value loss function value meets the first preset value and the strategy loss function value meets the second preset value, and it is determined that the training of the strategy network and the value network is completed, and the optimal switching strategy set of the terminal switching model is obtained.

3. The method according to claim 1 or 2, characterized in that: Determine the reward function value corresponding to the reward function, including: Determining, according to the coverage information of the t time slot, a switching overhead between the high-speed mobile terminal and a satellite covering the high-speed mobile terminal in the t time slot; Determining entropy weights of at least two of remaining service time, idle channels, and user signal-to-noise ratio; A reward function value corresponding to the reward function is determined according to the entropy weight, the state set in the t time slot, and the switching overhead of the t time slot.

4. The method according to claim 3, characterized in that The step of determining the entropy weights of at least two of the remaining service time, the idle channel, and the user signal-to-noise ratio comprises: Normalizing at least two of the remaining service time, the idle channel, and the user signal-to-noise ratio, and determining a proportion of at least two of the remaining service time, the idle channel, and the user signal-to-noise ratio; According to the proportion of at least two of the remaining service time, the idle channel and the user signal-to-noise ratio, the entropy value weights of at least two of the remaining service time, the idle channel and the user signal-to-noise ratio are determined.

5. The method according to claim 1 or 2, characterized in that: The experience pool at least includes: a regular experience pool and an effective experience pool; Extracting training samples from the experience pool includes: Allocating sample extraction weights to the conventional experience pool and the effective experience pool; When the training samples in the conventional experience pool and the effective experience pool both reach a first preset number, a second preset number of training samples are randomly extracted from the conventional experience pool and the effective experience pool according to the sample extraction weight.

6. The method according to claim 5, characterized in that The method further comprises: In each iterative training process, judging whether the training sample is a valid sample according to the temporal difference error of the value network; If so, the valid sample replaces the original four-tuple in the valid experience pool and is put into the valid experience pool.

7. The method according to claim 5, characterized in that Obtaining an optimal switching strategy set for the terminal switching model includes: When the training of the strategy network and the value network is completed, the optimal switching strategy set is obtained according to the state set and action set of the four-tuple in the effective experience pool.

8. The method according to claim 1, characterized in that Get the remaining service time, including: For any low-orbit satellite and a low-orbit satellite high-speed mobile system formed by any high-speed mobile terminal provided with service by the any low-orbit satellite, obtaining a first velocity vector of the any high-speed mobile terminal and a second velocity vector of the any low-orbit satellite; determining a composite velocity vector according to the first velocity vector and the second velocity vector; Determine the maximum coverage time of any one of the low-orbit satellites for any one of the high-speed mobile terminals according to the synthetic speed, the satellite orbit radius, the earth radius, the minimum elevation angle, and the shortest distance from the high-speed mobile terminal to the footprint trajectory plane of any one of the low-orbit satellites; Obtaining the entry time of any one of the high-speed mobile terminals into the coverage area of ​​any one of the low-orbit satellites; The remaining service time is determined according to the maximum coverage time and the entry time.

9. A high-speed mobile terminal switching device, characterized in that: include: A construction unit, used to take the high-speed mobile terminal as an intelligent agent and construct a terminal switching model between the high-speed mobile terminal and the low-orbit satellite; An acquisition unit, used for acquiring a quadruple of the interaction between the high-speed mobile terminal and the low-orbit satellite, and storing the quadruple in an experience pool; The quaternary group includes: a state set, an action set, a reward set of a time slot t, and a state set of a time slot t+1; the state set includes at least: coverage information between the low-orbit satellite and the high-speed mobile terminal, and at least two items of remaining service time, idle channels, and user signal-to-noise ratio; the action set is a switching action for the high-speed mobile terminal to select a target satellite from a set of candidate satellites for access; A training unit, used to extract training samples from the experience pool, iteratively train the switching process through the policy network and the value network, determine the loss function values ​​of the policy network and the value network according to the reward function and the action value function, and obtain the optimal switching strategy set of the terminal switching model when the training of the policy network and the value network is completed; A switching unit is used to perform switching according to the optimal switching strategy set.

10. An electronic device, characterized in that: include: processor; as well as A memory, configured to store executable instructions of the processor; The processor is configured to perform the method of any one of claims 1 to 8 by executing the executable instructions.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

12. A computer program product comprising: A computer program or instruction, characterized in that when the computer program or instruction is executed by a processor, the method described in any one of claims 1 to 8 is implemented.

Images