Communication satellite switching model training method and satellite communication connection method

Abstract

The invention provides a communication satellite switching model training method and a satellite communication connection method, and the training method comprises the steps: obtaining a first motion score corresponding to a plurality of pieces of first satellite state information and a motion of switching satellite communication through a first network model, and second action scores corresponding to the plurality of pieces of second satellite state information. Inputting the action corresponding to the highest second action score and the corresponding satellite state information into a second network model to obtain a third action score; obtaining a target action score according to the third action score and the action reward; training a first network model according to the first action score and the target action score; and determining the trained first network model as the communication satellite switching evaluation network. The obtained communication satellite switching evaluation network can output an action score in combination with the number of channel loads and the energy loss so as to guide and select the action communicating with the satellite with a small number of channel loads and low energy loss.

Description

Technical Field

[0001] This application relates to the technical field of communication satellite handover, and specifically relates to a method for training a communication satellite handover model and a satellite communication connection method. Background Art

[0002] As a non-terrestrial communication platform, the low Earth orbit (LEO) satellite network has shown advantages such as lower latency, stronger signals, and more economical deployment costs compared to the medium Earth orbit (MEO) and geostationary Earth orbit (GEO) satellite networks. However, due to the characteristics of the rapid and periodic movement of LEO satellites, their ground coverage range changes continuously, resulting in difficulty in maintaining a long-term connection between the satellite and the user's terminal device. Therefore, it is necessary to frequently switch between satellites to ensure long-term communication continuity. Among them, the core idea of traditional satellite handover strategy algorithms, such as the single-attribute decision method, is to use a single attribute factor as the decision criterion. However, the single-attribute handover algorithm is prone to causing an excessive number of terminal devices accessing the satellite with the optimal attribute in a certain aspect, resulting in an overloaded load on that satellite. When the satellite channel is overloaded, the communication quality will be reduced, resulting in the need for the terminal device to switch satellites again to achieve good communication. During the handover process, due to the need for the terminal device to maintain a high-energy consumption state, there is a defect of excessive energy consumption. Summary of the Invention

[0003] The purpose of this application is to overcome the deficiencies in the prior art and provide a method for training a communication satellite handover model and a satellite communication connection method, which can...

[0004] The first aspect of the embodiments of this application provides a method for training a communication satellite handover model, including:

[0005] Obtain a plurality of first satellite state information corresponding to the terminal device and a plurality of second satellite state information corresponding to the terminal device after performing the action of switching satellite communication;

[0006] Obtain the second channel load quantity and energy loss of the second satellite for communication after handover from the plurality of second satellite state information;

[0007] If the second channel load quantity is greater than a preset channel load threshold, determine a preset negative reward as the action reward for the satellite communication handover;

[0008] If the second channel load quantity is less than or equal to the channel load threshold, obtain the action reward for the satellite communication handover according to a preset handover communication duration value, the data transmission rate between the terminal device and the second satellite, the second channel load quantity, and the handover time and energy loss of the terminal device for satellite communication handover;

[0009] Through the first network model, obtain the first action scores corresponding to a number of the first satellite state information and the actions of switching satellite communication, and the second action scores corresponding to a number of the second satellite state information;

[0010] From the several second action scores output by the first network model, input the action corresponding to the highest second action score and the corresponding several second satellite state information into the second network model to obtain the third action score output by the second network model;

[0011] Obtain the target action score according to the third action score and the action reward;

[0012] Train the first network model according to the first action score and the target action score;

[0013] Determine the trained first network model as the communication satellite handover evaluation network.

[0014] A second aspect of the embodiments of the present application provides a satellite communication connection method, obtain a communication satellite handover evaluation network trained according to the communication satellite handover model training method as described above;

[0015] Input the real-time satellite state information corresponding to the terminal device into the communication satellite handover evaluation network to obtain the policy action corresponding to the highest action score;

[0016] Drive the terminal device to execute the policy action to connect to the satellite pointed to by the policy action.

[0017] Compared with the prior art, the present application calculates the corresponding action reward according to the number of second channel loads after the action of realizing switching satellite communication and the energy loss of executing the satellite handover, and combines the third action score output by the first network model and the second network model to train the first network model to obtain the communication satellite handover evaluation network. Among them, the third action score is the score estimated by the second network model for the action with the highest second action score estimated by the first network model. Therefore, the third action score is a comprehensive score combining the model characteristics of the first network model and the second network model, and the action reward is obtained according to the number of second channel loads and the energy loss. Therefore, the communication satellite handover evaluation network trained according to the third action score, the action reward and the first action score output by the first network model can output the action score in combination with the number of second channel loads and the energy loss, which is beneficial to selecting the action of communicating with the satellite with less channel load and lower energy loss according to multiple satellite state information corresponding to the terminal device.

[0018] In order to understand the present application more clearly, the following will describe the specific embodiments of the present application in conjunction with the accompanying drawings. Brief Description of the Drawings

[0019] Figure 1 It is a step diagram of a method for training a communication satellite handover model according to an embodiment of the present application.

[0020] Figure 2 It is a schematic flowchart of a method for training a communication satellite handover model according to an embodiment of the present application.

[0021] Figure 3 It is a step diagram of a satellite communication connection method according to an embodiment of the present application.

[0022] Figure 4 It is a schematic diagram of step S200 of a satellite communication connection method according to an embodiment of the present application. Detailed Description of the Embodiments

[0023] To make the objectives, technical solutions, and advantages of the present application clearer, the following will further describe in detail the embodiments of the present application with reference to the accompanying drawings.

[0024] It should be clear that the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the embodiments of the present application.

[0025] When the following description refers to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. In the description of the present application, it should be understood that the terms "first", "second", "third", etc. are only used to distinguish similar objects, and do not have to be used to describe a specific order or sequence, nor can they be understood as indicating or implying relative importance. For those of ordinary skill in the art, the specific meanings of the above terms in the present application can be understood according to specific circumstances. The singular forms of "a", "the", and "said" used in the present application and the appended claims are also intended to include the plural forms, unless the context clearly indicates otherwise. The word "if" / "when" used herein can be interpreted as "when...", "when...", or "in response to a determination".

[0026] In addition, in the description of the present application, unless otherwise specified, "a plurality of" means two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships can exist. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone. The character " / " generally represents an "or" relationship between the associated objects before and after.

[0027] Please refer to Figure 1, which is a flowchart of a communication satellite handover model training method according to an embodiment of the present application, including:

[0028] S1: Obtain a plurality of first satellite state information corresponding to the terminal device and a plurality of second satellite state information corresponding to the terminal device after the action of realizing satellite communication handover.

[0029] Among them, the terminal device refers to an electronic device with wireless communication functions, such as a smart phone. The terminal device is communicatively connected to the first satellite before switching satellite communication, and the terminal device is communicatively connected to the second satellite after the action of realizing satellite communication handover. Specifically, when the terminal device receives a satellite handover signaling from the base station, the terminal device obtains the satellite ephemeris information and satellite ID required for satellite handover. The terminal device converts the received ephemeris information into the Earth-centered Earth-fixed coordinate system to obtain the satellite coordinates, and then constructs a communication system model based on the satellite coordinates of the terminal device and each satellite to obtain data such as the candidate satellite set, reference signal received signal strength set, remaining visible time set, and load channel set required by the terminal device, so as to obtain a plurality of satellite state information; among them, a plurality of first satellite state information is a plurality of satellite state information corresponding to the terminal device before switching satellites, and a plurality of second satellite state information is a plurality of satellite state information corresponding to the terminal device after switching satellites. The plurality of satellite state information corresponding to the terminal device can be expressed as:

[0030]

[0031] Among them, is the satellite state information of u satellites corresponding to the terminal device at time t, t is the time, u is the total number of satellites, is the u-th satellite corresponding to the terminal device at time t, is the reference signal received signal strength value of the u-th satellite corresponding to the terminal device at time t, is the remaining visible time between the terminal device and the u-th satellite at time t, l t is the number of channel loads of the satellite at time t.

[0032] Among them, a plurality of first satellite state information corresponding to the terminal device refers to the satellite state of a plurality of satellites corresponding to the terminal device before switching satellite communication; a plurality of second satellite state information refers to the satellite state of a plurality of satellites corresponding to the terminal device after the action of realizing satellite communication handover.

[0033] S2: Obtain the second channel load number and energy loss of the second satellite for communication after handover from the plurality of second satellite state information.

[0034] Among them, the number of second channel loads for communication after switching can be directly obtained from several pieces of second satellite status information. The number of second channel loads is the number of loaded channels of the satellite to which the terminal device is currently connected for communication after switching. The energy loss of switching satellites can be obtained based on the switching power and switching time of the terminal device for satellite communication switching.

[0035] The switching of the terminal device for satellite communication is achieved within a switching frame. The structure of the switching frame includes two stages. The first stage is the preparation stage, in which the signaling for whether to switch satellites and the signaling for the information required for switching satellites are transmitted. The second stage is the transmission stage, in which the transmission of service data is achieved.

[0036] The switching frame needs to be located at the beginning of each time slot, and the connection link between the user and the satellite within each time slot is considered unchanged. Within the current switching frame, the terminal device can transmit the signaling for the information required for switching satellites multiple times until the transmission is successful and the action of switching satellites is completed. If the terminal device fails to complete the action of switching satellites when the current switching frame ends, it is regarded as a switching failure, and the signaling for the information required for switching satellites needs to be transmitted again in the next switching frame. After the terminal device completes the action of switching satellites within the current switching frame, the remaining time of this switching frame is used for the transmission of service data. If the terminal device does not need to switch satellites within the current switching frame, the current switching frame is used for the transmission of service data. Therefore, the switching time refers to the time consumed by the terminal device to complete the action of switching satellites within the current switching frame.

[0037] S3: If the number of second channel loads is greater than the preset channel load threshold, determine the preset negative reward as the action reward for the satellite communication switching.

[0038] Among them, the channel load threshold is the total number of channels used by the corresponding satellite for communication. If the number of second channel loads is greater than the preset channel load threshold, it will cause excessive satellite channel loads, increase the communication pressure of the satellite, and also reduce the communication quality of the satellite. Therefore, the corresponding action reward for switching satellites is a negative reward, and the negative reward can be -50, -100, -120, etc.

[0039] S4: If the number of second channel loads is less than or equal to the channel load threshold, obtain the action reward for the satellite communication switching according to the preset switching communication duration value, the data transmission rate between the terminal device and the second satellite, the number of second channel loads, and the switching time and energy loss of the terminal device for satellite communication switching.

[0040] Among them, when the number of second-channel loads is less than or equal to the channel load threshold, it indicates that the communication pressure of the satellite is within the range that the satellite can bear. At this time, the action reward for switching satellite communication can be calculated according to the switching communication duration value, data transmission rate, number of second-channel loads, and the switching time and energy consumption of the terminal device for switching satellite communication. Among them, the switching communication duration value is the duration of the switching frame.

[0041] Calculate the corresponding action reward according to the number of second-channel loads after the action of realizing the switching of satellite communication and the energy consumption of executing the satellite switching. The calculated action reward combines the information of the number of second-channel loads and energy consumption, and can be used to guide the training direction of the network model.

[0042] S5: Through the first network model, obtain the first action scores corresponding to a number of the first satellite state information and the action of switching satellite communication, and the second action scores corresponding to a number of the second satellite state information.

[0043] Such as Figure 2 shown, Figure 2 the estimation network of Figure 2 is the first network model of this application, the target network of is the second network model of this application, the environment is the satellite state information, and the experience pool is a training data storage space for storing the first satellite state information, the action of switching satellite communication, the second satellite state information, and the action reward. A number of the first satellite state information the action of switching satellite communication t as a group Save the experience to the experience pool, and then select several groups of experiences from the experience pool. Input the first satellite state information and the action of switching satellite communication in each group of experiences into the first network model one by one to obtain the first action scores output by the first network model, and then input the second satellite state information into the first network model to obtain the second action scores output by the first network model.

[0044] S6: Among the several second action scores output by the first network model, input the action corresponding to the highest second action score and the corresponding several second satellite state information into the second network model to obtain the third action score output by the second network model.

[0045] Among them, the network parameters of the second network model are updated according to a preset time period. The update process is to use the network parameters of the first network model as the new network parameters of the second network model. Therefore, the second network model can score the action of switching satellite communication with the highest second action score output by the first network model through network parameters with a longer update period to obtain a third action score.

[0046] S7: Obtain a target action score according to the third action score and the action reward.

[0047] Among them, the sum of the third action score after discount calculation and the action reward is the target action score.

[0048] S8: Train the first network model according to the first action score and the target action score.

[0049] Among them, training the first network model according to the first action score and the target action score can make the output of the first network model after training closer to the target action score.

[0050] S9: Determine the trained first network model as the communication satellite handover evaluation network.

[0051] Compared with the prior art, the present application calculates the corresponding action reward according to the number of second channel loads after the action of realizing the switching of satellite communication and the energy loss of executing the satellite handover, and combines the third action score output by the first network model and the second network model to train the first network model to obtain a communication satellite handover evaluation network. Among them, the third action score is the score obtained by the second network model estimating the action with the highest second action score estimated by the first network model. Therefore, the third action score is a comprehensive score combining the model characteristics of the first network model and the second network model, and the action reward is obtained according to the number of second channel loads and the energy loss. Therefore, the communication satellite handover evaluation network trained according to the third action score and the action reward can output an action score in combination with the number of second channel loads and the energy loss, which is beneficial to selecting an action of communicating with a satellite with a smaller channel load number and lower energy loss according to multiple satellite status information corresponding to the terminal device. Moreover, since the training process combines the second network model with a longer network parameter update period for action scoring, it can avoid the situation that the training speed of the first network model is too fast, resulting in unstable training and a single parameter training direction.

[0052] In a feasible embodiment, the step of S4: obtaining the action reward for switching satellite communication according to a preset handover communication duration value, the data transmission rate between the terminal device and the second satellite, the number of second channel loads, and the handover time and energy loss of the terminal device for switching satellite communication includes:

[0053] S41: Obtain the maximum second communication data according to the switching communication duration value, the switching time consumption, and the data transmission rate between the terminal device and the second satellite.

[0054] The maximum second communication data is obtained through the following formula:

[0055] D = C t × (T t - T' t )

[0056] In the above formula, D is the maximum second communication data, C t is the data transmission rate, T t is the switching communication duration value, and T' t is the switching time consumption.

[0057] S42: Obtain the second channel load influence value according to the preset amplification factor and the second channel load quantity.

[0058] Among them, the second channel load influence value is obtained through the following formula:

[0059] M = η × l t

[0060] M is the second channel load influence value, η is the preset amplification factor, and l t is the channel load quantity of the satellites for connection communication, which is the second channel load quantity in step S42. Among them, the amplification factor can take values such as 30, 40, 50, etc.

[0061] S43: Obtain the action reward according to the maximum second communication data, the energy loss, and the second channel load influence value.

[0062] Among them, the action reward is obtained through the following formula:

[0063] R t = D - E - M

[0064] R t is the action reward, and E is the energy loss.

[0065] In this embodiment, after calculating the maximum second communication data through the switching communication duration value, the switching time consumption, and the data transmission rate between the terminal device and the second satellite, and calculating the second channel load influence value according to the preset amplification factor and the second channel load quantity, combined with the energy loss of switching satellite communication, the corresponding action reward is obtained more comprehensively and accurately.

[0066] In a feasible embodiment, the energy loss of the terminal device for switching satellite communication is obtained through the following steps:

[0067] S401: Obtain the switching power of the terminal device for switching satellite communication.

[0068] Among them, the switching power of the terminal device for switching satellite communication can be obtained by testing the terminal device for switching satellite communication multiple times, or can be obtained from the device parameters of the terminal device provided by the manufacturer.

[0069] S402: Obtain the energy loss according to the switching power and the switching time.

[0070] Among them, the energy loss is obtained through the following formula:

[0071] E = T′ t ×P t

[0072] E is the energy loss, P t is the action power; T′ t is the switching time.

[0073] In a feasible embodiment, before the step S6: Among the several second action scores output by the first network model, input the action corresponding to the highest second action score and the corresponding several second satellite state information into the second network model to obtain the third action score output by the second network model, further including:

[0074] S601: Obtain the action reward for not switching satellite communication according to the number of first channel loads, data transmission rate of the first satellite connected for communication when the terminal device does not switch satellite communication, and a preset switching communication duration value.

[0075] Among them, since there is no energy loss for switching satellite communication in the action of not switching satellite communication, the action reward for not switching satellite communication can be calculated only according to the number of first channel loads, data transmission rate of the first satellite connected for communication by the terminal device, and a preset switching communication duration value.

[0076] S602: Input the action of not switching satellite communication and the corresponding several first satellite state information into the first network model to obtain the first action score and the second action score output by the first network model.

[0077] In this embodiment, by obtaining the action reward for not switching satellite communication and inputting the corresponding action and several pieces of the first satellite state information into the first network model, the second action score for not switching satellite communication output by the first network model can be obtained. The highest second action score output by the first network model incorporates the situation of not switching satellite communication, so as to more comprehensively obtain the second action scores for switching and not switching satellite communication.

[0078] In a feasible embodiment, the step S601: obtaining the action reward for not switching satellite communication according to the number of first channel loads, data transmission rate, and a preset switching communication duration value of the first satellite to which the terminal device is connected when not switching satellite communication includes:

[0079] S6011: obtaining the maximum value of the first communication data between the terminal device and the first satellite according to the switching communication duration value and the data transmission rate between the terminal device and the first satellite.

[0080] D′ = C t ×T t

[0081] D′ is the maximum value of the first communication data, C t is the data transmission rate, T t is the switching communication duration value.

[0082] S6012: obtaining the first channel load influence value according to a preset amplification factor and the number of first channel loads.

[0083] M′ = η × l t

[0084] M′ is the first channel load influence value, η is the preset amplification factor, l t is the number of channel loads of the satellite connected for communication, which is the number of first channel loads in step S42. Among them, the amplification factor can take values such as 30, 40, 50, etc.

[0085] S6013: obtaining the action reward according to the maximum value of the first communication data and the first channel load influence value.

[0086] R t = D′ - M′

[0087] R t is the action reward.

[0088] Combining the above content, the reward function for the actions of the terminal device to switch and not switch satellite communication can be represented by the following reward function:

[0089]

[0090] Among them, R t is the action reward, E is the energy loss, C t is the data transmission rate, T t is the switching communication duration value, T′ t is the switching time consumption, η is a preset amplification factor, l t is the number of channel loads of the satellites for connection communication.

[0091] In a feasible embodiment, the step S8: training the first network model according to the first action score and the target action score includes:

[0092] S81: Construct a target function according to the first action score and the target action score, and obtain the first function output of the target function.

[0093] Among them, an absolute value operation can be performed on the difference function of the first action score and the target action score to obtain the target function and the first function output of the target function, that is, through the following formula, the first function output is obtained:

[0094] δ = |Q(s t , a t ) - y t | = |Q(s t , a t , θ) - (R t+ γQ′(s t+1 , argmax a Q(s t+1 , a t ; θ); θ′))|

[0095] Among them, δ is the first function output, Q(s t , a t ) is the first action score, y t is the target action score, s t is the first satellite state information, θ is the network parameter of the first network model, a t is the action, R t is the action reward, γ is the discount rate, argmax a Q(s t+1 , a t ; θ) is the action corresponding to the highest second action score output by the first network model, Q′(s t+1 , argmax a Q(s t+1 , a t ; θ); θ′) is the third action score.

[0096] S82: Update the network parameters of the first network model according to the output of the first function to obtain the trained first network model.

[0097] Among them, since the output of the first function combines the first action score and the target action score, and the target action score is obtained by combining the number of channel loads and energy losses of switching satellite communication and the number of channel loads of not switching satellite channels, therefore, according to the output of the first function of the first action score and the target action score, the comprehensiveness of training the first network model can be improved.

[0098] In a feasible embodiment, the step of S82: updating the network parameters of the first network model according to the output of the first function to obtain the trained first network model includes:

[0099] S821: Determine the loss function according to the output of the first function to obtain the second function output of the loss function.

[0100] Among them, through the following formula, the second function output is obtained:

[0101]

[0102] Among them, loss is the second function output, and δ is the first function output.

[0103] S822: According to the second function output, use the gradient descent algorithm to update the network parameters of the first network model to obtain the trained first network model whose second function output is less than or equal to a preset function threshold.

[0104] Among them, the function threshold is set by the user.

[0105] In this embodiment, by using the gradient descent algorithm to update the network parameters of the first network model, the output of the first network model after update can be closer to the target action score.

[0106] In a feasible embodiment, after the step of S8: training the first network model according to the first action score and the target action score, it further includes:

[0107] S83: Based on a preset time period, update the network parameters of the second network model according to the network parameters of the first network model.

[0108] In this embodiment, since the network parameters of the first network model are updated in real time as the training progresses, and the network parameters of the second network model are updated according to a preset time period, it can avoid the situation that the training speed of the first network model is too fast, resulting in unstable training and a single parameter training direction, making the training of the first network model more comprehensive and stable.

[0109] Please refer to Figure 3 and 4 , the second embodiment of the present application provides a satellite communication connection method, including:

[0110] S100: Obtain a communication satellite handover evaluation network trained according to the communication satellite handover model training method described above.

[0111] S200: Input the real-time satellite status information corresponding to the terminal device into the communication satellite handover evaluation network to obtain a policy action corresponding to the highest action score.

[0112] Please refer to Figure 4 , step S200 can be decomposed into Figure 4 the four steps of handover information collection, information processing, handover decision, and handover execution in, among which, the real-time satellite status information corresponding to the terminal device can be obtained through Figure 4 the two steps of handover information collection and information processing shown, and the process of obtaining the highest action score through the communication satellite handover evaluation network in step S200 corresponds to Figure 4 the step of handover decision shown, and obtaining the policy action corresponding to the highest action score corresponds to Figure 4 the step of handover execution shown.

[0113] S300: Drive the terminal device to execute the policy action to connect to the satellite pointed to by the policy action.

[0114] It should be noted that the satellite communication connection method provided in the second embodiment of the present application and the communication satellite handover model training method of the first embodiment of the present application belong to the same concept. The implementation process is shown in detail in the first embodiment and will not be repeated here.

[0115] The device embodiments described above are merely illustrative. The components described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of the present application. Those of ordinary skill in the art can understand and implement it without creative efforts.

[0116] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0117] The present application is described with reference to the flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each flow and / or block in the flowchart and / or block diagram, as well as the combination of flows and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, such that the instructions executed by the processor of the computer or other programmable data processing devices generate means for implementing the selected functions in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks. These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, such that the instructions stored in the computer-readable memory generate a manufactured article including instruction means that implement the selected functions in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0118] These computer program instructions can also be loaded onto a computer or other programmable data processing device, such that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process, so that the instructions executed on the computer or other programmable device provide steps for implementing the selected functions in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0119] In a typical configuration, a computing device includes one or more processors (CPUs), an input / output interface, a network interface, and a memory.

[0120] The memory may include non-permanent memory in the form of computer-readable media, random access memory (RAM), and / or non-volatile memory such as read-only memory (ROM) or flash memory (flash RAM). The memory is an example of computer-readable media.

[0121] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0122] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0123] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A communication satellite switching model training method, characterized in that: include: Acquire a plurality of first satellite status information corresponding to the terminal device and a plurality of second satellite status information corresponding to the terminal device after the terminal device implements the action of switching satellite communication; Acquire the second channel load quantity and energy loss of the second satellite communicating after the switching from the plurality of second satellite status information; If the second channel load quantity is greater than a preset channel load threshold, determining a preset negative reward as the action reward for switching satellite communication; If the second channel load quantity is less than or equal to the channel load threshold, obtaining an action reward for switching satellite communication according to a preset switching communication duration value, a data transmission rate between the terminal device and the second satellite, the second channel load quantity, and a switching time and energy loss of the terminal device switching satellite communication; Obtaining, by means of a first network model, first action scores corresponding to a plurality of pieces of the first satellite state information and the action of switching satellite communications, and second action scores corresponding to a plurality of pieces of the second satellite state information; From the plurality of second action scores output by the first network model, inputting the action corresponding to the highest second action score and the corresponding plurality of second satellite state information into the second network model to obtain a third action score output by the second network model; Obtaining a target action score according to the third action score and the action reward; Training the first network model according to the first action score and the target action score; The first network model after training is determined as a communication satellite switching evaluation network.

2. The communication satellite switching model training method according to claim 1, characterized in that: The step of obtaining the action reward for switching satellite communication according to the preset switching communication duration value, the data transmission rate between the terminal device and the second satellite, the load quantity of the second channel, and the switching time and energy loss of the terminal device switching satellite communication includes: Obtaining a second communication data maximum value according to the switching communication duration value, the switching time, and the data transmission rate between the terminal device and the second satellite; Obtaining a second channel load impact value according to a preset amplification factor and the second channel load quantity; The action reward is obtained according to the second communication data maximum value, the energy loss and the second channel load impact value.

3. The communication satellite switching model training method according to claim 1, characterized in that: The energy loss of the terminal device switching satellite communication is obtained by the following steps: Acquiring a switching power for the terminal device to switch satellite communication; The energy loss is obtained according to the switching power and the switching time.

4. The communication satellite switching model training method according to claim 1, characterized in that: Before the step of inputting the action corresponding to the highest second action score and the corresponding second satellite state information into the second network model from the plurality of second action scores output by the first network model to obtain the third action score output by the second network model, the method further includes: According to the load quantity of the first channel of the first satellite connected to the communication when the terminal device does not switch the satellite communication, the data transmission rate and the preset switching communication duration value, the action reward of not switching the satellite communication is obtained; The action of not switching satellite communication and the corresponding plurality of first satellite status information are input into the first network model to obtain a first action score and a second action score output by the first network model.

5. The communication satellite switching model training method according to claim 4, characterized in that: The step of obtaining the action reward for not switching satellite communication according to the first channel load quantity, data transmission rate and preset switching communication duration value of the first satellite connected to the communication when the terminal device does not switch satellite communication comprises: Obtaining a maximum value of first communication data between the terminal device and the first satellite according to the switching communication duration value and the data transmission rate between the terminal device and the first satellite; Obtaining a first channel load impact value according to a preset amplification factor and the first channel load quantity; The action reward is obtained according to the first communication data maximum value and the first channel load impact value.

6. The communication satellite switching model training method according to claim 1, characterized in that: The step of training the first network model according to the first action score and the target action score comprises: constructing an objective function according to the first action score and the target action score, and obtaining a first function output of the objective function; The network parameters of the first network model are updated according to the output of the first function to obtain a trained first network model.

7. The communication satellite switching model training method according to claim 6, characterized in that: The step of constructing an objective function according to the first action score and the target action score to obtain a first function output of the objective function comprises: An absolute value operation is performed on a difference function between the first action score and the target action score to obtain the target function and a first function output of the target function.

8. The communication satellite switching model training method according to claim 6, characterized in that: The step of updating the network parameters of the first network model according to the first function output to obtain the trained first network model includes: Determine a loss function according to the first function output, and obtain a second function output of the loss function; According to the output of the second function, a gradient descent algorithm is used to update the network parameters of the first network model to obtain a trained first network model in which the output of the second function is less than or equal to a preset function threshold.

9. The communication satellite switching model training method according to any one of claims 1 to 8, characterized in that: After the step of training the first network model according to the first action score and the target action score, the method further includes: Based on a preset time period, the network parameters of the second network model are updated according to the network parameters of the first network model.

10. A satellite communication connection method, characterized in that: include: Acquire a communication satellite handover evaluation network trained by the communication satellite handover model training method according to any one of claims 1 to 9; Inputting the real-time satellite status information corresponding to the terminal device into the communication satellite switching evaluation network to obtain the strategic action corresponding to the highest action score; The terminal device is driven to execute the strategic action to connect to the satellite pointed to by the strategic action.

Images