Beidou short message communication method and device, equipment and storage medium

By loading multiple agent collaboration algorithms for each satellite in the Beidou short message communication system, the resource allocation is optimized, and the problem of large delay in the Beidou short message communication system is solved, and effective support for delay-sensitive services is achieved. Meet user communication needs.

CN118971943APending Publication Date: 2024-11-15HUNAN DAJIA DATA TECH CO LTD
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202411086206.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-08
Publication Date
2024-11-15

AI Technical Summary

Technical Problem

During the data processing, the Beidou short message communication system relies on the central cloud platform of the ground station, resulting in a large delay, which cannot meet the needs of delay-sensitive services.

Method used

In the Beidou short message communication system, each satellite loads a multi-agent collaborative algorithm, allocates resources through interactive collaboration information, determines the satellites used for services, optimizes the allocation of computing, storage and communication resources, and realizes the allocation of multiple satellites between them multi-agent collaboration and intelligent computing.

Benefits of technology

It reduces latency, improves system effectiveness, meets the needs of delay-sensitive services, and better meets the communication needs of users.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118971943A_ABST
    Figure CN118971943A_ABST
Patent Text Reader

Abstract

The invention provides a Beidou short message communication method, device and equipment and a storage medium, the Beidou short message communication method is applied to any satellite in a Beidou short message communication system, and each satellite in the Beidou short message communication system is loaded with the same multi-agent collaborative algorithm. A plurality of multi-agent collaborative algorithms are constructed based on interactive collaborative information of a plurality of satellites; the plurality of satellites distribute resources of the plurality of satellites through the loaded multi-agent collaborative algorithm so as to determine satellites used for serving each user side serving the Beidou short message communication system at the current moment; the Beidou short message communication method comprises the following steps: acquiring target message data sent by an initiating end, wherein the initiating end is a user end; and transmitting the target message data to the target end through a satellite which corresponds to the user end and is used for service. According to the invention, the satellite resources can be reasonably distributed, the time delay is reduced, and the effectiveness of the whole system is maximized.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of satellite communication technology, and in particular to a Beidou short message communication method, device, equipment and storage medium. Background Art

[0002] Compared with the GPS system, the BeiDou satellite navigation system can not only realize continuous and accurate positioning, navigation and time calibration services on a global scale, but also has a unique short message communication function. The short message communication function is based on code division multiple access technology and can provide regional and global short message communication services, allowing users to send and receive short messages within a region or globally. The communication service has been widely used in many fields such as meteorology, hydrology, geology and oceanography.

[0003] Although the Beidou short message communication system provides a solution for global communications, data processing in some communication processes still needs to rely on the central cloud platform of the ground station. After collecting data, such as image information, users need to process the data, but due to limited personal computing resources and computing power, the data needs to be sent back to the central cloud platform for processing. In this process, the data sent by the user needs to be relayed by multiple satellites before it can reach the central cloud of the ground station for processing. The ground station then sends the processed data to multiple satellites for relay transmission and then transfers the data to the user. This processing method will cause a large time delay and cannot meet the needs of delay-sensitive services. Summary of the invention

[0004] The present application aims to propose a Beidou short message communication method, device, equipment and storage medium, which can reasonably allocate satellite resources, maximize the overall system utility, achieve effective support for delay-sensitive services, and better meet the communication needs of users.

[0005] The Beidou short message communication method according to the first aspect of the present application is applied to any satellite in the Beidou short message communication system, each satellite in the Beidou short message communication system is loaded with the same multi-agent collaborative algorithm, and multiple multi-agent collaborative algorithms are constructed based on interactive collaboration information of multiple satellites, and the interactive collaboration information represents the communication information of multiple satellites in the Beidou short message communication system;

[0006] The multiple satellites allocate resources of the multiple satellites through the loaded multi-agent collaborative algorithm to determine the satellite used to serve each user terminal served by the Beidou short message communication system at the current moment, wherein the resources include computing resources, storage resources and communication resources of the satellite;

[0007] The Beidou short message communication method comprises the following steps:

[0008] Acquire target message data sent by an initiator, where the initiator is the user end;

[0009] The target message data is transmitted to the target terminal through the satellite for service corresponding to the user terminal.

[0010] The Beidou short message communication method according to the embodiment of the present application has at least the following beneficial effects:

[0011] The multi-agent collaborative algorithm loaded on each satellite in the Beidou short message communication system, each satellite is regarded as an agent among the multi-agents, and can obtain the communication status of multiple satellites according to the interactive collaboration information of multiple satellites, and thereby allocate the resources of multiple satellites, including the computing resources, storage resources and communication resources of the satellites, and then determine which satellite is selected for service for each user end served by the Beidou short message communication system at the current moment. The user end, i.e., the initiator, sends the target message data in the form of a short message signal, and the satellite used to serve this user end determined by the multi-agent collaborative algorithm performs data processing, transmits the target message data to the target end, and completes the communication needs of the user end. The Beidou short message communication method of the embodiment of the present application, through the multi-agent collaborative algorithm, can realize the multi-agent collaboration and intelligent computing optimization between multiple satellites in the Beidou short message communication system, make full use of the freedom of resources on the satellite, and can dynamically and reasonably allocate resources according to the communication information of multiple satellites, so that the overall utility of the system is maximized, the delay is reduced, and effective support for delay-sensitive services is achieved, and the communication needs of users are better met.

[0012] According to some embodiments of the present application, before obtaining the target message data sent by the initiator, the following steps are also included:

[0013] Determining the multi-agent collaboration algorithm according to the interactive collaboration information of the plurality of satellites;

[0014] Iterate the multi-agent collaborative algorithm loaded on the plurality of satellites to obtain a resource adjustment action for each satellite;

[0015] The satellite for serving each of the user terminals is determined according to the resource adjustment action corresponding to each of the satellites.

[0016] According to some embodiments of the present application, determining the multi-agent collaboration algorithm according to the interactive collaboration information of the plurality of satellites comprises the following steps:

[0017] Acquire a random strategy function, wherein the random strategy function is used to obtain an action decision according to the computing resources, the storage resources, the communication resources of the satellite, the amount of short message data of multiple satellites, and the number of user terminals served by multiple satellites, and the action decision is used to determine the amount of the computing resources, the storage resources, and the communication resources allocated by the satellite to the user terminal;

[0018] Acquire a reward function, wherein the reward function reflects satisfaction of the user terminal of the satellite service;

[0019] Obtaining a transition probability, where the transition probability is the probability that the satellite will transition to a state at a next moment after executing the action decision;

[0020] The multi-agent collaborative algorithm is determined based on the random strategy function, the reward function and the transition probability.

[0021] According to some embodiments of the present application, the iterative operation of the multi-agent collaborative algorithm loaded on the plurality of satellites to obtain a resource adjustment action for each satellite includes:

[0022] Acquire a first observation state, where the first observation state includes the computing resources, the storage resources, the communication resources of the satellite itself, the amount of short message data of multiple satellites, and the number of user terminals served by multiple satellites;

[0023] Based on the random strategy function, obtaining the action decision;

[0024] Execute the action decision and obtain a corresponding reward, wherein the reward is obtained according to the reward function;

[0025] According to the transition probability, the state at the next moment is jumped, and the above process is repeated until the number of iterations reaches a preset iteration threshold or the multi-agent collaborative algorithm meets the preset convergence requirement, and the resource adjustment action of each satellite is obtained.

[0026] According to some embodiments of the present application, after acquiring the first observation state, the method further includes:

[0027] Broadcasting the first observation status over the entire network via an intersatellite link;

[0028] Acquire a second observation state, where the second observation state includes the computing resources, the storage resources, the communication resources of the other satellites, the amount of short message data of the multiple satellites, and the number of user terminals served by the multiple satellites, and the second observation state is broadcasted by the other satellites over the entire network through intersatellite links;

[0029] According to the first observation state and the second observation state, a global state is determined, and the global state includes the computing resources, the storage resources, the communication resources and the short message data volume of all the satellites, and the number of user terminals served by all the satellites in the Beidou short message communication system.

[0030] According to some embodiments of the present application, in the action decision, when it is decided to allocate the computing resources and the storage resources to the user terminal and not allocate the communication resources, the user terminal is served by the ground station.

[0031] According to some embodiments of the present application, the multi-agent collaborative algorithm satisfies a preset convergence requirement if the reward function satisfies a preset condition.

[0032] The Beidou short message communication device according to the second aspect of the present application is applied to any satellite in the Beidou short message communication system, each satellite in the Beidou short message communication system is loaded with the same multi-agent collaborative algorithm, and multiple multi-agent collaborative algorithms are constructed based on interactive collaboration information of multiple satellites, and the interactive collaboration information represents the communication information of multiple satellites in the Beidou short message communication system;

[0033] The multiple satellites allocate resources of the multiple satellites through the loaded multi-agent collaborative algorithm to determine the satellite used to serve each user terminal served by the Beidou short message communication system at the current moment, wherein the resources include computing resources, storage resources and communication resources of the satellite;

[0034] The Beidou short message communication device comprises:

[0035] A receiving unit, used for acquiring target message data sent by an initiator, wherein the initiator is the user end;

[0036] The sending unit is used to transmit the target message data to the target terminal through the satellite used for service corresponding to the user terminal.

[0037] Since the Beidou short message communication device adopts all the technical solutions of the Beidou short message communication method of the above embodiment, it at least has all the beneficial effects brought by the technical solutions of the above embodiment.

[0038] According to an electronic device of the third aspect embodiment of the present application, the electronic device includes a processor and a memory, wherein the memory stores a program or instruction that can be run on the processor, and when the program or instruction is executed by the processor, the steps of the Beidou short message communication method as described in any one of the first aspect embodiments are implemented. Since the electronic device adopts all the technical solutions of the Beidou short message communication method of the above embodiment, it at least has all the beneficial effects brought by the technical solutions of the above embodiment.

[0039] According to the computer-readable storage medium of the fourth aspect embodiment of the present application, computer executable instructions are stored, and the computer executable instructions are used to execute the Beidou short message communication method as described in the first aspect embodiment. Since the computer-readable storage medium adopts all the technical solutions of the Beidou short message communication method of the above embodiment, it at least has all the beneficial effects brought by the technical solutions of the above embodiment.

[0040] Other features and advantages of the present application will be set forth in the following description, and in part will become apparent from the description, or may be understood by practicing the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0042] Figure 1 It is a flowchart of an embodiment of the Beidou short message communication method of the present application;

[0043] Figure 2 It is a structural schematic diagram of an embodiment of the Beidou short message communication device of the present application;

[0044] Figure 3 It is a schematic diagram of the hardware structure of an embodiment of the electronic device of the present application. DETAILED DESCRIPTION

[0045] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be understood as limiting the present application.

[0046] In the description of this application, if there is a description of first, second, etc., it is only for the purpose of distinguishing technical features, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features or implicitly indicating the order of the indicated technical features.

[0047] In the description of the present application, it should be understood that the descriptions involving orientation, such as the orientation or positional relationship indicated as up, down, etc., are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present application.

[0048] In the description of this application, it should be noted that, unless otherwise clearly defined, terms such as setting, installing, connecting, etc. should be understood in a broad sense, and technicians in the relevant technical field can reasonably determine the specific meaning of the above terms in this application based on the specific content of the technical solution.

[0049] The technical solution of the present application will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described below are only part of the embodiments of the present application, not all of the embodiments.

[0050] Figure 1 This is a flowchart of the Beidou short message communication method of the embodiment provided in this application. Figure 1 , the embodiments of the present application are further described.

[0051] The embodiment of the present application proposes a Beidou short message communication method, which is applied to any satellite in the Beidou short message communication system. Each satellite in the Beidou short message communication system is loaded with the same multi-agent collaborative algorithm. Multiple multi-agent collaborative algorithms are constructed based on interactive collaboration information of multiple satellites. The interactive collaboration information represents the communication information of multiple satellites in the Beidou short message communication system.

[0052] Multiple satellites allocate resources of multiple satellites through the loaded multi-agent collaborative algorithm to determine the satellite used to serve each user terminal of the Beidou short message communication system at the current moment, where the resources include the satellite's computing resources, storage resources and communication resources;

[0053] The Beidou short message communication method includes the following steps:

[0054] Step 101, obtaining target message data sent by an initiator, where the initiator is a user end;

[0055] Step 102: Transmit the target message data to the target end through the satellite for service corresponding to the user end.

[0056] The multi-agent collaborative algorithm loaded on each satellite in the Beidou short message communication system, each satellite is regarded as an agent among the multi-agents, and can obtain the communication status of multiple satellites according to the interactive collaboration information of multiple satellites, and thereby allocate the resources of multiple satellites, including the computing resources, storage resources and communication resources of the satellites, and then determine which satellite is selected for service for each user end served by the Beidou short message communication system at the current moment. The user end, i.e., the initiator, sends the target message data in the form of a short message signal, and the satellite used to serve this user end determined by the multi-agent collaborative algorithm performs data processing, transmits the target message data to the target end, and completes the communication needs of the user end. The Beidou short message communication method of the embodiment of the present application, through the multi-agent collaborative algorithm, can realize the multi-agent collaboration and intelligent computing optimization between multiple satellites in the Beidou short message communication system, make full use of the freedom of resources on the satellite, and can dynamically and reasonably allocate resources according to the communication information of multiple satellites, so that the overall utility of the system is maximized, the delay is reduced, and effective support for delay-sensitive services is achieved, and the communication needs of users are better met.

[0057] The core of the above-mentioned multi-agent collaborative algorithm is to use multi-agent collaboration to achieve collaboration between Beidou satellites. Each agent represents a Beidou satellite and is responsible for the resource allocation and task offloading decisions of the satellite. Based on the multi-agent collaborative algorithm, each agent, that is, each Beidou satellite, can take into account the uncertainty in the operation process and make more reasonable decisions, thereby achieving rapid response to time-sensitive applications and improving the efficiency of the entire system.

[0058] In the above step 101, target message data sent by an initiator is obtained, where the initiator is a user end.

[0059] The target message data is the data that the user end needs to transmit to the target end.

[0060] In the above step 102, the target message data is transmitted to the target end through the satellite for service corresponding to the user end.

[0061] The satellite for service corresponding to the above-mentioned user end is determined through multi-agent collaborative algorithm calculation, and completes the direct communication service from the user end to the target end.

[0062] The above-mentioned satellite used to serve each user terminal of the Beidou short message communication system at the current moment is determined. The process of data communication of the traditional Beidou short message communication system is often transmitted through the initiator, satellite, ground station, satellite, and target terminal, with a large delay. In this application, after the satellite used for service is determined through a multi-agent collaborative algorithm, it only needs to pass through the initiator, satellite, and target terminal, and only the intermediate service satellite selected between the initiator and the target terminal, so the transmission process is simpler and the delay is reduced.

[0063] For example, if the corresponding satellite used for service is satellite a, the user end sends the short message data to satellite a, which is processed by satellite a and then sent to the target end.

[0064] When the user terminal accesses the Beidou short message communication system for the first time and the corresponding satellite for service has not been determined, a request message can be sent from the user terminal to any satellite in the system. After receiving the user's request message, the system immediately confirms the satellite for service corresponding to the user terminal service request through a multi-agent collaborative algorithm. The subsequent formal communication service is carried out by the corresponding satellite for service.

[0065] The target message data is transmitted to the target terminal through the satellite for service corresponding to the user terminal. In some cases, a ground station instead of a satellite may be selected to serve the user terminal.

[0066] In some implementations, before obtaining the target message data sent by the initiator, the following steps are also included:

[0067] Determine the multi-agent collaboration algorithm based on the interactive collaboration information of multiple satellites;

[0068] Iterate the multi-agent collaborative algorithm loaded on multiple satellites to obtain the resource adjustment actions of each satellite;

[0069] The satellite for serving each user terminal is determined according to the resource adjustment action corresponding to each satellite.

[0070] In this embodiment, before providing services for the communication needs of the user side, a multi-agent collaborative algorithm will be determined in advance and the algorithm will be iteratively calculated to obtain a satellite for providing services. Communication will then be carried out through the satellite. By determining and iteratively calculating the multi-agent collaborative algorithm, a resource allocation method that maximizes the effect of the entire system can be obtained, thereby reducing the delay in the user side communication service process.

[0071] The resource adjustment action of each satellite mentioned above can allocate and adjust the computing resources, storage resources, and communication resources of each satellite. Specifically, it can determine the resource allocation situation of each satellite for the user end it serves, that is, the computing resources, storage resources, and communication resources allocated by each satellite to each user end it serves.

[0072] The interactive collaboration information of the above-mentioned multiple satellites may include historical data obtained based on resource adjustment actions in the historical interactive collaboration process of the multiple satellites and the state transition process after the resource adjustment actions, and the multi-agent collaboration algorithm is determined by analyzing these historical data.

[0073] The satellite for serving each user terminal is determined according to the resource adjustment action corresponding to each satellite. In some cases, a ground station may be selected instead of a satellite to serve the user terminal.

[0074] In some implementations, determining a multi-agent collaboration algorithm based on interactive collaboration information of multiple satellites includes the following steps:

[0075] Obtaining a random strategy function, the random strategy function is used to obtain an action decision according to the satellite's computing resources, storage resources, communication resources, the amount of short message data of multiple satellites, and the number of user terminals served by multiple satellites, and the action decision is used to determine the amount of computing resources, storage resources, and communication resources allocated by the satellite to the user terminal;

[0076] Obtain a reward function, which reflects the user satisfaction of the satellite service;

[0077] Obtain the transition probability, which is the probability that the satellite will transition to the next state after executing the action decision;

[0078] Determine the multi-agent collaborative algorithm based on the random strategy function, reward function and transition probability.

[0079] In this embodiment, the multi-agent collaborative algorithm is determined by determining key parameters, specifically including a random strategy function, a reward function, and a transition probability, so that the multi-agent collaborative algorithm can achieve better computing results.

[0080] The above-mentioned random strategy function can be obtained from the interactive cooperation information of multiple satellites. It can be obtained by obtaining historical data such as resource adjustment actions in the historical interactive cooperation process of multiple satellites and the state transition process after the resource adjustment action. The strategy function is obtained by analyzing these historical data.

[0081] The above-mentioned reward function may include a user satisfaction item and a penalty item. The user satisfaction item is determined by the degree to which the satellite satisfies the needs of each user terminal. Specifically, the user satisfaction item may be higher when the user satisfaction is higher. The penalty item is determined by the number of user terminals served by the satellite. Specifically, the penalty item may be greater when the number of user terminals served by the satellite is smaller.

[0082] The above-mentioned transfer probability can be obtained from the interactive cooperation information of multiple satellites, and can be obtained by obtaining historical data such as resource adjustment actions in the historical interactive cooperation process of multiple satellites and the state transition process after the resource adjustment action. By analyzing these historical data, a dynamic transfer probability set is obtained.

[0083] In some implementations, the multi-agent collaborative algorithm loaded on multiple satellites is iteratively operated to obtain resource adjustment actions for each satellite, including:

[0084] Acquire a first observation state, where the first observation state includes the satellite's own computing resources, storage resources, communication resources, the amount of short message data of multiple satellites, and the number of user terminals served by the multiple satellites;

[0085] Based on the random policy function, the action decision is obtained;

[0086] Execute action decisions and obtain corresponding rewards, which are obtained according to the reward function;

[0087] According to the transition probability, it jumps to the next state and repeats the above process until the number of iterations reaches the preset iteration threshold or the multi-agent collaborative algorithm meets the preset convergence requirements, and the resource adjustment action of each satellite is obtained.

[0088] In this implementation, the satellite executes action decisions based on the observed state and obtains corresponding rewards. The system then jumps to the next state with a transfer probability, and continuously repeats this process, which can realize the iterative operation of the multi-agent collaborative algorithm. Until the number of iterations reaches a preset iteration threshold or the multi-agent collaborative algorithm meets the preset convergence requirements, the Nash equilibrium state can be achieved, and the resource adjustment action of each satellite can be obtained, which can achieve the optimal resource allocation method for the Beidou short message communication system and each satellite in the system.

[0089] In the above process, each Beidou satellite acts as a single intelligent entity. At each decision-making moment, it makes an action decision based on the first observation state, that is, its own observation state, that is, it determines the amount of computing resources, storage resources, and communication resources allocated to each user, thereby maximizing its own long-term rewards.

[0090] In the above process, each agent can use a greedy algorithm to make action decisions and run iteratively.

[0091] The first observation state is obtained, and the first observation state includes the satellite's own computing resources, storage resources, communication resources, the amount of short message data of multiple satellites, and the number of user terminals served by multiple satellites. Since there are multiple satellites, that is, multiple intelligent agents, in this Beidou short message communication system, there is a competitive and cooperative relationship between the intelligent agents, and the action decision of each intelligent agent will be affected by the joint action decision of other intelligent agents. Due to factors such as delays in the transmission of each other's state information by intersatellite links, the interaction between multiple intelligent agents is limited. Each intelligent agent cannot observe the state of other intelligent agents in real time, and can only observe part of the environmental state. Therefore, in each decision time slot, only based on the state observed by itself, that is, the first observation state obtained above, learn the optimal strategy to make an action decision that maximizes long-term returns.

[0092] The above random strategy function can obtain an action decision based on the first observed state obtained. In a decision time slot, based on the random strategy function, an action decision is obtained, and then the state of the next moment is jumped to according to the transition probability, and the above process is repeated until the number of iterations reaches the preset iteration threshold or the multi-agent collaborative algorithm meets the preset convergence requirements. At this time, the optimal strategy function can be obtained, and the resource adjustment action of each satellite can be obtained according to the optimal strategy function.

[0093] The above random strategy function can be trained for multiple rounds to obtain a more stable strategy.

[0094] For example, during training, all agents select a random sub-strategy in each round and obtain cumulative rewards from the environment; the environment state is reset after the round ends; and after training for several rounds, by integrating different sub-strategies, the goal of improving strategy stability and improving strategy generalization ability can be achieved.

[0095] The resource adjustment action of each satellite mentioned above can be an action decision obtained according to the optimal strategy function in the last iteration when the number of iterations reaches the preset iteration threshold or the multi-agent collaborative algorithm meets the preset convergence requirements. The amount of computing resources, storage resources and communication resources allocated by each satellite to the user terminal is determined according to this action decision, that is, the resource adjustment action for each satellite.

[0096] The above process continues until the number of iterations reaches a preset iteration threshold or the multi-agent collaborative algorithm meets the preset convergence requirements. The number of iterations can be set by yourself. For details, you can refer to the historical data of the algorithm operation for setting. The preset convergence requirements can also be set according to the expected iteration direction.

[0097] In the above process, each agent can use a greedy algorithm and run iteratively.

[0098] In some implementations, after acquiring the first observation state, the method further includes:

[0099] Broadcasting the first observation status to the entire network via intersatellite links;

[0100] Acquire a second observation state, which includes computing resources, storage resources, communication resources of other satellites, the amount of short message data of multiple satellites, and the number of user terminals served by multiple satellites. The second observation state is broadcasted by other satellites over the entire network through intersatellite links;

[0101] The global state is determined according to the first observation state and the second observation state. The global state includes computing resources, storage resources, communication resources of all satellites in the Beidou short message communication system, the amount of short message data of all satellites, and the number of user terminals served by all satellites.

[0102] In this implementation, after the satellite obtains the first observation state observed by itself, it broadcasts it through the intersatellite link throughout the entire network, and can promptly transmit its own information to other satellites. By obtaining the second observation state, it can promptly receive information sent by other satellites, and determine the global state based on the obtained first observation state and second observation state. This can help the satellite more comprehensively obtain the status of satellites and user terminals in the current system, better realize the collaborative work between multiple intelligent agents, and thus make better action decisions.

[0103] The above-mentioned global state can be determined jointly based on the first observation state and the second observation state. After the global state is determined, the step of executing the random strategy function to obtain the action decision can not only be based on the data of the first observation state, but can be based on the data of the global state. Since the data of the global state is more comprehensive, better action decisions can be obtained.

[0104] The above process of determining the global state may not need to be executed in every decision time slot, but only in some decision time slots. Specifically, the above part of the decision time slots may be the moments jointly agreed upon by the satellites in the system, and each satellite broadcasts the first observation state to the entire network at the agreed moment.

[0105] In some implementations, when it is decided in the action decision to allocate computing resources and storage resources to the user terminal but not to allocate communication resources, the user terminal is served by the ground station.

[0106] In this embodiment, the action decision includes not only the action of determining whether the user terminal is served by the satellite, but also the action of determining whether the user terminal is served by the ground station. This can meet the need for the ground station to serve the user terminal without selecting the satellite in special circumstances.

[0107] For the above action decision, when the computing resources and storage resources allocated by the satellite to a certain user terminal are not zero, and the number of communication resources is zero, it means that the business of the user terminal will be forwarded to the ground station for processing; when the number of computing resources, storage resources and communication resources allocated by the satellite to a certain user terminal is zero, it means that the business of the user terminal is processed by other satellites; and when the number of computing resources, storage resources and communication resources allocated by the satellite to a certain user terminal is not zero, it means that the business of the user terminal is processed by this satellite.

[0108] In some embodiments, the multi-agent collaborative algorithm satisfies a preset convergence requirement when the reward function satisfies a preset condition.

[0109] The above reward function satisfies the preset conditions to maximize the expected value of the long-term reward value, or the reward value reaches a certain preset value.

[0110] In some implementations, the state of each satellite, i.e., each agent, is described by the following tuple, which is (S, O, A, P, R, γ):

[0111] Among them, the set S represents the possible states of all satellites;

[0112] O={o m (t)|m∈M} represents the set of states observed by the satellite itself, o m (t) represents the first observation state of satellite m, and M represents the total number of satellites;

[0113] A={a1(t)×a2(t)×...×a M (t)} represents the set of joint action decisions of satellites, a m (t) represents the action decision of satellite m, and a -m (t) represents the joint action decision of the agents other than satellite m;

[0114] P={p(s(t+1)|s(t),a m (t),a -m (t))} is a set of transition probabilities, indicating that the satellite has taken a joint action decision a m (t)×a -m After (t), the probability of the state jumping from s(t) to s(t+1);

[0115] R = {r m (s(t),π m )|m∈M} represents the agent reward set, r m (s(t),π m ) is the instant reward of satellite m;

[0116] γ represents the reward discount factor;

[0117] π m represents the random strategy function of satellite m, with π m :o m (t)→a m (t).

[0118] At each decision time slot t, satellite m takes its first observation state o as the basis. m (t) and the random strategy function π m Execute action decision a m (t), in order to maximize the expected value of its long-term reward. The specific function value is as follows:

[0119]

[0120] In this process, all satellites make action decisions based on the observed state and receive corresponding rewards. The system then jumps to the next state with the transition probability, and the process is repeated continuously. After a finite number of steps, the Nash equilibrium state can be achieved when the following conditions are met. The formula is as follows:

[0121]

[0122] in, Represents the optimal policy function of m.

[0123] In this application, each BeiDou satellite acts as a single intelligent agent. At each decision moment, it determines the computing resources, storage resources, and communication resources allocated to the user based on its own observation state, and maximizes its own long-term discounted reward. Therefore, the specific definition of the multi-tuple (S, O, A, P, R, γ) is as follows:

[0124] First, define a binary group X m (t) = {V m (t),N m (t)} represents the current state of the satellite, which contains the data volume V of the short message m (t) and the number of users N m (t);

[0125] The first observation state O = {o m (t)|m∈M}. The first observation state of satellite m at time t is defined as its own computing resource b1 m (t), storage resource b2 m (t), communication resources b3 m (t) and the state of the data volume of the short message of all satellites and the number of users Z = {X1(t), X2(t), ..., X M (t)}, then there is o m (t) = {b1 m (t), b2 m (t), b3 m (t), Z};

[0126] Global state Q. The global state includes the observation state information of all satellites, including the computing resources, storage resources, communication resources of all satellites, the data volume of short messages of all satellites and the state of the number of users. The global state at time t can be defined as q(t) = {b11(t), b21(t), b31(t), b12(t), b22(t), b32(t), ..., b1 M (t), b2 M (t), b3 M(t), Z}; in the process of each satellite independently running the algorithm, it is necessary to send its own observation state o at the agreed time. m (t) = {b1 m (t), b2 m (t), b3 m (t), Z} is broadcast to the entire network through intersatellite links and is used by each satellite to construct the local global state Q, thereby realizing local algorithm iteration.

[0127] Action decision A = {a1(t)×a2(t)×...×a M (t)}. Define the action decision of satellite m at time t as a m (t) = {b1 m1 (t), b2 m1 (t), b3 m1 (t), b1 m2 (t), b2 m2 (t), b3 m2 (t), ..., b1 mNm (t), b2 mNm (t), b3 mNm (t)}, where b1 mNm (t), b2 mNm (t), b3 mNm (t) respectively represent the amount of computing resources, storage resources, and communication resources allocated by satellite m to the Nm-th user at time t. It should be pointed out here that if b1 mNm (t) and b2 mNm (t) is not 0, b3 mNm When (t) is 0, it means that the service will be forwarded to the ground station for processing, and the rest of the cases indicate that the service is processed on the satellite;

[0128] The reward function is R = {r m (s(t),π m )|m∈M}, the reward function of satellite m at time t can be defined as r m (t), which is defined as follows:

[0129]

[0130] Here, Φ a It indicates the satisfaction of satellite m in meeting the needs of user a. The value is between 0 and 1. The larger the value, the better the satellite meets the needs of the user. At the same time, a penalty term is defined:

[0131]

[0132] where u a =0 or 1, when b1 ma(t)≠0,b2 ma When (t)≠0, u a =1, otherwise 0, which means the fewer users the satellite serves, the greater the penalty.

[0133] Each satellite runs the multi-agent collaborative algorithm according to its actual state, selects its own action decision through the greedy algorithm, and runs it iteratively to ensure that its own reward function is maximized until the Nash equilibrium state is achieved and the algorithm of each satellite converges. Then each satellite obtains the resource adjustment action a corresponding to each satellite according to the final action decision. m (t) = {b1 m1 (t), b2 m1 (t), b3 m1 (t), b1 m2 (t), b2 m2 (t), b3 m2 (t), ..., b1 mNm (t), b2 mNm (t), b3 mNm (t)} to allocate resources, where b1 mNm (t), b2 mNm (t), b3 mNm (t) respectively represent the satellite m giving the Nth m The number of computing resources, storage resources, and communication resources allocated to each user, when b1 mNm (t) and b2 mNm (t) is not 0, b3 mNm When (t) is 0, it means that the service will be forwarded to the ground station for processing, and the rest of the cases indicate that the service is processed on the satellite.

[0134] In some embodiments, the Beidou short message communication method of the present application can realize multi-agent collaboration and intelligent computing optimization among multiple satellites, achieve effective support for delay-sensitive services, make full use of the freedom of satellite resources and ground station processing resources, dynamically and in real time monitor the satellite load, service latency requirements, and user concurrency, dynamically allocate resources according to the traffic requirements of access users, and achieve rapid response to time-sensitive applications through cloud-edge-end collaboration based on satellite multi-agents, while achieving improved efficiency of the entire system and better meeting user communication needs.

[0135] The Beidou short message communication method provided in the embodiment of the present application may be executed by the Beidou short message communication device 200. In the embodiment of the present application, the Beidou short message communication device 200 executing the Beidou short message communication method is taken as an example to illustrate the Beidou short message communication device 200 provided in the embodiment of the present application.

[0136] See also Figure 2, is a schematic diagram of the structure of a Beidou short message communication device 200 provided in an embodiment of the present application. Figure 2 As shown, the Beidou short message communication device 200 is applied to any satellite in the Beidou short message communication system. Each satellite in the Beidou short message communication system is loaded with the same multi-agent collaborative algorithm. Multiple multi-agent collaborative algorithms are constructed based on the interactive collaboration information of multiple satellites. The interactive collaboration information represents the communication information of multiple satellites in the Beidou short message communication system.

[0137] Multiple satellites allocate resources of multiple satellites through the loaded multi-agent collaborative algorithm to determine the satellite used to serve each user terminal of the Beidou short message communication system at the current moment, where the resources include the satellite's computing resources, storage resources and communication resources;

[0138] Beidou short message communication device includes:

[0139] The receiving unit 201 is used to obtain the target message data sent by the initiator, where the initiator is the user end;

[0140] The sending unit 202 is used to transmit the target message data to the target end through the satellite for service corresponding to the user end. Since the Beidou short message communication device 200 adopts all the technical solutions of the Beidou short message communication method of the above embodiment, it at least has all the beneficial effects brought by the technical solutions of the above embodiment, which will not be repeated here.

[0141] Figure 3 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application.

[0142] The electronic device may include a processor 301 and a memory 302 storing computer program instructions.

[0143] Specifically, the processor 301 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0144] The memory 302 may include a large capacity memory for data or instructions. By way of example and not limitation, the memory 302 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive or a combination of two or more of these. In appropriate cases, the memory 302 may include a removable or non-removable (or fixed) medium. In appropriate cases, the memory 302 may be inside or outside the integrated gateway disaster recovery device. In a specific embodiment, the memory 302 is a non-volatile solid-state memory.

[0145] In some embodiments, the memory 302 may include a read-only memory (ROM), a random access memory (RAM), a magnetic disk storage medium device, an optical storage medium device, a flash memory device, an electrical, optical or other physical / tangible memory storage device. Thus, in general, the memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., a memory device) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the method according to an aspect of the present disclosure.

[0146] The processor 301 implements any one of the Beidou short message communication methods in the above embodiments by reading and executing computer program instructions stored in the memory 302.

[0147] In one example, the electronic device may further include a communication interface 303 and a bus 310. Figure 3 As shown, the processor 301, the memory 302, and the communication interface 303 are connected via a bus 310 and communicate with each other.

[0148] The communication interface 303 is mainly used to implement communication between various modules, devices, units and / or equipment in the embodiments of the present application.

[0149] Bus 310 includes hardware, software or both, and the parts of online data flow billing equipment are coupled to each other. For example, but not limitation, bus may include accelerated graphics port (AGP) or other graphics bus, enhanced industrial standard architecture (EISA) bus, front-end bus (FSB), hypertransport (HT) interconnection, industrial standard architecture (ISA) bus, infinite bandwidth interconnection, low pin count (LPC) bus, memory bus, micro channel architecture (MCA) bus, peripheral component interconnection (PCI) bus, PCI-Express (PCI-X) bus, serial advanced technology attachment (SATA) bus, video electronics standard association local (VLB) bus or other suitable bus or two or more of these combinations. In appropriate cases, bus 310 may include one or more buses. Although the present application embodiment describes and shows a specific bus, the present application considers any suitable bus or interconnection.

[0150] The electronic device can execute the Beidou short message communication method in the embodiment of the present application, thereby realizing the combination Figure 1 and Figure 2 The Beidou short message communication method and device are described.

[0151] In addition, in combination with the Beidou short message communication method in the above embodiment, the embodiment of the present application can provide a computer storage medium for implementation. The computer storage medium stores computer program instructions; when the computer program instructions are executed by the processor, any one of the Beidou short message communication methods in the above embodiment is implemented.

[0152] It should be clear that the present application is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present application is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present application.

[0153] The functional blocks shown in the above-described block diagram can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present application are programs or code segments that are used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link by a data signal carried in a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (RF) links, etc. The code segment can be downloaded via a computer network such as the Internet, an intranet, etc.

[0154] It should also be noted that the exemplary embodiments mentioned in this application describe some methods or systems based on a series of steps or devices. However, this application is not limited to the order of the above steps, that is, the steps can be performed in the order mentioned in the embodiment, or in a different order from the embodiment, or several steps can be performed simultaneously.

[0155] Aspects of the present disclosure are described above with reference to the flowchart and / or block diagram of the method, device (system) and computer program product according to the embodiment of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device to produce a machine so that these instructions executed by the processor of the computer or other programmable data processing device enable the implementation of the function / action specified in one or more boxes of the flowchart and / or block diagram. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field programmable logic circuit. It can also be understood that each box in the block diagram and / or flowchart and the combination of boxes in the block diagram and / or flowchart can also be implemented by dedicated hardware that performs a specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions.

[0156] The above is only a specific implementation of the present application. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, modules and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. It should be understood that the protection scope of the present application is not limited to this. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in this application, and these modifications or replacements should be included in the protection scope of this application.

Claims

1. A Beidou short message communication method, characterized in that: Applied to any satellite in the Beidou short message communication system, each satellite in the Beidou short message communication system is loaded with the same multi-agent collaborative algorithm, multiple multi-agent collaborative algorithms are constructed based on the interactive collaboration information of multiple satellites, and the interactive collaboration information represents the communication information of multiple satellites in the Beidou short message communication system; The multiple satellites allocate resources of the multiple satellites through the loaded multi-agent collaborative algorithm to determine the satellite used to serve each user terminal served by the Beidou short message communication system at the current moment, wherein the resources include computing resources, storage resources and communication resources of the satellite; The Beidou short message communication method comprises the following steps: Acquire target message data sent by an initiator, where the initiator is the user end; The target message data is transmitted to the target terminal through the satellite for service corresponding to the user terminal.

2. The Beidou short message communication method according to claim 1, characterized in that: Before obtaining the target message data sent by the initiator, the following steps are also included: Determining the multi-agent collaboration algorithm according to the interactive collaboration information of the plurality of satellites; Iterate the multi-agent collaborative algorithm loaded on the plurality of satellites to obtain a resource adjustment action for each satellite; The satellite for serving each of the user terminals is determined according to the resource adjustment action corresponding to each of the satellites.

3. The Beidou short message communication method according to claim 2, characterized in that: Determining the multi-agent collaboration algorithm according to the interactive collaboration information of the plurality of satellites comprises the following steps: Acquire a random strategy function, wherein the random strategy function is used to obtain an action decision according to the computing resources, the storage resources, the communication resources of the satellite, the amount of short message data of multiple satellites, and the number of user terminals served by multiple satellites, and the action decision is used to determine the amount of the computing resources, the storage resources, and the communication resources allocated by the satellite to the user terminal; Acquire a reward function, wherein the reward function reflects satisfaction of the user terminal of the satellite service; Obtaining a transition probability, where the transition probability is the probability that the satellite will transition to a state at a next moment after executing the action decision; The multi-agent collaborative algorithm is determined based on the random strategy function, the reward function and the transition probability.

4. The Beidou short message communication method according to claim 3, characterized in that: The iterative operation of the multi-agent collaborative algorithm loaded on the plurality of satellites to obtain a resource adjustment action for each satellite includes: Acquire a first observation state, where the first observation state includes the computing resources, the storage resources, the communication resources of the satellite itself, the amount of short message data of multiple satellites, and the number of user terminals served by multiple satellites; Based on the random strategy function, obtaining the action decision; Execute the action decision and obtain a corresponding reward, wherein the reward is obtained according to the reward function; According to the transition probability, the state at the next moment is jumped, and the above process is repeated until the number of iterations reaches a preset iteration threshold or the multi-agent collaborative algorithm meets the preset convergence requirement, and the resource adjustment action of each satellite is obtained.

5. The Beidou short message communication method according to claim 4, characterized in that: After acquiring the first observation state, the method further includes: Broadcasting the first observation status over the entire network via an intersatellite link; Acquire a second observation state, where the second observation state includes the computing resources, the storage resources, the communication resources of the other satellites, the amount of short message data of the multiple satellites, and the number of user terminals served by the multiple satellites, and the second observation state is broadcasted by the other satellites over the entire network through intersatellite links; According to the first observation state and the second observation state, a global state is determined, and the global state includes the computing resources, the storage resources, the communication resources and the short message data volume of all the satellites, and the number of user terminals served by all the satellites in the Beidou short message communication system.

6. The Beidou short message communication method according to claim 4, characterized in that: In the action decision, when it is decided to allocate the computing resources and the storage resources to the user terminal and not allocate the communication resources, the user terminal is served by the ground station.

7. The Beidou short message communication method according to claim 4, characterized in that: The multi-agent collaborative algorithm satisfies the preset convergence requirement if the reward function satisfies the preset conditions.

8. A Beidou short message communication device, characterized in that: Applied to any satellite in the Beidou short message communication system, each satellite in the Beidou short message communication system is loaded with the same multi-agent collaborative algorithm, multiple multi-agent collaborative algorithms are constructed based on the interactive collaboration information of multiple satellites, and the interactive collaboration information represents the communication information of multiple satellites in the Beidou short message communication system; The multiple satellites allocate resources of the multiple satellites through the loaded multi-agent collaborative algorithm to determine the satellite used to serve each user terminal served by the Beidou short message communication system at the current moment, wherein the resources include computing resources, storage resources and communication resources of the satellite; The Beidou short message communication device comprises: A receiving unit, used for acquiring target message data sent by an initiator, wherein the initiator is the user end; The sending unit is used to transmit the target message data to the target terminal through the satellite used for service corresponding to the user terminal.

9. An electronic device, characterized in that: It includes a processor and a memory, the memory stores a program or instruction that can be run on the processor, and when the program or instruction is executed by the processor, the steps of the Beidou short message communication method as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute the Beidou short message communication method as described in any one of claims 1 to 7.

Citation Information

Cited By

  • Multi-satellite cooperative short message communication method, device and equipment and storage medium

    CN121194137A