Cooperative beam scheduling method and device based on satellite clustering, equipment and medium

Abstract

The invention provides a cooperative beam scheduling method and device based on satellite clustering, equipment and a medium. The method comprises the following steps: determining a corresponding service area for each satellite cluster according to a reference service volume corresponding to the average communication capability of all satellite clusters; for each satellite in each satellite cluster, inputting visible time period information of the satellite to a service cell, working state information of all satellites in the satellite cluster to which the satellite belongs and service volume information of all service cells in a service area corresponding to the satellite cluster to which the satellite belongs into a deep Q network DQN model, and obtaining a scheme for carrying out beam hopping scheduling on the service cell in which the satellite is visible through the output of the DQN model.

Description

Technical Field

[0001] The present disclosure relates to the field of satellite communication technology, and in particular to a satellite clustering-based coordinated beam scheduling method, device, equipment and medium. Background Art

[0002] Due to the uneven geographical distribution and time-varying characteristics of ground users, the service sizes of various cells under satellite coverage vary significantly. How to flexibly and effectively allocate satellite resources and match them with non-uniform services is a major problem in the field of satellite communications. As an emerging technology in satellite communications, beam-hopping technology makes full use of the spatiotemporal freedom of beams, providing great flexibility for satellite resource allocation. Beam-hopping technology divides the satellite coverage area into multiple cells, but only illuminates some of them in one time slot. The satellite illuminates different cells in different time slots, and schedules beams in a time-division multiplexing manner to dynamically allocate communication resources to different users, thereby improving the system's service capabilities and resource allocation efficiency.

[0003] However, the current beam-hopping scheduling schemes usually only consider the beam-hopping scheduling of a single satellite. For a large-scale low-orbit satellite constellation, since the coverage areas of each satellite are highly overlapped, if the adjacent satellites are independent, it will cause problems such as uneven load distribution and mutual interference between the beams of the satellites. Therefore, with the overall service performance of the cluster as the optimization goal, it is of great significance to study the intra-cluster coordinated beam-hopping technology for satellite clusters to meet the needs of unevenly distributed services and achieve efficient management of the large-scale constellation network. Summary of the invention

[0004] The present disclosure provides a satellite cluster-based coordinated beam scheduling method, device, equipment and medium; it can balance the services of each service area, and provide a coordinated beam hopping scheduling solution within the cluster with the overall service performance of the cluster as the optimization target.

[0005] The technical solution of the present disclosure is achieved as follows: In a first aspect, the present disclosure provides a coordinated beam scheduling method based on satellite clustering, the method comprising: Determine a corresponding service area for each satellite cluster according to a reference traffic volume corresponding to an average communication capability of all satellite clusters; wherein a difference between the traffic volume of each service area and the reference traffic volume is less than a set first threshold, and each service area includes one or more service cells; For each satellite in each satellite cluster, the visible time period information of the satellite to the service cell, the working status information of all satellites in the satellite cluster to which the satellite belongs, and the traffic volume information of all service cells in the service area corresponding to the satellite cluster to which the satellite belongs are input into the Deep Q-Network (DQN) model, so as to obtain the beam-hopping scheduling scheme for the satellite to be visible to the service cell through the output of the DQN model.

[0006] In a second aspect, the present disclosure provides a coordinated beam scheduling device based on satellite clustering, comprising: a determination part and a beam scheduling scheme output part; wherein, The determining part is configured to determine a corresponding service area for each satellite cluster according to a reference traffic volume corresponding to the average communication capacity of all satellite clusters; wherein the difference between the traffic volume of each service area and the reference traffic volume is less than a set threshold, and each service area includes one or more service cells; The beam scheduling scheme output part is configured to input, for each satellite in each satellite cluster, the visible time period information of the satellite for the service cell, the working status information of all satellites in the satellite cluster to which the satellite belongs, and the traffic volume information of all service cells in the service area corresponding to the satellite cluster to which the satellite belongs into the deep Q network DQN model, so as to obtain the beam hopping scheduling scheme for the satellite to be a visible service cell through the output of the DQN model.

[0007] In a third aspect, the present disclosure provides a computing device, comprising: a processor and a memory; the processor is used to execute instructions stored in the memory to implement the satellite clustering-based collaborative beam scheduling method as described in the first aspect.

[0008] In a fourth aspect, the present disclosure provides a computer-readable storage medium storing at least one instruction, wherein the at least one instruction is used to be executed by a processor to implement the satellite clustering-based coordinated beam scheduling method as described in the first aspect.

[0009] The present disclosure provides a coordinated beam scheduling method, device, equipment and medium based on satellite clustering; the corresponding service area is divided for each satellite cluster based on the balance of ground service request volume and satellite communication capacity, so that the service of each service area is balanced and adapted to the corresponding cluster communication capacity. In addition, the satellite visibility time period, the working status of all satellites in the cluster and the service volume information are used as the input of the DQN model, so that the output beam hopping scheduling scheme takes into account the satellite visibility, the service volume of the cluster and the on-board resources at the same time, and can obtain an optimized beam hopping scheduling scheme for the satellite cluster as a whole, thereby ensuring that all satellites in the same cluster can coordinate to complete the allocation of beams for the service cells in the service area. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 A schematic diagram of a satellite network architecture provided by the present invention.

[0011] Figure 2 A schematic flow chart of a coordinated beam scheduling method based on satellite clustering provided in the present disclosure.

[0012] Figure 3 A schematic diagram of a flow chart for determining a service area for a satellite cluster provided in the present disclosure.

[0013] Figure 4 A schematic diagram of the output beam hopping scheduling solution provided in the present invention.

[0014] Figure 5 Schematic diagram of satellite migration provided for the present disclosure.

[0015] Figure 6 A schematic diagram of the composition of a satellite clustering-based coordinated beam scheduling device provided in the present invention.

[0016] Figure 7 A schematic diagram of the composition of another satellite clustering-based collaborative beam scheduling device provided in the present invention.

[0017] Figure 8 A schematic diagram of the structure of a computing device provided by the present disclosure. DETAILED DESCRIPTION

[0018] The technical solutions in the present disclosure will be described clearly and completely below in conjunction with the accompanying drawings in the present disclosure.

[0019] See also Figure 1 , which shows a schematic diagram of a satellite network architecture that can be applied to the technical solution of the present disclosure. Taking the low earth orbit (LEO) satellite constellation as an example, although Figure 1 Not shown in FIG, the LEO satellite constellation can include multiple LEO star clusters. Figure 1In the present invention, only adjacent LEO cluster-1 and LEO cluster-2 in the LEO satellite constellation are taken as examples to facilitate the explanation of the technical solution of the present invention.

[0020] exist Figure 1 In the above, any one of LEO Cluster-1 and LEO Cluster-2 may include at least 4 LEO satellites. Figure 1 In the examples, four satellites are used. In addition, each LEO cluster includes a cluster head satellite, which can not only communicate with other satellites in the cluster as shown by the dotted line, but also communicate with the cluster head satellites of other adjacent clusters through the intersatellite links shown by the lightning line. Figure 1 In this case, the head satellite of LEO Cluster-1 can communicate bidirectionally with the head satellite of LEO Cluster-2.

[0021] exist Figure 1 In the example, satellite S1 in LEO Cluster-2 is able to provide beams for three circular areas on the ground to achieve satellite-to-ground data transmission, as shown by the dotted line. It should be noted that when S1 can only provide a single beam, the single beam can be used in a time division multiplexing manner to illuminate the three circular ground areas in different time slots in turn, thereby achieving beam hopping scheduling. Of course, although Figure 1 Not shown, when S1 is capable of providing multiple beams (for example, 3 beams), the multiple beams can also be irradiated to three groups of ground areas in sequence according to different time slots in a time division multiplexing manner, and each group of ground areas includes three circular ground areas.

[0022] In order to improve the overall service performance of the cluster, the present disclosure hopes to enable the intra-cluster coordinated beam hopping technology of the satellite cluster to meet more balanced service requirements. Figure 2 , which shows a collaborative beam scheduling method based on satellite clustering provided by the present disclosure, and the method includes step S201 and step S202.

[0023] In step S201, a corresponding service area is determined for each satellite cluster according to a reference traffic volume corresponding to the average communication capabilities of all satellite clusters.

[0024] In the present disclosure, through step S201, a corresponding satellite cluster can be determined for each service area on the surface of the earth, and each satellite cluster corresponds to a service area.

[0025] For any service area, the service requests issued in the area are used to characterize the maximum service volume of the service area. In order to balance the service volume between the service areas, the present disclosure uses the average communication capacity of all satellite clusters as an evaluation standard, not only to divide the service areas, but also to ensure that the service volume of each service area is close to the evaluation standard. If the service volume of all service areas is close to the evaluation standard, not only can the service volume between the service areas be balanced, but also the service volume of each service area can be adapted to the communication capacity of the corresponding satellite cluster.

[0026] In order to judge whether it is close to the evaluation standard, the present disclosure exemplarily judges by setting a first threshold. Specifically, a benchmark traffic volume is set according to the average communication capacity of all satellite clusters, and the difference between the traffic volume of the service area and the benchmark traffic volume is compared with the first threshold: when the difference between the traffic volume of each service area and the benchmark traffic volume is less than the first threshold, it means that each service area not only has a satellite cluster corresponding to it, but also the traffic volume of the service area matches the communication capacity of the corresponding satellite cluster. In some examples, each service area includes one or more service cells.

[0027] In step S202, for each satellite in each satellite cluster, the visible time period information of the satellite to the service cell, the working status information of all satellites in the satellite cluster to which the satellite belongs, and the traffic volume information of all service cells in the service area corresponding to the satellite cluster to which the satellite belongs are input into the DQN model, so as to obtain a beam hopping scheduling scheme for the satellite to be visible to the service cell through the output of the DQN model.

[0028] In the present disclosure, for each satellite cluster, each satellite in the cluster is modeled as an agent, and a DQN model is set in the agent. For the DQN model, action decisions are made through the input state, and the network parameters of the DQN model are updated based on the rewards obtained from the action decisions, so that the final DQN model can give the action decision with the highest reward.

[0029] In the present disclosure, exemplarily, for each satellite (or intelligent agent), the state of the input DQN model includes the visible time period information of the satellite to the service cell, the working state information of all satellites in the satellite cluster to which the satellite belongs, and the traffic volume information of all service cells in the service area corresponding to the satellite cluster to which the satellite belongs. Specifically, the working state information of all satellites in the satellite cluster to which the satellite belongs may include the remaining power of all satellites in the satellite cluster, the number of beams that can be provided, the bandwidth information of the beams, the service cells allocated in the service area, etc., and the traffic volume information of all service cells in the service area corresponding to the satellite cluster to which the satellite belongs may include the number of service requests issued by all service cells, etc.

[0030] For each satellite (or agent), based on the above input, the action decision output by the DQN model is the beam hopping scheduling scheme for the satellite for the visible service cells in the service area. The reward used to update the network parameters of the DQN model is a joint reward, which includes a reward for the ratio of the traffic volume of the service cells scheduled with beam illumination based on the action decisions of all satellites in the cluster where the satellite is located to the total traffic volume of all service cells in the service area, and a reward for the beam interference in the cluster where the satellite is located based on the action decisions of all satellites.

[0031] for Figure 2 The technical solution shown divides the corresponding service area for each satellite cluster based on the balance between the ground service request volume and the satellite communication capacity, so that the service of each service area is balanced and adapted to the corresponding cluster communication capacity. In addition, the satellite visibility time period, the working status of all satellites in the cluster and the service volume information are used as the input of the DQN model, so that the output beam hopping scheduling scheme takes into account the satellite visibility, the service volume of the cluster and the on-board resources at the same time, and can obtain an optimized beam hopping scheduling scheme for the satellite cluster as a whole, thereby ensuring that all satellites in the same cluster can collaboratively complete the allocation of beams to the service cells in the service area.

[0032] based on Figure 2 In some possible implementations of the technical solution shown, see Figure 3 The step of determining a corresponding service area for each satellite cluster according to a reference traffic volume corresponding to an average communication capability of all satellite clusters includes: S301: construct at least one satellite cluster in a satellite constellation; S302: Determine a reference traffic volume according to the average communication capacity of all satellite clusters; S303: Divide the earth's surface into a plurality of initial service areas of equal area according to the number of star clusters; S304: Compare the number of service requests in each initial service area with the benchmark service volume, and adjust the initial service area according to the comparison result so that the difference between the adjusted service area and the benchmark service volume is less than a set first threshold; S305: Determine a corresponding adjusted service area for each satellite cluster according to the visibility of the service cells in the service area.

[0033] In the above implementation, specifically, for a giant LEO satellite constellation, first, at least one satellite cluster is constructed according to the exemplary requirement that the number of satellites in each satellite cluster is not less than 4 satellites, and the number of satellite clusters is set to N.

[0034] Next, the communication capacity of each satellite cluster is determined by summing the communication capacities of the satellites included in each satellite cluster, and a reference traffic volume is determined based on the communication capacities of all satellite clusters. Figure 2 As described in the illustrated solution, the service area is determined based on the benchmark traffic volume as an evaluation standard, so that the traffic volume of the service area finally obtained is adapted to the communication capacity of the corresponding satellite cluster.

[0035] Then, the surface of the earth is divided into N initial service areas of the same area according to the number of satellite clusters N. It can be understood that there is a large difference in the number of business requests between the initial service areas obtained by area division. For example, under the same area, the number of business requests in cities is greater than the number of business requests in rural areas, mountains, forests, glaciers or deserts. The number of business requests can also represent the business volume. Therefore, the business volume is very unbalanced between the initial service areas. Therefore, it is necessary to merge or decompose the initial service areas based on the goal of balancing the business volume so that the business volume between the adjusted service areas is balanced.

[0036] Based on the above-mentioned business balance goal, after the initial service areas are divided, the business request volume of each initial service area is compared with the benchmark business volume, and the initial service area is adjusted according to the comparison result. For example, when the business request volume of an initial service area exceeds the benchmark business volume by a large amount, the initial service area is decomposed, for example, to obtain one or more service areas and other service areas whose difference between the business request volume and the benchmark business volume is less than the first threshold. When the business request volume of an initial service area is less than the benchmark business volume, the initial service area can be merged with other adjacent initial service areas so that the difference between the business volume of the merged service area and the benchmark business volume is less than the first threshold. Through the above-mentioned adjustment, the difference between each adjusted service area and the benchmark business volume can be made less than the first threshold, that is, the business volume between each adjusted service area is balanced.

[0037] Finally, according to the visibility of each satellite cluster to the service cells in the adjusted service area, a corresponding adjusted service area is allocated to each satellite cluster. Figure 1 Taking the architecture shown as an example, the traffic volume of the service area corresponding to LEO cluster-2 is balanced with the traffic volume of the service area corresponding to LEO cluster-1.

[0038] After balancing the traffic volume in the service area corresponding to each satellite cluster, it is also necessary to balance the traffic volume in the service cells in the service area. Based on this, in some examples, the method further includes: For all service cells in each adjusted service area, taking the traffic volume of the first service cell with the largest traffic volume as a benchmark, merging the other second service cells so that the difference between the traffic volume of the merged service cell and the traffic volume of the first service cell is less than a set second threshold; When the number of the second service cells to be merged meets a set third threshold, the second service cells that meet the third threshold are merged.

[0039] Specifically, combined Figure 1 The architecture shown in the figure, the service area corresponding to LEO Cluster-2 can be composed of Figure 1 The number of communication objects in the service cell represents the traffic volume of the cell. Figure 1 In the example, the communication objects may include users, terminals, cars, servers, etc. The service cell with the largest traffic volume includes three communication objects. The present disclosure takes this service cell as a benchmark and may merge other second service cells to obtain a merged service cell A1 and a merged service cell A2. Each merged service cell includes three communication objects, so that the traffic volume between the service cells shown in the circles is balanced.

[0040] Of course, since the width of the satellite beam is limited and cannot cover a large area, the present disclosure sets a third threshold value to limit the area of ​​the merged service cell, that is, when the number of the merged second service cells reaches the third threshold value, it means that the area of ​​these merged second service cells has reached the upper limit of the area that can be covered by the satellite beam. In this case, even if the traffic volume of these merged second service cells is not balanced with the traffic volume of the first service cell (that is, the difference is still greater than the second threshold value), only the second service cells whose number reaches the third threshold value are merged, and other second service cells are no longer merged.

[0041] based on Figure 2In the technical solution shown in the figure, in some possible implementations, for each satellite in each satellite cluster, the visible time period information of the satellite for the service cell, the working status information of all satellites in the satellite cluster to which the satellite belongs, and the traffic volume information of all service cells in the service area corresponding to the satellite cluster to which the satellite belongs are input into the deep Q network DQN model, so as to obtain the beam hopping scheduling scheme of the satellite as a visible service cell through the output of the DQN model, including: Taking each satellite in the satellite cluster as an agent with a DQN model; For each satellite, the visible time period information of the satellite to the service cell, the working status information of all satellites in the satellite cluster to which the satellite belongs, and the traffic volume information of all service cells in the service area corresponding to the satellite cluster to which the satellite belongs are input as the current state into the DQN model of each agent to decide the beam hopping scheduling strategy of each satellite in the current time slot; Obtaining a joint reward of a traffic volume proportion reward and a beam interference reward of the satellite cluster in the current time slot according to a beam hopping scheduling strategy of all satellites in the current time slot; Update the network parameters of the DQN model of each agent according to the joint reward until the joint reward is maximized; When the joint reward is maximum, a beam hopping scheduling scheme for each satellite in the current time slot that is visible to the serving cell is obtained according to the beam hopping scheduling strategy of each satellite in the current time slot determined by the DQN model of each agent.

[0042] For the above example, specifically, for any star cluster, each satellite in the star cluster is modeled as an agent with a DQN model, combined with Figure 4 , the multi-agents in the cluster can output beam-hopping scheduling schemes through online training.

[0043] Specifically, let the number of satellites in the cluster be N, and the state of each satellite (e.g. the i-th) in the DQN model input be ,in, t Indicates time slot, It is the visible time period information of the service cell in the service area corresponding to the i-th satellite pair cluster. It indicates the working status information of all satellites in the cluster, such as the remaining power of all satellites in the satellite cluster, the number of beams that can be provided, the bandwidth information of the beams, the service cells allocated in the service area, etc., which are transmitted to every other satellite in the same cluster through the cluster head satellite in the cluster. It is the traffic volume information of all service cells in the service area corresponding to the star cluster, that is, the traffic request volume information of all service cells. This information can also be transmitted to each other satellite in the same star cluster through the cluster head satellite in the star cluster.

[0044] The state of each satellite is input into the DQN model of the corresponding agent. Each DQN model will make action decisions based on the input state. In this disclosure, the action decided by the DQN model is the action of each satellite in the current time slot. t For example, the action output by the DQN model of each satellite (such as the i-th satellite) is ,in, Indicates the number of all service cells in the service area of ​​the cluster; represents the maximum number of beams that the i-th satellite can provide; Indicates The service cells are illuminated by the beam; Indicates The service cells are not illuminated by the beam.

[0045] It should be noted that the actions decided by all satellites (agents) are combined to form all satellites in the cluster in the current time slot. t The beam-hopping scheduling strategy set Based on the beam hopping scheduling strategy set, a joint reward of the traffic volume proportion reward and the beam interference reward of the satellite cluster in the current time slot can be obtained.

[0046] Specifically, the traffic volume ratio reward of the satellite cluster in the current time slot includes: the ratio of the traffic volume of the service cell where the beam is scheduled in the current time slot to the total traffic volume in the service area, that is, ,in, Indicates that in the current time slot t The traffic volume of the service cell where the beam is scheduled, Indicates that in the current time slot t The total traffic volume in the service area of ​​the satellite cluster in the current time slot. The beam interference reward of the satellite cluster in the current time slot includes: the sum of the number of adjacent beams of each satellite in the satellite cluster in the current time slot, that is, , which can measure the spatial interference between beams between satellites in a cluster, where represents the number of adjacent beams of the ith satellite, is the total number of satellites in the cluster. When the beam patterns of each satellite have no adjacent interference in space, For the above two rewards, the present disclosure expects to increase the traffic volume share reward and reduce the beam interference reward. Therefore, in some examples, the combined reward of the above two rewards can be shown as follows: .

[0047] After obtaining the joint reward, the network parameters of the DQN model of each agent can be updated until the joint reward is maximized. When the DQN models of all agents are updated to obtain the maximum joint reward, it means that the beam hopping scheduling strategy output by the model at this time is an optimized beam hopping scheduling scheme for the satellite cluster as a whole, and can ensure that all satellites in the same cluster can coordinate to complete the allocation of beams for the service cells in the service area. In this way, each satellite obtains a beam hopping scheduling scheme for the visible service cells in the current time slot.

[0048] It should be noted that, in the present disclosure, after obtaining the optimized beam hopping scheduling scheme of the current time slot (the action of the current time slot), the state can also be updated according to the scheme for use in the process of obtaining the beam hopping scheduling scheme of the subsequent time slot.

[0049] In some possible implementations, the method further includes: When the traffic volume of the service area corresponding to the first satellite cluster exceeds the communication capacity of the first satellite cluster, determining a transfer satellite in a second satellite cluster adjacent to the first satellite cluster; The transfer satellite is assigned to the first satellite cluster and the experience in the DQN model of all satellites in the first satellite cluster is transferred to the transfer satellite.

[0050] Regarding the above implementation, it should be noted that an experience pool is usually provided in the DQN model to save the states, actions, rewards and states updated based on actions of all satellites in the process of obtaining the beam hopping scheduling scheme in each time slot as experience.

[0051] In some examples, considering the time-varying nature of the services, if the traffic volume in an exemplary service area changes and exceeds the communication capacity of the corresponding satellite cluster, the cluster head satellite of the cluster can communicate with the cluster head satellite of the adjacent cluster to apply for the use of unoccupied satellites from the adjacent cluster. Figure 5 See Figure 5 In the figure above, assume that the traffic volume in the service area of ​​LEO Cluster-1 increases and exceeds the communication capacity of LEO Cluster-1. Then the cluster head satellite of LEO Cluster-1 can communicate through the intersatellite link to apply to LEO Cluster-2 for calling an unoccupied satellite. Assume that the satellite SA in LEO Cluster-2 is unoccupied, then it can Figure 5As shown in the figure below, the satellite SA is transferred and called to LEO cluster-1. However, since the environment of the transferred satellite SA before the call is different from the environment after the call, the intelligent agent of the satellite SA needs to be retrained. The present disclosure adopts a transfer learning method, using the experience pool of the DQN model to transfer the experience of the satellite originally in LEO cluster-1 to the intelligent agent of the satellite SA, so that the satellite SA can learn and adapt to the environment of LEO cluster-1 faster and make reasonable and accurate decisions.

[0052] Based on the same inventive concept as the above technical solution, see Figure 6 , which shows a satellite clustering-based coordinated beam scheduling device 60 provided by the present disclosure, the device 60 includes: a determination part 601 and a beam scheduling scheme output part 602; wherein, The determining part 601 is configured to determine a corresponding service area for each satellite cluster according to a reference traffic volume corresponding to an average communication capability of all satellite clusters; wherein the difference between the traffic volume of each service area and the reference traffic volume is less than a set threshold, and each service area includes one or more service cells; The beam scheduling scheme output part 602 is configured to input, for each satellite in each satellite cluster, the visible time period information of the satellite for the service cell, the working status information of all satellites in the satellite cluster to which the satellite belongs, and the traffic volume information of all service cells in the service area corresponding to the satellite cluster to which the satellite belongs into the deep Q network DQN model, so as to obtain the beam hopping scheduling scheme of the satellite for the visible service cell through the output of the DQN model.

[0053] The determining part 601 is configured to: constructing at least one satellite cluster in the satellite constellation; Determine the benchmark traffic volume based on the average communication capacity of all satellite clusters; Divide the earth's surface into multiple initial service areas of equal area according to the number of star clusters; Comparing the number of service requests of each initial service area with the benchmark service volume, and adjusting the initial service area according to the comparison result so that the difference between the adjusted service area and the benchmark service volume is less than a set first threshold; A corresponding adjusted service area is determined for each satellite cluster according to the visibility of the service cells in the service area.

[0054] The determining part 601 is further configured to: For all service cells within each adjusted service area, the traffic volume of the first service cell with the largest traffic volume is used as a benchmark, and other second service cells are merged so that the difference between the traffic volume of the merged service cell and the traffic volume of the first service cell is less than a set second threshold.

[0055] The beam scheduling solution output part 602 is configured to: Taking each satellite in the satellite cluster as an agent with a DQN model; For each satellite, the visible time period information of the satellite to the service cell, the working status information of all satellites in the satellite cluster to which the satellite belongs, and the traffic volume information of all service cells in the service area corresponding to the satellite cluster to which the satellite belongs are input as the current state into the DQN model of each agent to decide the beam hopping scheduling strategy of each satellite in the current time slot; Obtaining a joint reward of a traffic volume proportion reward and a beam interference reward of the satellite cluster in the current time slot according to a beam hopping scheduling strategy of all satellites in the current time slot; Update the network parameters of the DQN model of each agent according to the joint reward until the joint reward is maximized; When the joint reward is maximum, a beam hopping scheduling scheme for each satellite in the current time slot that is visible to the serving cell is obtained according to the beam hopping scheduling strategy of each satellite in the current time slot determined by the DQN model of each agent.

[0056] In some examples, the reward for the proportion of traffic of the satellite cluster in the current time slot includes: the ratio of the traffic volume of the service cell in which the beam is scheduled in the current time slot to the total traffic volume in the service area; The beam interference reward of the satellite cluster in the current time slot includes: the sum of the number of adjacent beams of each satellite of the satellite cluster in the current time slot.

[0057] For some examples, see Figure 7 , the device 60 further comprises a migration part 603, configured to: When the traffic volume of the service area corresponding to the first satellite cluster exceeds the communication capacity of the first satellite cluster, determining a transfer satellite in a second satellite cluster adjacent to the first satellite cluster; The transfer satellite is assigned to the first satellite cluster and the experience in the DQN model of all satellites in the first satellite cluster is transferred to the transfer satellite.

[0058] Please refer to Figure 8, which shows a structural block diagram of a computing device provided by an exemplary embodiment of the present disclosure. In some examples, the computing device 80 can be at least one of a smart phone, a smart watch, a desktop computer, a laptop, a virtual reality terminal, an augmented reality terminal, a wireless terminal, and a laptop portable computer. The computing device 80 has a communication function and can access a wired network or a wireless network. The computing device 80 can generally refer to one of a plurality of terminals, and those skilled in the art can know that the number of the above terminals can be more or less. In some examples, the computing device 80 can receive data based on the wired network or wireless network to which it is connected. It can be understood that the computing device 80 undertakes the calculation and processing work of the technical solution of the present disclosure, and the present disclosure does not limit this.

[0059] like Figure 8 As shown, the computing device in the present disclosure may include one or more of the following components: a processor 810 and a memory 820 .

[0060] Optionally, the processor 810 uses various interfaces and lines to connect various parts of the entire computing device, and executes various functions of the computing device and processes data by running or executing instructions, programs, code sets or instruction sets stored in the memory 820, and calling data stored in the memory 820. Optionally, the processor 810 can be implemented in at least one hardware form of digital signal processing (DSP), field-programmable gate array (FPGA), and programmable logic array (PLA). The processor 810 can integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), a neural network processor (NPU), and a baseband chip. Among them, the CPU mainly processes the operating system, user interface, and application programs; the GPU is responsible for rendering and drawing the content that needs to be displayed on the touch display; the NPU is used to implement artificial intelligence (AI) functions; and the baseband chip is used to process wireless communications. It is understandable that the above baseband chip may not be integrated into the processor 810, but may be implemented by a separate chip.

[0061] The memory 820 may include a random access memory (RAM) or a read-only memory (ROM). Optionally, the memory 820 includes a non-transitory computer-readable storage medium. The memory 820 may be used to store instructions, programs, codes, code sets, or instruction sets. The memory 820 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above various method embodiments, etc.; the data storage area may store data created according to the use of the computing device, etc.

[0062] In addition, those skilled in the art can understand that the structure of the computing device shown in the above drawings does not constitute a limitation on the computing device, and the computing device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently. For example, the computing device also includes a display screen, a camera assembly, a microphone, a speaker, a radio frequency circuit, an input unit, a sensor (such as an acceleration sensor, an angular velocity sensor, a light sensor, etc.), an audio circuit, a WiFi module, a power supply, a Bluetooth module and other components, which will not be described in detail here.

[0063] The present disclosure also provides a computer-readable storage medium storing at least one instruction, wherein the at least one instruction is used to be executed by a processor to implement the satellite clustering-based coordinated beam scheduling method as described in the above embodiments.

[0064] The present disclosure also provides a computer program product, which includes computer instructions, which are stored in a computer-readable storage medium; a processor of a computing device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computing device executes to implement the satellite clustering-based coordinated beam scheduling method described in the above-mentioned embodiments.

[0065] Those skilled in the art should be aware that in one or more of the above examples, the functions described in the present disclosure can be implemented using hardware, software, firmware, or any combination thereof. When implemented using software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or codes on a computer-readable medium. Computer-readable media include computer storage media and communication media, wherein communication media include any media that facilitates the transmission of computer programs from one place to another. Storage media can be any available media that can be accessed by a general or special-purpose computer.

[0066] It should be noted that the technical solutions described in the present disclosure can be combined arbitrarily without conflict.

[0067] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A coordinated beam scheduling method based on satellite clustering, characterized in that: The method comprises: Determine a corresponding service area for each satellite cluster according to a reference traffic volume corresponding to an average communication capability of all satellite clusters; wherein a difference between the traffic volume of each service area and the reference traffic volume is less than a set first threshold, and each service area includes one or more service cells; For each satellite in each satellite cluster, the visible time period information of the satellite to the service cell, the working status information of all satellites in the satellite cluster to which the satellite belongs, and the traffic volume information of all service cells in the service area corresponding to the satellite cluster to which the satellite belongs are input into the deep Q network DQN model, so as to obtain the beam hopping scheduling scheme for the satellite as a visible service cell through the output of the DQN model.

2. The method according to claim 1, characterized in that The determining of a corresponding service area for each satellite cluster according to a reference traffic volume corresponding to an average communication capability of all satellite clusters comprises: constructing at least one satellite cluster in the satellite constellation; Determine the benchmark traffic volume based on the average communication capacity of all satellite clusters; Divide the earth's surface into multiple initial service areas of equal area according to the number of star clusters; Comparing the number of service requests of each initial service area with the benchmark service volume, and adjusting the initial service area according to the comparison result so that the difference between the adjusted service area and the benchmark service volume is less than a set first threshold; A corresponding adjusted service area is determined for each satellite cluster according to the visibility of the service cells in the service area.

3. The method according to claim 2, characterized in that The method further comprises: For all service cells in each adjusted service area, based on the traffic volume of the first service cell with the largest traffic volume, other second service cells are merged so that the difference between the traffic volume of the merged service cell and the traffic volume of the first service cell is less than a set second threshold; When the number of the second service cells to be merged meets a set third threshold, the second service cells that meet the third threshold are merged.

4. The method according to claim 1, characterized in that: The method includes inputting, for each satellite in each satellite cluster, information about the visible time period of the satellite for the service cell, working status information of all satellites in the satellite cluster to which the satellite belongs, and traffic information of all service cells in the service area corresponding to the satellite cluster to which the satellite belongs into a deep Q network DQN model, so as to obtain a beam hopping scheduling scheme for the satellite as a visible service cell through the output of the DQN model, including: Taking each satellite in the satellite cluster as an agent with a DQN model; For each satellite, the visible time period information of the satellite to the service cell, the working status information of all satellites in the satellite cluster to which the satellite belongs, and the traffic volume information of all service cells in the service area corresponding to the satellite cluster to which the satellite belongs are input as the current state into the DQN model of each agent to decide the beam hopping scheduling strategy of each satellite in the current time slot; Obtaining a joint reward of a traffic volume proportion reward and a beam interference reward of the satellite cluster in the current time slot according to a beam hopping scheduling strategy of all satellites in the current time slot; Update the network parameters of the DQN model of each agent according to the joint reward until the joint reward is maximized; When the joint reward is maximum, a beam hopping scheduling scheme for each satellite in the current time slot that is visible to the serving cell is obtained according to the beam hopping scheduling strategy of each satellite in the current time slot determined by the DQN model of each agent.

5. The method according to claim 4, characterized in that The satellite cluster's traffic volume ratio reward in the current time slot includes: the ratio of the traffic volume of the service cell where the beam is scheduled in the current time slot to the total traffic volume in the service area; The beam interference reward of the satellite cluster in the current time slot includes: the sum of the number of adjacent beams of each satellite of the satellite cluster in the current time slot.

6. The method according to claim 4, characterized in that The method further comprises: Each satellite other than the cluster head satellite in each satellite cluster receives the working status information of all satellites in the satellite cluster and the traffic volume information of all service cells in the service area corresponding to the satellite cluster, which are sent by the cluster head satellite.

7. The method according to claim 1, characterized in that The method further comprises: When the traffic volume of the service area corresponding to the first satellite cluster exceeds the communication capacity of the first satellite cluster, determining a transfer satellite in a second satellite cluster adjacent to the first satellite cluster; The transfer satellite is assigned to the first satellite cluster and the experience in the DQN model of all satellites in the first satellite cluster is transferred to the transfer satellite.

8. A coordinated beam scheduling device based on satellite clustering, characterized in that: The device comprises: a determination part and a beam scheduling scheme output part; wherein, The determining part is configured to determine a corresponding service area for each satellite cluster according to a reference traffic volume corresponding to an average communication capability of all satellite clusters; wherein a difference between the traffic volume of each service area and the reference traffic volume is less than a set threshold, and each service area includes one or more service cells; The beam scheduling scheme output part is configured to input, for each satellite in each satellite cluster, the visible time period information of the satellite for the service cell, the working status information of all satellites in the satellite cluster to which the satellite belongs, and the traffic volume information of all service cells in the service area corresponding to the satellite cluster to which the satellite belongs into the deep Q network DQN model, so as to obtain the beam hopping scheduling scheme of the satellite for the visible service cell through the output of the DQN model.

9. A computing device, characterized in that The computing device comprises: a processor and a memory; the processor is used to execute instructions stored in the memory to implement the satellite clustering-based coordinated beam scheduling method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores at least one instruction, and the at least one instruction is used to be executed by a processor to implement the satellite clustering-based coordinated beam scheduling method according to any one of claims 1 to 7.

Images