A base station common resource allocation method and device, a storage medium and an electronic device

Abstract

The specification discloses a base station resource allocation method and device, a storage medium and an electronic device. When the base station resource allocation method provided by the specification is used to allocate resources to the base station in a satellite-ground cooperative system, a resource allocation network capable of outputting a resource allocation strategy of the base station according to an observation state of the base station can be arranged in each base station, and the resource allocation network is pre-trained by a satellite in a centralized manner through reinforcement learning, and then in the application stage, the base station can not rely on the satellite, but only rely on its own observation state to allocate resources through the resource allocation network. Using the method can effectively reduce the frequent two-way communication between the satellite and the base station in the edge computing application of the satellite-ground cooperative system, reduce the delay caused by communication, speed up the processing efficiency of the computing task, and greatly improve the user experience.

Description

TECHNICAL FIELD

[0001] The present specification relates to the technical field of computer technology, and in particular to a base station computing resource allocation method and device, a storage medium and an electronic device. BACKGROUND

[0002] Nowadays, the ground network is increasingly difficult to meet the seamless global coverage demand, and the satellite-ground collaborative communication system integrating satellite and ground network will be an important development direction of future mobile communication. Limited by the communication rate of satellite-ground link, the user service delay based on traditional cloud computing is large. With the miniaturization and computing power improvement of computing devices, the processing capacity of edge devices and terminals under the satellite-ground network system is greatly enhanced. By introducing edge servers, computing is performed at the local edge layer to avoid uploading all computing to the cloud, which can effectively avoid the high network response delay of traditional cloud computing.

[0003] Limited by factors such as power and volume, the computing capacity of edge servers is usually limited, so targeted edge computing resource allocation is needed to minimize the task response delay of the satellite-ground collaborative communication system. At present, the existing base station computing resource allocation strategy usually considers centralized decision-making. In order to obtain global task information, all users need to complete communication with the satellite, thereby generating a two-way propagation delay of user-base station-satellite-base station-user. Taking a satellite with an orbital height of 8000km as an example, even if the user-base station propagation delay is ignored, the two-way propagation delay of a single user is as high as about 53 milliseconds, which greatly affects the user experience.

[0004] Therefore, how to improve the resource allocation strategy of the satellite-ground collaborative communication system to reduce the propagation delay is a problem to be solved. SUMMARY

[0005] The present specification provides a base station computing resource allocation method and device, a storage medium and an electronic device to at least partially solve the above problems existing in the prior art.

[0006] The present specification adopts the following technical solutions:

[0007] The present specification provides a base station computing resource allocation method, which is applied to a satellite-ground collaborative system including at least a satellite and a base station, and the method comprises:

[0008] Initializing the satellite-ground collaborative system;

[0009] Training the computing resource allocation network of the base station using the satellite;

[0010] For each base station, in response to the base station receiving a computing task, using the computing resource allocation network of the base station, determining the computing resource allocation strategy of the base station according to the observation state of the base station;

[0011] The base station executes the computing task received by the base station according to the computing and communication resource allocation strategy.

[0012] Optionally, the satellite-ground cooperative system is initialized, specifically including:

[0013] The system parameters of the satellite-ground cooperative system and the observation state of each base station are initialized.

[0014] Optionally, the system parameters include at least one of a communication parameter, a computing parameter, a task parameter, and a reinforcement learning parameter.

[0015] Optionally, the communication parameter at least includes the number of base stations, the number of users covered by each base station, and the orbit height of the satellite; the computing parameter at least includes the computing load of the edge computing server, the maximum available computing resource of the base station, and the maximum available computing resource of the satellite; the task parameter at least includes the size of the computing amount of the task; and the reinforcement learning parameter at least includes the maximum epoch length, the discount factor, the experience replay pool size, the batch size, the learning rate of the computing and communication resource allocation network of the base station, the learning rate of the evaluation network, the size of the hidden layer, the update coefficient of the evaluation network, the maximum time slot length, and the maximum allowed number of executions.

[0016] Optionally, the observation state of the base station at least includes the current computing amount of the task, the computing amount of the task in the last round, the satellite computing capacity allocation result in the last round, and the computing and communication resource allocation result in the last round.

[0017] Optionally, the base station executes the computing task received by the base station according to the computing and communication resource allocation strategy, specifically including:

[0018] The base station determines the allocation proportion of the computing task of the user covered by the base station according to the computing and communication resource allocation strategy;

[0019] According to the allocation proportion of the computing task, a task edge computing execution strategy of the user covered by the base station is determined;

[0020] The computing task is executed according to the task edge computing execution strategy.

[0021] Optionally, the satellite trains the computing and communication resource allocation network of the base station, specifically including:

[0022] The current observation state of each base station is determined.

[0023] For each base station, the current observation state is input into the computing and communication resource allocation network to be trained, so as to obtain the current computing and communication resource allocation strategy of the base station output by the computing and communication resource allocation network.

[0024] According to the current computing and communication resource allocation strategy, the allocation proportion of the current computing task of the base station is determined.

[0025] determining a next observation state of the base station, a time consumption of performing the allocated computing task and a communication delay according to a current observation state of the base station, a current computing resource allocation strategy of the base station, and a current global state of the satellite;

[0026] the satellite obtaining the current observation state of each base station as a current global state, obtaining the current computing resource allocation strategy of each base station as a global allocation strategy, obtaining the next observation state of each base station as a next global state, and returning the next observation state of each base station to the corresponding base station respectively;

[0027] for each base station, the satellite taking the current global state and the global allocation strategy, the next observation state and a total time consumption of performing all computing tasks as inputs, updating a parameter of an evaluation network of the base station; determining an allocation strategy loss value of the base station through the evaluation network, and updating the computing resource allocation network according to the allocation strategy loss value.

[0028] The present specification provides a base station computing resource allocation device, the device is applied to a satellite-ground cooperative system, the satellite-ground cooperative system at least includes a satellite and a base station, the device comprises:

[0029] an initialization module configured to initialize the satellite-ground cooperative system;

[0030] a training module configured to train a computing resource allocation network of the base station by the satellite;

[0031] an allocation module configured to, for each base station, in response to the base station receiving a computing task, determine a computing resource allocation strategy of the base station according to an observation state of the base station by the computing resource allocation network of the base station;

[0032] an execution module configured to perform the computing task received by the base station according to the computing resource allocation strategy by the base station.

[0033] The present specification provides a computer readable storage medium, the storage medium stores a computer program, the computer program is executed by a processor to implement the above-mentioned base station computing resource allocation method.

[0034] The present specification provides an electronic device, comprising a memory, a processor and a computer program stored in the memory and executable on the processor, the processor executes the program to implement the above-mentioned base station computing resource allocation method.

[0035] The above-mentioned at least one technical scheme adopted by the present specification can achieve the following beneficial effects:

[0036] In the base station computing resource allocation method provided in the specification, the satellite-ground cooperative system is initialized; the computing resource allocation network of the base station is trained by the satellite; for each base station, in response to the base station receiving a computing task, the computing resource allocation network of the base station is used to determine the computing resource allocation strategy of the base station according to the observation state of the base station; and the base station executes the computing task received by the base station according to the computing resource allocation strategy.

[0037] When the base station computing resource allocation method provided in the specification is used to allocate computing resources to the base stations in the satellite-ground cooperative system, a computing resource allocation network capable of outputting the computing resource allocation strategy of the base station according to the observation state of the base station can be arranged at each base station, and the centralized training by the satellite through reinforcement learning is performed in advance, and then in the application stage, the base station can not rely on the satellite, but only rely on the observation state of the base station to realize the allocation of computing resources through the computing resource allocation network. Using this method can effectively reduce the frequent two-way communication between the satellite and the base station in the application, reduce the delay caused by communication, speed up the processing efficiency of the computing task, and greatly improve the user experience. BRIEF DESCRIPTION OF DRAWINGS

[0038] The drawings described herein are used to provide further understanding of the specification, and form a part of the specification. The illustrative embodiments of the specification and their descriptions are used to explain the specification, and do not constitute an improper limitation on the specification. In the drawings:

[0039] Figure 1 It is a flowchart of a base station computing resource allocation method provided in the specification;

[0040] Figure 2 It is a system architecture diagram of a feasible satellite-ground cooperative system provided in the specification;

[0041] Figure 3 It is a step diagram of training the computing resource allocation network and the evaluation network provided in the specification;

[0042] Figure 4 It is a step diagram of allocating computing resources in a satellite-ground cooperative system provided in the specification;

[0043] Figure 5 It is an effect diagram of the base station computing resource allocation method provided in the specification in actual test;

[0044] Figure 6 It is a diagram of a base station computing resource allocation device provided in the specification;

[0045] Figure 7 It is a diagram of a base station computing resource allocation device provided in the specification;Figure 1 An electronic device schematic diagram. DETAILED DESCRIPTION

[0046] For the purposes of the present description, the technical solutions and advantages, the technical solutions of the present description will be described in detail below with reference to specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of the present description, not all. Based on the embodiments in the present description, all other embodiments obtained by those skilled in the art without creative labor are within the scope of the present application.

[0047] The technical solutions provided by the embodiments of the present description will be described in detail below with reference to the drawings.

[0048] Figure 1 The flowchart of the method for allocating computing resources of a base station in the present description is shown in the figure, which specifically includes the following steps:

[0049] S100: Initialize the satellite-ground cooperative system.

[0050] The base station computing resource allocation provided by the present description is applied to a satellite-ground cooperative system including at least several satellites and several base stations, and each base station can cover several users. Figure 2 The system architecture of a feasible satellite-ground cooperative system provided by the present description is shown in the figure. In a satellite-ground cooperative system, there can be several satellites running in space and several base stations running on the ground. Generally, a satellite is responsible for realizing the docking communication between multiple base stations. The correspondence between the satellite and the base station is generally divided according to the area, that is, each satellite corresponds to and communicates with the base stations in its signal coverage range. Each base station serves several users in the base station coverage range, and each user establishes a connection with the base station according to its own needs.

[0051] Figure 2 Taking a satellite and several base stations as an example, the relationship between the satellite and the base station is shown. As shown in Figure 2 , there is an edge server in each satellite or base station for realizing computing function. For any base station, when the base station receives a computing task sent by a user, in order to complete the processing of the computing task more quickly, the base station can transfer part of the computing task to the satellite for processing. Corresponding to Figure 2 , it is the computing migration from the user to the base station, and the computing migration from the base station to the satellite. The computing task processed by the base station can directly return the computing result to the user by the base station itself, and the computing task processed by the satellite can return the computing result to the ground station and then return it to the user. Among them, the ground station in the satellite-ground cooperative system is not the focus of the present method, and the present description will not be discussed further.

[0052] All steps in the method can be implemented by any electronic device with computing function, such as terminals, servers and the like, and the description takes the edge server on the satellite and the base station in the star-ground cooperative system as the main body to illustrate the implementation of the steps in the method. It is worth mentioning that in the description, a group of corresponding satellites and base stations, that is, a satellite and the base stations covered by the satellite, are mainly illustrated. For each group of satellites and base stations in the star-ground cooperative system, the allocation of computing resources can be implemented according to the base station computing resource allocation method provided in the description.

[0053] The method is applied when any base station in the star-ground cooperative system receives a computing task that needs to be processed, and a reasonable computing resource allocation strategy is given to the base station without relying on the satellite, so as to ensure the efficient operation of the star-ground cooperative system as a whole and quickly process the computing task by the satellite and the base station. The computing resource allocation strategy includes base station computing capacity allocation, proportion of allocated computing of the base station and star-ground communication resource allocation. The base station computing capacity allocation represents the computing capacity of the computing task allocated by the base station to each user covered by the base station in the batch of computing tasks. The proportion of allocated computing of the base station represents the proportion of the computing task allocated to the base station itself in the total user computing task in the batch of computing tasks. Based on this, the star-ground cooperative system can be initialized in this step. That is, before the star-ground cooperative system is formally put into application, the settings of the star-ground cooperative system are initialized.

[0054] Specifically, the system parameters of the star-ground cooperative system and the observation state of each base station can be initialized. In the method, the parameters of the star-ground cooperative system can include but are not limited to communication parameters, computing parameters, task parameters and reinforcement learning parameters. The communication parameters can at least include the number of base stations, the number of covered users of each base station, the satellite orbit height, the base station satellite uplink bandwidth and the base station satellite uplink transmission power. The computing parameters can at least include the edge computing server computing load, the maximum available computing resource of the base station and the maximum available computing resource of the satellite. The task parameters can at least include the task computing amount size. The reinforcement learning parameters can at least include the maximum epoch length, the discount factor, the experience replay pool size, the batch size, the base station computing resource allocation network learning rate, the evaluation network learning rate, the hidden layer size, the evaluation network update coefficient, the maximum time slot length and the maximum allowed execution number.

[0055] Each base station has a separate observation state, and the observation state of a base station can at least include the current covered task computing amount size, the last covered task computing amount size, the last satellite computing capacity allocation result and the last computing resource allocation strategy.

[0056] In initializing the parameters, for the parameter items with clear data such as communication parameters, calculation parameters, task parameters, etc., the determined data can be directly filled in as parameter values; and for the parameters such as reinforcement learning parameters which need to be determined through subsequent training and learning of the neural network model, and the observation state of the base station, a random initialization mode can be used for setting.

[0057] In a specific embodiment, in initializing the parameters, the communication parameters can include the number of base stations covered by the satellite N, the maximum number of users covered by each base station K n , the satellite orbit height H, the base station-satellite uplink bandwidth B b , the uplink transmission power between the nth base station and the satellite The calculation parameters include the edge computing server per-bit calculation load of the kth user of the nth base station C n,k , the maximum available calculation resource capacity of the base station F b , the maximum available calculation resource of the satellite F s ; the task parameters are mainly the user task calculation amount, i.e., the calculation task amount of the kth user of the nth base station at time t The reinforcement learning parameters include the maximum epoch length M, the discount factor γ, the experience replay pool size , the batch size S, the base station calculation resource allocation strategy network learning rate β A , the reinforcement learning critic network learning rate β C , the hidden layer size d, the critic network soft update coefficient τ, the maximum time slot length T, and the maximum allowed number of executions T m .

[0058] S102: Train the calculation resource allocation network of the base station by the satellite.

[0059] After the initialization of the satellite-ground cooperative system is completed in step S100, the satellite can train the computational resource allocation network contained in the base stations covered by the satellite in this step. In the method, each base station contains a computational resource allocation network for giving the computational resource allocation strategy of the base station when the base station receives a user computing task. When the computational resource allocation network is trained in this step, the current observation state of each base station can be determined, the current observation state is input into the computational resource allocation network to be trained to obtain the current computational resource allocation strategy of the base station output by the computational resource allocation network, the allocation proportion of the current computing task of the base station is determined according to the current computational resource allocation strategy, the current computational resource allocation strategy of the base station is used for allocation, the base station executes the computing task allocated to itself, and the computing task allocated to the satellite, the current observation state and the current computational resource allocation strategy are transmitted to the satellite, the next observation state of the base station, the time consumption and communication delay of executing the allocated computing task are determined, the current observation state of each base station is obtained as a current global state, the current computational resource allocation strategy of each base station is obtained as a global allocation strategy, the next observation state of each base station is obtained as a next global state, and the next observation state of each base station is returned to the corresponding base station, for each base station, the satellite updates the parameters of the evaluation network of the base station by taking the current global state and the global allocation strategy, the next observation state and the total time consumption of executing all computing tasks as inputs, determines the allocation strategy loss value of the base station through the evaluation network, and updates the computational resource allocation network according to the allocation strategy loss value.

[0060] In the above method, the role of the computational resource allocation network is to determine the computational resource allocation strategy of the base station, and the role of the evaluation network is to evaluate the computational resource allocation strategy output by the computational resource allocation network and feed back a reward value. It should be noted that the role of the reward value is to adjust the parameters of the computational resource allocation network, that is, in the method, only the evaluation network exists in the training stage, and after the actual application, only the computational resource allocation network is needed, and the evaluation network no longer exists. The input of the computational resource allocation network is the current observation state of the base station, and the output is the computational resource allocation strategy; the input of the evaluation network is the current global state, and the output is the reinforcement learning action reward value; the current global state is composed of the current observation states of all base stations.

[0061] In a specific embodiment, the current observation state of the base station includes the current coverage task computing amount size of all users covered by the base station the last round coverage task computing amount size D t-1 the last round satellite computing capacity allocation result F s,t-1 and the last round computational resource allocation strategy compositions of the current task computation size compositions of the current task computation size, the last round satellite computation capacity allocation result is the computation capacity result of all users covered by all base stations in the satellite coverage range when submitting the computation task in the last round, the satellite processes, the last round task computation size is composed of the last round total user task computation size, The last round computation resource allocation strategy includes the last round base station computation capacity allocation result of all users covered by the current base station, the satellite-ground communication resource allocation result and the allocated proportion of the base station computation task, More preferably, the allocated proportion of the base station computation task can be calculated by the following formula Where c is the speed of light, is the satellite-ground uplink transmission rate, and N0 is the noise size, is the satellite-ground link channel fading, both of which can be obtained according to traditional communication technology, which is not the focus of the present method, and the present specification will not be discussed further. More preferably, the current observation state of the base station In the present embodiment, it can be normalized, that is, (where D max is the maximum possible value of the computation task size).

[0062] The current global state is composed of the current observation state of all base stations, that is, The global allocation strategy is composed of the computation resource allocation strategies of all base stations, that is, Further, when initializing the current observation state and the current global state of all base stations, that is, t = 1, since the last time (t = 0) historical information is temporarily unavailable (superscript t-1), D t-1 Random generation can be used for initialization, and F s,t-1 The average allocation method is used for initialization. The computation resource allocation strategy network of any base station The input is the current observation state of the base station The network parameter is θ n , and the output is the computation resource allocation strategy of the base station The target computation resource allocation strategy network is In the present embodiment, the direct output value of the computation resource allocation network of the base station is processed as follows to ensure The element value range in is [-1, 1], that is, Again use There can be 2 in each base station, that is, The target evaluation network The input is the current global state and the current resource allocation strategy of the base station, and the output is the reward value of the reinforcement learning action. θ n , The parameters can be randomly initialized, and the initialized θ n , is assigned to to complete the parameter initialization of the evaluation network.

[0063] Further, when training the resource allocation network, there can be multiple different training methods. The present specification provides a specific embodiment for reference. Figure 3 A step diagram for training the resource allocation network and the evaluation network is provided in the present specification. Specifically, the steps given can be used for training: Figure 3

[0064] S1020: Obtain the current global state of the space-ground cooperative system Set the time slot number t = 1;

[0065] S1022: For each base station (n takes 1, 2, …, N in turn), use the base station's resource allocation network and add noise, i.e., according to give the current resource allocation strategy of the base station

[0066] S1024: According to the current resource allocation strategy , allocate resources, and obtain the resource allocation strategy a t of each base station, and based on this, calculate the service response delay r t of the space-ground cooperative system when processing user computing tasks, and the next global state s t+1 of the system at the next time, put the (s t , a t , r t , s t+1 ) four-tuple into the experience replay pool, and set the base station index value n to 1;

[0067] S1026: For the base station n corresponding to the current base station index value, randomly take a batch size of (s t , a t , r t , s t+1 ) four-tuples from the experience replay pool to form a training batch {(s i , a i , r i , s i+1} i＝1,…,S

[0068] ​S1028: Calculate the evaluation network update intermediate value of the current base station n and the network loss value and update the evaluation network of the current base station accordingly

[0069] S10210: Calculate the sampled policy gradient, and calculate the computational resource allocation network loss value of the current base station accordingly Then update the parameters of the computational resource allocation network of the current base station accordingly;

[0070] S10212: Update the target evaluation network and the target computational resource allocation network of the current base station in a soft update manner At the same time, the base station index value is incremented by one (n←n+1);

[0071] S10214: Determine whether the base station index value exceeds the number of base stations in the current region (whether n is greater than N), and if so, the iteration ends and enters sub-step S10216, otherwise the iteration continues and returns to sub-step S1026;

[0072] S10216: Update the average system response delay at the current epoch Then replace the current global state (s t ←s t+1 ) with the next global state of the system at the next time, and the time slot number is incremented by one (t←t+1);

[0073] S10218: Determine whether the time slot number exceeds the maximum time slot length (whether t is greater than T), and if so, the iteration ends and enters sub-step S10220, otherwise the iteration continues and returns to sub-step S1022, wherein the maximum time slot length can be pre-set according to specific requirements;

[0074] S10220: If the epoch number is 1, set the optimal average system response delay to the average system response delay at the current epoch, i.e. Save the parameters θ n of the computational resource allocation network of all base stations; if the epoch number is greater than 1, determine whether the average system response delay at the current epoch is less than the optimal average system response delay, and if so, replace the optimal average system response delay with the average system response delay at the current epoch, i.e. and save the parameters θ n of the computational resource allocation network of all base stations; if not, do nothing; after the above operations, the epoch number is incremented by one (e←e+1);

[0075] S10222: Determine whether the maximum epoch length is exceeded (e is greater than M), if exceeded, the iteration is ended, the training is ended, and the training completion flags of all base stations are set to 1, indicating that the training is completed; if not exceeded, the iteration is continued, and the sub-step S1020 is returned, wherein the epoch length is used to represent the training round, and the maximum epoch length can be pre-set according to specific requirements.

[0076] S104: For each base station, in response to the base station receiving a computing task, using the general computing resource allocation network of the base station, determining the general computing resource allocation strategy of the base station according to the observation state of the base station.

[0077] After the training of all base stations is completed in step S102, the satellite-ground cooperative system can be put into application in this step. At this time, users can submit computing tasks to the base station to which the coverage range where the user is located belongs. Each base station processes all computing tasks submitted by users in a fixed period. At the time corresponding to the period, the base station can input the observation state of the base station at the time into the computing resource allocation network to determine the general computing resource allocation strategy, that is, to determine the task capacity of the computing task processed by the base station itself, and further to determine the task capacity of the computing task uploaded to the satellite and processed by the satellite.

[0078] Figure 4 A step diagram for allocating computing resources in a satellite-ground cooperative system is provided for the present specification. Specifically, the computing resource allocation of the base station can be performed according to the steps shown in Figure 4 .

[0079] S1040: All base stations detect whether the training completion flag is 1, if yes, each base station loads the parameter information of the corresponding base station computing resource allocation network stored after the training in step S102 to obtain the general computing resource allocation network used in the execution phase, sets the execution number sequence to 1, and continues to execute sub-step S1042; if there is a base station with a training completion flag of 0, that is, a base station that has not completed training, the information of the training completion is waited for, and then the present step is executed again;

[0080] S1042: All base stations obtain the observation state of the base station respectively (all base stations are distributed and parallel processed);

[0081] S1044: Each base station inputs the observation state into the general computing resource allocation network of the current base station to obtain the general computing resource allocation strategy and output, including the base station computing capacity , the base station to satellite uplink communication resource allocation result , and the proportion of the allocated computing task of the base station

[0082] S1046: Perform edge computing of the satellite-ground cooperative system based on the base station computing resource allocation result, process the computing task, and update the observation state of the base station at the next time based on this, and the execution number is incremented by one;

[0083] S1048: Determine whether the execution number is greater than the maximum allowed execution number. If yes, end step four, and update and maintain the computing resource allocation network of each base station. If no, return to step S1042, and continue to wait for the next round of computing resource allocation.

[0084] S106: The base station executes the computing task received by the base station according to the computing resource allocation strategy.

[0085] In step S104, after determining the computing resource allocation strategy of each base station, the computing task received by the base station can be executed by using the computing resource allocation strategy of the base station for each base station in this step, and the execution of the computing task in this round of computing resource allocation is completed.

[0086] When a base station executes a computing task, the base station can specifically determine the allocated proportion of the computing task of the users covered by the base station according to the computing resource allocation strategy; determine the task edge computing execution strategy of the users covered by the base station according to the allocated proportion of the computing task; and execute the computing task according to the task edge computing execution strategy.

[0087] When the base station computing resource allocation method provided in the specification is used to allocate computing resources to the base stations in the satellite-ground cooperative system, a computing resource allocation network capable of outputting the computing resource allocation strategy of the base station according to the observation state of the base station can be set for each base station, and the central training through the satellite is performed in advance through reinforcement learning, and then in the application stage, the base station can not rely on the satellite, but only rely on its own observation state to realize the allocation of computing resources through the computing resource allocation network. Using this method can effectively reduce the frequent two-way communication between the satellite and the base station in the application, reduce the delay caused by communication, speed up the processing efficiency of the computing task, and greatly improve the user experience.

[0088] Figure 5 The effect diagram of the base station computing resource allocation method provided in the specification when actually tested. As shown in the satellite-ground cooperative system architecture Figure 2 , the satellite height is set to 8000 km, the satellite covers 10 base stations, each base station covers 5 users, the maximum allowed execution number is 101 after training, the user base station communication adopts the TDMA mode, and the optimal time slot allocation strategy is adopted, the base station and satellite uplink communication adopts the TDMA mode, and the time slot allocation adopts the base station to satellite uplink communication resource allocation result Calculate the average system total user total delay after 100 times (the first time is started). Reference Figure 5 It can be seen that, compared with the traditional method, the computing resource allocation method applied to the distributed base station in the satellite-ground cooperative system provided in the specification can effectively reduce the average system total user total delay. For example, when the satellite orbit height is 8000km, the delay is reduced by 27.5% compared with the traditional centralized decision method, and the delay is reduced by 18.35% compared with the average allocation method.

[0089] The above is the base station computing resource allocation method provided by the specification. Based on the same idea, the specification also provides a corresponding base station computing resource allocation device, as shown in Figure 6

[0090] Figure 6 A base station computing resource allocation device provided by the specification is shown in the figure, which specifically includes:

[0091] The initialization module 200 is configured to initialize the satellite-ground cooperative system.

[0092] The training module 202 is configured to train the computing resource allocation network of the base station by the satellite.

[0093] The allocation module 204 is configured to, for each base station, in response to the base station receiving a computing task, determine a computing resource allocation strategy of the base station according to an observation state of the base station by using the computing resource allocation network of the base station.

[0094] The execution module 206 is configured to execute the computing task received by the base station according to the computing resource allocation strategy.

[0095] Optionally, the initialization module 200 is specifically configured to initialize system parameters of the satellite-ground cooperative system and observation states of each base station.

[0096] Optionally, the system parameters include at least one of communication parameters, computing parameters, task parameters, and reinforcement learning parameters.

[0097] Optionally, the communication parameters at least include the number of base stations, the number of covered users of each base station, and the satellite orbit height; the computing parameters at least include the edge computing server computing load, the maximum available computing resource of the base station, and the maximum available computing resource of the satellite; the task parameters at least include the task computing amount; and the reinforcement learning parameters at least include the maximum epoch length, the discount factor, the experience replay pool size, the batch size, the base station computing resource allocation network learning rate, the evaluation network learning rate, the hidden layer size, the evaluation network update coefficient, the maximum time slot length, and the maximum allowed execution times.

[0098] ​Optionally, the observation state of the base station at least includes a current computing task amount, a last round computing task amount, a last round satellite computing capacity allocation result, and a last round computing resource allocation result.

[0099] Optionally, the execution module 206 is specifically configured to determine, by the base station, a computing task allocation proportion of the base station according to the computing resource allocation strategy; determine a task edge computing execution strategy of the base station according to the computing task allocation proportion; and execute the computing task according to the task edge computing execution strategy.

[0100] Optionally, the training module 202 is specifically configured to determine a current observation state of each base station; input, for each base station, the current observation state into a to-be-trained computing resource allocation network to obtain a current computing resource allocation strategy of the base station output by the computing resource allocation network; determine a current computing task allocation proportion of the base station according to the current computing resource allocation strategy; perform allocation according to the current computing resource allocation strategy of the base station, execute the computing task allocated to the base station, and transmit the computing task allocated to the satellite, the current observation state, and the current computing resource allocation strategy to the satellite to determine a next observation state of the base station, a time consumption of executing the allocated computing task, and a communication delay; acquire, by the satellite, the current observation state of each base station as a current global state, acquire the current computing resource allocation strategy of each base station as a global allocation strategy, acquire the next observation state of each base station as a next global state, and respectively return the next observation state of each base station to the corresponding base station; for each base station, the satellite inputs the current global state and the global allocation strategy, the next observation state and a total time consumption of executing all computing tasks as inputs, updates a parameter of an evaluation network of the base station; determines, by the evaluation network, a distribution strategy loss value of the base station, and updates the computing resource allocation network according to the distribution strategy loss value.

[0101] The specification also provides a computer readable storage medium storing a computer program, which can be used to execute the above method. Figure 1 The provided base station computing resource allocation method.

[0102] The specification also provides an electronic device. Figure 7 The schematic structural diagram of the electronic device is shown. As shown in the figure, Figure 7 According to the above method, the base station computing resource allocation method is provided. Figure 1The base station calculates the resource allocation method. Of course, in addition to the software implementation, the present specification does not exclude other implementations, such as logic devices or a combination of software and hardware, and the like, that is, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or a logic device.

[0103] For a technical improvement, it can be obvious whether the improvement is in hardware (e.g., improvement of circuit structures of diodes, transistors, switches, etc.) or in software (e.g., improvement of method flow). However, with the development of technology, many improvements of method flow today can be considered as direct improvements of hardware circuit structures. Designers almost always obtain the corresponding hardware circuit structures by programming the improved method flow into hardware circuits. Therefore, it cannot be said that an improvement of method flow cannot be implemented by hardware entity modules. For example, a programmable logic device (PLD) (e.g., a field programmable gate array (FPGA)) is an integrated circuit whose logic function is determined by user programming of the device. A designer programs a digital system "integrated" on a PLD by himself / herself, without having to ask a chip manufacturer to design and manufacture a special integrated circuit chip. Moreover, instead of manually manufacturing an integrated circuit chip, such programming is now mostly implemented by "logic compiler" software, which is similar to a software compiler used when developing programs, and the original code before compilation also needs to be written in a specific programming language, which is called a hardware description language (HDL), and there are many types of HDL, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc., and the most commonly used are VHDL (Very-High-Speed Integrated Circuit Hardware Description Language) and Verilog. It should be clear to those skilled in the art that only a little logical programming of the method flow in the above-mentioned hardware description languages and programming into an integrated circuit can easily obtain a hardware circuit that implements the logical method flow.

[0104] The controller can be implemented in any suitable way, for example, the controller can take the form of a microprocessor or processor and a computer readable medium storing computer readable program code, such as software or firmware, executable by the (micro)processor, logic gates, switches, an application specific integrated circuit (ASIC), a programmable logic controller and an embedded microcontroller, examples of which include but are not limited to the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20 and Silicone Labs C8051F320, the memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also know that, in addition to being implemented in pure computer readable program code, the controller can equally well be implemented to perform the same functions using logic gates, switches, an application specific integrated circuit, a programmable logic controller and an embedded microcontroller, etc. by means of a logical programming of the method steps. The controller can thus be considered as a hardware component, and the means comprised therein for performing the various functions can be considered as structures within the hardware component. Alternatively, the means for performing the various functions can even be considered as both a software module implementing the method and a structure within the hardware component.

[0105] The systems, apparatuses, modules or units illustrated by the above embodiments can be specifically implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer can be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0106] For the sake of description, the above apparatuses are described in functional division and are described respectively. Of course, the functions of the units can be implemented in the same or multiple software and / or hardware when implementing the present specification.

[0107] Those skilled in the art will understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer usable program code.

[0108] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flowcharts and / or blocks in the flowcharts and / or combination thereof. Figure 1 one or more flowcharts and / or blocks in the flowcharts and / or combination thereof.

[0109] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the function specified in the flowchart block or blocks. Figure 1 one or more flowcharts and / or blocks in the flowcharts and / or combination thereof. Figure 1 one or more flowcharts and / or blocks in the flowcharts and / or combination thereof.

[0110] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flowcharts and / or blocks in the flowcharts and / or combination thereof. ​ one or more flowcharts and / or blocks in the flowcharts and / or combination thereof.

[0111] In one typical configuration, the computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0112] The memory can include non-persistent memory and / or volatile memory, such as random access memory (RAM) and / or cache memory, non-volatile memory, such as read-only memory (ROM), EPROM, and / or flash memory. The memory is an example of computer-readable media.

[0113] Computer-readable media includes permanent and non-permanent, movable and non-movable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible to a computing device. According to the definition herein, computer-readable media does not include transitory media such as modulated data signals and carriers.

[0114] It should also be noted that the terms "comprising", "containing", or any other variant thereof are intended to cover non-exclusive inclusion, such that a process, method, article or apparatus that comprises a list of elements does not only include those elements, but can also include other elements not expressly listed or inherent to such process, method, article or apparatus. Without more limitations, the element defined by the statement "comprising a" does not exclude the presence of additional identical elements in the process, method, article or apparatus that includes the element.

[0115] Those skilled in the art will appreciate that embodiments of the present specification can be provided as methods, systems or computer program products. Therefore, the present specification can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Moreover, the present specification can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0116] The present specification can be described in the general context of computer-executable instructions, such as program modules, executed by computers. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The present specification can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in both local and remote computer storage media including storage devices.

[0117] The various embodiments described in this specification are described using a numbering of embodiments approach: these are each individually integrated contributions pertaining to different but related aspects of the description. Each of the various embodiments can stand on its own, and each can be combined with the subject matter of other embodiments to produce further embodiments. Where appropriate, therefore, the contents of the specification can be regarded as incorporating text of the detailed description under the provisions of 35 U.S.C. § 112(f), including the equivalents of any amended claims.

[0118] The above description is embodied in the form of embodiments only and is not intended to limit the present specification. The present specification can be variously changed and modified by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present specification should be included in the scope of the claims of the present application.

Claims

1. A method of allocating resources in a base station, characterized by The method is applied to a satellite-ground cooperative system, the satellite-ground cooperative system comprising at least satellites and base stations, the base stations representing base stations operating on the ground; each base station serving a plurality of users within a base station coverage range; each satellite corresponding to and interfacing with a base station within its own signal coverage range; the method comprising: initializing the satellite-ground cooperative system; training, by the satellites, a computational resource allocation network of the base stations by means of reinforcement learning; for each base station, in response to the base station receiving a computing task, determining, by means of the computational resource allocation network of the base station, a computational resource allocation strategy of the base station according to an observation state of the base station; the observation state of the base station comprising at least a current computing task computing amount, a previous computing task computing amount, a previous satellite computing capacity allocation result, and a previous computational resource allocation result; the base station executing the computing task received by the base station according to the computational resource allocation strategy.

2. The method of claim 1, wherein, The initializing of the satellite-ground cooperative system comprises: initializing system parameters of the satellite-ground cooperative system and observation states of the base stations.

3. The method of claim 2, wherein, The system parameters comprise at least one of a communication parameter, a computing parameter, a task parameter, and a reinforcement learning parameter.

4. The method of claim 3, wherein, The communication parameter comprises at least a number of base stations, a number of users covered by each base station, and a satellite orbit height; the computing parameter comprises at least an edge computing server computing load, a maximum available computing resource of a base station, and a maximum available computing resource of a satellite; the task parameter comprises at least a computing amount size of a task; and the reinforcement learning parameter comprises at least a maximum epoch length, a discount factor, an experience replay pool size, a batch size, a base station computational resource allocation network learning rate, an evaluation network learning rate, a hidden layer size, an evaluation network update coefficient, a maximum time slot length, and a maximum allowed number of executions.

5. The method of claim 1, wherein, The base station executing the computing task received by the base station according to the computational resource allocation strategy comprises: the base station determining, according to the computational resource allocation strategy, an allocation proportion of computing tasks of users covered by the base station; determining, according to the allocation proportion of the computing tasks, a task edge computing execution strategy of the users covered by the base station; executing the computing tasks according to the task edge computing execution strategy.

6. The method of claim 1, wherein, The training, by the satellites, of the computational resource allocation network of the base stations comprises: determining a current observation state of each base station; for each base station, inputting the current observation state into the computational resource allocation network to be trained to obtain a current computational resource allocation strategy of the base station output by the computational resource allocation network; determining, according to the current computational resource allocation strategy, an allocation proportion of a current computing task of the base station; allocating, according to the current computational resource allocation strategy of the base station, the base station executing the computing task allocated to itself and transmitting, to the satellites, the computing task allocated to the satellites, the current observation state, and the current computational resource allocation strategy, determining a next observation state of the base station, a time consumption of executing the allocated computing task, and a communication delay. The satellite obtains a current observation state of each base station as a current global state, obtains a current overall resource allocation strategy of each base station as a global allocation strategy, obtains a next observation state of each base station as a next global state, and returns the next observation state of each base station to the corresponding base station respectively; For each base station, the satellite takes the current global state and the global allocation strategy, the next observation state and the total time consumed for executing all computing tasks as inputs, and updates parameters of an evaluation network of the base station; The evaluation network is used to determine an allocation strategy loss value of the base station, and the overall resource allocation network is updated according to the allocation strategy loss value.

7. A base station computing resource allocation device, characterized in that, The device is applied to a star-ground cooperative system, and the star-ground cooperative system at least includes satellites and base stations. The base station represents a base station running on the ground. Each base station serves a plurality of users in the coverage range of the base station. Each satellite corresponds to and communicates with a base station in the signal coverage range of the satellite. The device comprises: An initialization module is configured to initialize the star-ground cooperative system. A training module is configured to train an overall resource allocation network of the base station by using the satellite in a manner of reinforcement learning. An allocation module is configured to, for each base station, in response to the base station receiving a computing task, determine an overall resource allocation strategy of the base station by using the overall resource allocation network of the base station and according to an observation state of the base station. The observation state of the base station at least includes a current computing task computing amount, a last round computing task computing amount, a last round satellite computing capacity allocation result and a last overall resource allocation result. An execution module is configured to execute the computing task received by the base station according to the overall resource allocation strategy.

8. A computer-readable storage medium, characterized in that, The storage medium stores a computer program. When the computer program is executed by the processor, the method in any one of claims 1-6 is implemented.

9. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the program to implement the method in any one of claims 1-6.

Images