Base station general computing resource allocation method and device, storage medium and electronic equipment

Abstract

The invention discloses a base station general computing resource allocation method and device, a storage medium and electronic equipment. When the base station general calculation resource allocation method provided by the specification is adopted to allocate general calculation resources to base stations in a satellite-ground cooperative system, a general calculation resource allocation network capable of outputting general calculation resource allocation strategies of the base stations according to observation states of the base stations can be arranged in each base station; centralized training is carried out through a satellite in a reinforcement learning mode in advance, and then in the application stage, the base station does not depend on the satellite and only depends on the observation state of the base station to achieve allocation of general computing resources through the general computing resource allocation network. By adopting the method, frequent two-way communication between the satellite and the base station can be effectively reduced when the edge computing of the satellite-ground cooperative system is applied, the time delay generated by communication is reduced, the processing efficiency of a computing task is accelerated, and the user experience is greatly improved.

Description

Technical Field

[0001] The present specification relates to the field of computer technology, and in particular to a method, device, storage medium and electronic device for allocating computing resources of a base station. Background Art

[0002] Today, ground networks are increasingly unable to meet today's seamless global coverage needs. The satellite-ground collaborative communication system that integrates satellite and ground networks will be an important development direction for future mobile communications. Limited by the communication rate of the satellite-ground link, user service delays based on traditional cloud computing are relatively large. With the miniaturization of computing devices and the improvement of computing power, the processing capabilities of edge devices and terminals under the satellite-ground network system have been greatly enhanced. By introducing edge servers and performing calculations at the local edge layer, all calculations can be avoided from being uploaded to the cloud, which can effectively avoid the high network response delay of traditional cloud computing.

[0003] Due to factors such as power and volume, the computing capacity of edge servers is usually limited. Therefore, targeted edge computing resource allocation is required to minimize the task response delay of the satellite-ground collaborative communication system. At present, the existing base station computing resource allocation strategy usually considers centralized decision-making. In order to obtain global task information, all users need to communicate with the satellite, which in turn generates a two-way propagation delay between user-base station-satellite-base station-user. Taking a satellite at an orbital altitude of 8,000 km as an example, even if the user-base station propagation delay is ignored, the two-way propagation delay of a single user is as high as about 53 milliseconds, which greatly affects the user experience.

[0004] Therefore, how to improve the resource allocation strategy of the satellite-ground collaborative communication system to reduce the propagation delay is an urgent problem to be solved. Summary of the invention

[0005] The present specification provides a base station computing resource allocation method, device, storage medium and electronic device to at least partially solve the above-mentioned problems existing in the prior art.

[0006] This manual adopts the following technical solutions:

[0007] This specification provides a method for allocating base station computing resources, which is applied to a satellite-ground coordination system, wherein the satellite-ground coordination system includes at least a satellite and a base station, and the method includes:

[0008] Initializing the satellite-ground coordination system;

[0009] Using the satellite to train a computing resource allocation network of the base station;

[0010] For each base station, in response to the base station receiving the computing task, using the general computing resource allocation network of the base station, and determining the general computing resource allocation strategy of the base station according to the observed state of the base station;

[0011] The base station executes the computing task received by the base station according to the general computing resource allocation strategy.

[0012] Optionally, initializing the satellite-ground coordination system specifically includes:

[0013] Initialize the system parameters of the satellite-ground coordination system and the observation status of each base station.

[0014] Optionally, the system parameters include at least one of communication parameters, computing parameters, task parameters, and reinforcement learning parameters.

[0015] Optionally, the communication parameters include at least the number of base stations, the number of users covered by each base station, and the satellite orbit altitude; the computing parameters include at least the computing load of the edge computing server, the maximum available computing resources of the base station, and the maximum available computing resources of the satellite; the task parameters include at least the size of the task computing amount; the reinforcement learning parameters include at least the maximum epoch length, discount factor, experience replay pool size, batch size, base station computing resource allocation network learning rate, evaluation network learning rate, hidden layer size, evaluation network update coefficient, maximum time slot length, and maximum allowed number of executions.

[0016] Optionally, the observation state of the base station includes at least the calculation amount of the current calculation task, the calculation amount of the previous round of calculation tasks, the previous round of satellite calculation capacity allocation results, and the previous general calculation resource allocation results.

[0017] Optionally, the base station executes the computing task received by the base station according to the general computing resource allocation strategy, specifically including:

[0018] The base station determines, according to the general computing resource allocation strategy, the proportion of computing tasks allocated to users covered by the base station;

[0019] Determine the edge computing execution strategy for the tasks covered by the base station according to the proportion of the computing tasks allocated;

[0020] Execute the computing task according to the task edge computing execution strategy.

[0021] Optionally, the satellite is used to train a computing resource allocation network of the base station, specifically including:

[0022] Determine the current observation status of each base station;

[0023] For each base station, input the current observation state into a general computing resource allocation network to be trained, and obtain a current general computing resource allocation strategy of the base station output by the general computing resource allocation network;

[0024] Determine the current computing task allocation ratio of the base station according to the current computing resource allocation strategy;

[0025] The base station performs the computing task assigned to itself according to the current computing resource allocation strategy of the base station, and transmits the computing task assigned to the satellite, the current observation state and the current computing resource allocation strategy to the satellite, and determines the next observation state of the base station, the time consumption for executing the assigned computing task and the communication delay;

[0026] The satellite obtains the current observation state of each base station as the current global state, obtains the current computing resource allocation strategy of each base station as the global allocation strategy, obtains the next observation state of each base station as the next global state, and transmits the next observation state of each base station back to the corresponding base station respectively;

[0027] For each base station, the satellite takes the current global state and the global allocation strategy, the next observation state and the total time consumed to execute all computing tasks as input, updates the parameters of the evaluation network of the base station; determines the allocation strategy loss value of the base station through the evaluation network, and updates the general computing resource allocation network according to the allocation strategy loss value.

[0028] This specification provides a base station general computing resource allocation device, which is applied to a satellite-ground coordination system, wherein the satellite-ground coordination system at least includes a satellite and a base station, and the device includes:

[0029] An initialization module, used to initialize the satellite-ground coordination system;

[0030] A training module, used for training a computing resource allocation network of the base station using the satellite;

[0031] An allocation module, configured to, for each base station, in response to the base station receiving a computing task, use the general computing resource allocation network of the base station and determine the general computing resource allocation strategy of the base station according to the observed state of the base station;

[0032] An execution module is used for the base station to execute the computing task received by the base station according to the general computing resource allocation strategy.

[0033] This specification provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned base station computing resource allocation method is implemented.

[0034] This specification provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned base station computing resource allocation method when executing the program.

[0035] At least one of the above technical solutions adopted in this specification can achieve the following beneficial effects:

[0036] In the base station computing resource allocation method provided in the present specification, the satellite-ground coordination system is initialized; the computing resource allocation network of the base station is trained using the satellite; for each base station, in response to the base station receiving a computing task, the computing resource allocation network of the base station is used to determine the computing resource allocation strategy of the base station according to the observation state of the base station; the base station executes the computing task received by the base station according to the computing resource allocation strategy.

[0037] When using the base station general computing resource allocation method provided in this specification to allocate general computing resources to base stations in the satellite-ground collaborative system, a general computing resource allocation network that can output the general computing resource allocation strategy of the base station according to the observation state of the base station can be set at each base station, and centralized training can be performed through satellites in advance by means of reinforcement learning. Then, in the application stage, the base station can rely on its own observation state instead of satellites to achieve computing resource allocation through the general computing resource allocation network. The use of this method can effectively reduce the frequent two-way communication between satellites and base stations during application, reduce the delay caused by communication, speed up the processing efficiency of computing tasks, and greatly improve user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The drawings described herein are used to provide a further understanding of this specification and constitute a part of this specification. The illustrative embodiments and descriptions of this specification are used to explain this specification and do not constitute an improper limitation on this specification. In the drawings:

[0039] Figure 1 A schematic diagram of a flow chart of a base station general computing resource allocation method in this specification;

[0040] Figure 2 A schematic diagram of the system architecture of a feasible satellite-ground coordination system provided in this specification;

[0041] Figure 3 A schematic diagram of the steps of training a general computing resource allocation network and an evaluation network provided in this specification;

[0042] Figure 4 A schematic diagram of the steps for allocating computing resources in a satellite-ground collaborative system provided in this specification;

[0043] Figure 5 This is a schematic diagram of the effect achieved by the base station general computing resource allocation method provided in this specification during actual testing;

[0044] Figure 6 A schematic diagram of a base station computing resource allocation device provided in this specification;

[0045] Figure 7 The corresponding Figure 1 Schematic diagram of electronic equipment. DETAILED DESCRIPTION

[0046] In order to make the purpose, technical solutions and advantages of this specification more clear, the technical solutions of this specification will be clearly and completely described below in combination with the specific embodiments of this specification and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0047] The technical solutions provided by the embodiments of this specification are described in detail below in conjunction with the accompanying drawings.

[0048] Figure 1 This is a flow chart of a method for calculating resource allocation of a base station in this specification, which specifically includes the following steps:

[0049] S100: Initializing the satellite-ground coordination system.

[0050] The base station computing resource allocation provided in this specification is applied to a satellite-ground coordination system including at least several satellites and several base stations, and each base station can cover several users. Figure 2 A schematic diagram of the system architecture of a feasible satellite-ground coordination system provided for this specification. A satellite-ground coordination system may include several satellites operating in space and several base stations operating on the ground. Usually, a satellite is responsible for achieving docking and communication with multiple base stations. The correspondence between satellites and base stations is generally divided according to regions, that is, each satellite corresponds to and communicates with base stations within its own signal coverage, and each base station serves several users within the coverage of the base station, and each user establishes a connection with the base station according to his or her own needs.

[0051] Figure 2 Taking a satellite and several base stations as an example, the relationship between satellites and base stations is shown. Figure 2 As shown in Figure 1, each satellite or base station has an edge server for implementing computing functions. For any base station, when the base station receives a computing task sent by a user, in order to complete the processing of the computing task more quickly, the base station can transfer part of the computing task to the satellite for processing. Figure 2 In this case, the calculation migration is from the user to the base station, and from the base station to the satellite. The calculation task processed by the base station can be directly returned to the user by the base station itself, and the calculation task processed by the satellite can return the calculation result to the ground station and then to the user. Among them, the ground station in the satellite-ground collaborative system is not the focus of this method, and this specification will not discuss it further.

[0052] All steps in this method can be implemented by any electronic device with computing functions, such as terminals, servers and other devices. This specification is mainly described by taking the devices that execute the steps in this method as edge servers on satellites and base stations in a satellite-ground coordination system. It is worth mentioning that in this specification, a group of corresponding satellites and base stations, that is, a satellite and the base stations covered by the satellite, are mainly described. For each group of satellites and base stations in the satellite-ground coordination system, the allocation of computing resources can be achieved according to the base station computing resource allocation method provided in this specification.

[0053] This method is applied to give a reasonable general computing resource allocation strategy to the base station without relying on the satellite when any base station in the satellite-ground coordination system receives a computing task that needs to be processed, so as to ensure the overall efficient operation of the satellite-ground coordination system while quickly utilizing satellites and base stations to process computing tasks. Among them, the general computing resource allocation strategy includes base station computing capacity allocation, the proportion of computing allocated by the base station, and satellite-ground communication resource allocation. The base station computing capacity allocation represents the computing capacity of the computing tasks allocated by the base station to each user covered by the base station in the batch of computing tasks. The proportion of computing allocated by the base station represents the proportion of computing tasks allocated to the base station itself by each computing task in the batch of computing tasks to all user computing tasks. Based on this, in this step, the satellite-ground coordination system can be initialized first. That is, before the satellite-ground coordination system is officially put into use, the various settings of the satellite-ground coordination system are initialized.

[0054] Specifically, the system parameters of the satellite-ground coordination system and the observation status of each base station can be initialized. In this method, the parameters of the satellite-ground coordination system may include but are not limited to communication parameters, computing parameters, task parameters, and reinforcement learning parameters. Among them, the communication parameters may at least include the number of base stations, the number of users covered by each base station, the satellite orbit height, the base station satellite uplink bandwidth, and the base station satellite uplink transmission power; the computing parameters may at least include the computing load of the edge computing server, the maximum available computing resources of the base station, and the maximum available computing resources of the satellite; the task parameters may at least include the size of the task calculation amount; the reinforcement learning parameters may at least include the maximum epoch length, discount factor, experience replay pool size, batch size, base station general computing resource allocation network learning rate, evaluation network learning rate, hidden layer size, evaluation network update coefficient, maximum time slot length, and maximum allowed number of executions.

[0055] Each base station has a separate observation state. The observation state of a base station may at least include the current coverage task calculation size, the previous round of coverage task calculation size, the previous round of satellite computing capacity allocation results, and the previous general computing resource allocation strategy.

[0056] When initializing various parameters, for parameter items with clear data such as communication parameters, calculation parameters, and task parameters, the determined data can be directly filled in as parameter values; and for reinforcement learning parameters and other parameters that need to be determined through subsequent training and learning of the neural network model, as well as the observation status of the base station, they can be set by random initialization.

[0057] In a specific embodiment, when initializing various parameters, the communication parameters may be set to include the number of base stations N covered by the satellite, the maximum number of users K covered by each base station, n , satellite orbit height H, base station satellite uplink bandwidth B b , uplink transmission power between the nth base station and the satellite The calculation parameters include the edge computing server per bit computing load C of the kth user under the nth base station n,k , the maximum available computing resource capacity of the base station F b , the maximum available computing resource F of the satellite s The task parameters are mainly the amount of calculation of the user task, that is, the amount of calculation task of the kth user under the nth base station at time t The reinforcement learning parameters include the maximum epoch length M, the discount factor γ, and the experience replay pool size. Batch size S, base station computing resource allocation strategy network learning rate β A , reinforcement learning critic network learning rate β C , hidden layer size d, critic network soft update coefficient τ, maximum time slot length T, maximum allowed execution times T m .

[0058] S102: Using the satellite to train the computing resource allocation network of the base station.

[0059] After completing the initialization of the satellite-ground coordination system in step S100, in this step, the satellite can be used to train the general computing resource allocation network contained in the base stations covered by it. In this method, each base station contains a general computing resource allocation network, which is used to give the general computing resource allocation strategy of the base station when the base station receives a user computing task. When training the general computing resource allocation network in this step, it can be specifically determined that the current observation state of each base station; for each base station, the current observation state is input into the general computing resource allocation network to be trained to obtain the current general computing resource allocation strategy of the base station output by the general computing resource allocation network; the allocation ratio of the current computing task of the base station is determined according to the current general computing resource allocation strategy; the base station is allocated according to the current general computing resource allocation strategy of the base station, and the base station executes the computing task assigned to itself, and transmits the computing task assigned to the satellite, the current observation state and the current general computing resource allocation strategy to the satellite, determines the next observation state of the base station, executes the allocated The satellite obtains the current observation state of each base station as the current global state, obtains the current general computing resource allocation strategy of each base station as the global allocation strategy, obtains the next observation state of each base station as the next global state, and transmits the next observation state of each base station back to the corresponding base station respectively; for each base station, the satellite takes the current global state and the global allocation strategy, the next observation state and the total time consumed for executing all computing tasks as input, and updates the parameters of the evaluation network of the base station; determines the allocation strategy loss value of the base station through the evaluation network, and updates the general computing resource allocation network according to the allocation strategy loss value.

[0060] In the above method, the role of the general computing resource allocation network is to determine the general computing resource allocation strategy of the base station, and the role of the evaluation network is to evaluate the general computing resource allocation strategy output by the general computing resource allocation network and feedback the reward value. It should be noted that the role of the reward value is to adjust the parameters of the computing resource allocation network. That is to say, in this method, the evaluation network only exists in the training stage. After it is actually put into use, only the general computing resource allocation network needs to be used, and the evaluation network no longer exists. Among them, the input of the general computing resource allocation network is the current observation state of the base station, and the output is the general computing resource allocation strategy; the input of the evaluation network is the current global state, and the output is the reinforcement learning action reward value; the current global state is composed of the current observation states of all base stations.

[0061] In a specific embodiment, the current observation state of the base station The current coverage task calculation size of all users covered by the base station The amount of computation of the last round of coverage tasks D t-1 , the capacity allocation result of the last round of satellite calculation F s,t-1 and the resource allocation strategy of the previous round Composition, where the current coverage task calculation size It is composed of the computational size of all tasks submitted by all users covered by the base station at time t. The satellite computing capacity allocation result in the last round is the computing capacity result processed by the satellite when all users covered by all base stations within the satellite coverage area submitted computing tasks in the last round. The computational size of the previous round of tasks is composed of the computational size of all user tasks in the previous round. The resource allocation strategy of the previous round of general calculation includes the previous round of base station calculation capacity allocation results of all users covered by the current base station, the satellite-to-ground communication resource allocation results and the allocation ratio of base station calculation tasks. More optimally, the proportion of base station computing tasks can be calculated using the following formula: Where c is the speed of light, is the satellite-to-ground uplink transmission rate, N0 is the noise level, The satellite-to-ground link channel fading, both of which can be obtained based on traditional communication technology, are not the focus of this method, and are not further discussed in this manual. Preferably, the current observation state of the base station In this embodiment, it can be normalized and used, that is, (where D max is the maximum possible value of the computing task amount).

[0062] The current global state consists of the current observed states of all base stations, namely The global allocation strategy consists of the computing resource allocation strategies of all base stations, namely Furthermore, when initializing the current observation state and the current global state of all base stations, that is, at t = 1, since the historical information of the last time (t = 0) is not available (the superscript is t-1), D t-1 It can be initialized by randomly generating and F s,t-1 The average allocation method is used for initialization. The general resource allocation strategy network of any base station The input is the current observation status of the base station The network parameters are θ n , the output is the computing resource allocation strategy of the base station The target computation resource allocation strategy network is: In this embodiment, the direct output value of the base station's general resource allocation network is processed as follows to ensure The value range of the elements is [-1,1], that is, Reuse There can be two evaluation networks in each base station, namely Target Evaluation Network The inputs are the current global state and the current computing resource allocation strategy of the base station, and the output is the reward value of the reinforcement learning action. n , All parameters can be randomly initialized, and then the initialized θ n , Assign to To complete the parameter initialization of the evaluation network.

[0063] Furthermore, when training the computing resource allocation network, there may be a variety of different training methods. This specification provides a specific embodiment for reference. Figure 3 This is a schematic diagram of the steps of training a computing resource allocation network and an evaluation network provided in this specification. Specifically, Figure 3 The steps for training are given:

[0064] S1020: Get the current global status of the satellite-ground coordination system Set the time slot number to t=1;

[0065] S1022: For each base station (n is 1, 2, ..., N in sequence), use the general computing resource allocation network of the base station And add noise, that is, according to Give the current computing resource allocation strategy of the base station

[0066] S1024: According to the current general computing resource allocation strategy Perform general resource allocation and obtain the general resource allocation strategy a for each base station t , and based on this, calculate the service response delay r of the satellite-ground collaborative system when processing user computing tasks t and the next global state s of the system at the next moment t+1 , will (s t 、a t 、r t 、s t+1 ) The four-tuple is put into the experience replay pool, and the base station index value n is set to 1;

[0067] S1026: For the base station n corresponding to the current base station index value, randomly take out a batch size (s) from the experience replay pool. t 、a t 、r t 、s t+1 ) four-tuple, forming a training batch {(s i ,a i ,r i ,s i+1 )} i＝1,…,S

[0068] S1028: Calculate the intermediate value of the evaluation network update for the current base station n and network loss value And update the evaluation network of the current base station accordingly

[0069] S10210: Calculate the sampled policy gradient and calculate the general resource allocation network loss value of the current base station based on it Then update the parameters of the computing resource allocation network of the current base station accordingly;

[0070] S10212: Update the target review network and target general resource allocation network of the current base station in a soft update manner At the same time, the base station index value is increased by one (n←n+1);

[0071] S10214: Determine whether the base station index value exceeds the number of base stations in the current area (whether n is greater than N). If so, the iteration ends and the process goes to sub-step S10216. If not, the iteration continues and the process goes back to sub-step S1026.

[0072] S10216: Update the average system response delay in the current epoch Then use the next global state of the system at the next moment to replace the current global state (s t ←s t+1 ) , the time slot number is increased by one (t←t+1);

[0073] S10218: Determine whether the time slot number exceeds the maximum time slot length (whether t is greater than T). If so, the iteration ends and enters sub-step S10220. If not, the iteration continues and returns to sub-step S1022. The maximum time slot length can be preset according to specific needs.

[0074] S10220: If the epoch number is 1, the optimal average system response delay is set to the average system response delay in the current epoch, that is, Save the parameters θ of the resource allocation network for all base stations n ; If the epoch number is greater than 1, determine whether the average system response delay in the current epoch is less than the optimal average system response delay. If so, replace the optimal average system response delay with the average system response delay in the current epoch, that is, And save the parameters θ of the resource allocation network for all base stations n , if it is not less than , no operation is performed; after completing the above operation, the epoch number is increased by one (e←e+1);

[0075] S10222: Determine whether the maximum epoch length is exceeded (whether e is greater than M). If so, the iteration ends, the training ends, and the training completion labels on all base stations are set to 1, indicating that the training is completed; if not, continue the iteration and return to sub-step S1020, where the epoch length is used to characterize the training rounds, and the maximum epoch length can be pre-set according to specific needs.

[0076] S104: For each base station, in response to the base station receiving a computing task, a general computing resource allocation network of the base station is used to determine a general computing resource allocation strategy of the base station according to an observed state of the base station.

[0077] After completing the training of all base stations in step S102, the satellite-ground coordination system can be put into use in this step. At this time, the user can submit computing tasks to the base station in the coverage area where he is located. Each base station will process all computing tasks submitted by all users according to a fixed period. At the time corresponding to the period, the base station can input the observation state of the base station at that time into the computing resource allocation network to determine the general computing resource allocation strategy, that is, determine the task capacity of the computing tasks processed by the base station itself, which is equivalent to determining the task capacity of the computing tasks uploaded to the satellite and processed by the satellite.

[0078] Figure 4 This is a schematic diagram of the steps for allocating computing resources in a satellite-ground collaborative system provided in this specification. Specifically, Figure 4 The steps shown perform the computational resource allocation of the base station:

[0079] S1040: All base stations detect whether the training tag is completed. If it is 1, each base station loads the parameter information of the corresponding base station computing resource allocation network stored after the training is completed in step S102, obtains the general computing resource allocation network used in the execution stage, sets the execution number to 1, and continues to execute sub-step S1042; if there is a base station with a training completion tag of 0, that is, the training has not been completed, wait for the training completion information before executing this step;

[0080] S1042: All base stations obtain the observation status of the base stations respectively (all base stations perform distributed and parallel operations);

[0081] S1044: Each base station inputs the observation state into the general computing resource allocation network of the current base station, obtains and outputs the general computing resource allocation strategy, including the base station calculation capacity Result of uplink communication resource allocation from base station to satellite And the proportion of calculations allocated by base stations

[0082] S1046: Execute edge computing of the satellite-ground collaborative system based on the calculation resource allocation result of the base station, process the computing task, and update the observation state of the base station at the next moment based on the result, and increase the execution count sequence number by one;

[0083] S1048: Determine whether the execution count sequence number is greater than the maximum allowed execution count. If so, end step 4 and update and maintain the computing resource allocation network of each base station. If so, return to sub-step S1042 and continue to wait for the next round of computing resource allocation. The maximum allowed execution count can be pre-set according to specific needs.

[0084] S106: The base station executes the computing task received by the base station according to the general computing resource allocation strategy.

[0085] In step S104, after determining the general computing resource allocation strategy for each base station, in this step, for each base station, the general computing resource allocation strategy of the base station is used to execute the computing task received by the base station, thereby completing the general computing resource allocation and the execution of the computing task.

[0086] When any base station executes a computing task, the base station can specifically determine the proportion of computing tasks allocated to users covered by the base station according to the general computing resource allocation strategy; determine the task edge computing execution strategy for users covered by the base station according to the proportion of computing tasks allocated; and execute the computing task according to the task edge computing execution strategy.

[0087] When using the base station computing resource allocation method provided in this specification to allocate computing resources to base stations in the satellite-ground collaborative system, a computing resource allocation network that can output the computing resource allocation strategy of the base station according to the observation state of the base station can be set at each base station, and centralized training can be performed through satellites in advance by means of reinforcement learning. Then, in the application stage, the base station can rely on its own observation state instead of satellites to achieve computing resource allocation through the computing resource allocation network. The use of this method can effectively reduce the frequent two-way communication between satellites and base stations during application, reduce the delay caused by communication, speed up the processing efficiency of computing tasks, and greatly improve user experience.

[0088] Figure 5 This is a schematic diagram of the effect achieved by the base station computing resource allocation method provided in this specification during actual testing. Figure 2 The satellite-ground collaborative system architecture shown in the figure sets the satellite altitude to 8000km, the satellite covers 10 base stations, and each base station covers 5 users. After the training is completed, the maximum number of executions allowed is 101. The user-base station communication adopts the TDMA method and adopts the optimal time slot allocation strategy. The base station and satellite uplink communication adopts the TDMA method, and the time slot allocation adopts the base station to satellite uplink communication resource allocation result. Calculate the average total delay of all users in the system for the next 100 times (the first time is startup). Figure 5 It can be seen that, compared with the traditional method, the computing resource allocation method for distributed base stations in the satellite-ground collaborative system proposed in this specification can effectively reduce the average total delay of all users in the system. For example, when the satellite orbit altitude is 8000km, the delay is reduced by 27.5% compared with the traditional centralized decision-making method, and the delay is reduced by 18.35% compared with the average allocation method.

[0089] The above is the base station computing resource allocation method provided in this specification. Based on the same idea, this specification also provides a corresponding base station computing resource allocation device, such as Figure 6 shown.

[0090] Figure 6 A schematic diagram of a base station computing resource allocation device provided in this specification specifically includes:

[0091] An initialization module 200 is used to initialize the satellite-ground coordination system;

[0092] A training module 202, configured to train a computing resource allocation network of the base station using the satellite;

[0093] The allocation module 204 is used to determine, for each base station, in response to the base station receiving the computing task, a general computing resource allocation network of the base station and, according to the observed state of the base station, a general computing resource allocation strategy of the base station;

[0094] The execution module 206 is used for the base station to execute the computing task received by the base station according to the general computing resource allocation strategy.

[0095] Optionally, the initialization module 200 is specifically used to initialize system parameters of the satellite-ground coordination system and the observation status of each base station.

[0096] Optionally, the system parameters include at least one of communication parameters, computing parameters, task parameters, and reinforcement learning parameters.

[0097] Optionally, the communication parameters include at least the number of base stations, the number of users covered by each base station, and the satellite orbit altitude; the computing parameters include at least the computing load of the edge computing server, the maximum available computing resources of the base station, and the maximum available computing resources of the satellite; the task parameters include at least the size of the task computing amount; the reinforcement learning parameters include at least the maximum epoch length, discount factor, experience replay pool size, batch size, base station computing resource allocation network learning rate, evaluation network learning rate, hidden layer size, evaluation network update coefficient, maximum time slot length, and maximum allowed number of executions.

[0098] Optionally, the observation state of the base station includes at least the calculation amount of the current calculation task, the calculation amount of the previous round of calculation tasks, the previous round of satellite calculation capacity allocation results, and the previous general calculation resource allocation results.

[0099] Optionally, the execution module 206 is specifically used for the base station to determine the proportion of computing tasks allocated to users covered by the base station according to the general computing resource allocation strategy; determine the task edge computing execution strategy for users covered by the base station according to the proportion of computing tasks allocated; and execute the computing tasks according to the task edge computing execution strategy.

[0100] Optionally, the training module 202 is specifically used to determine the current observation state of each base station; for each base station, the current observation state is input into the general computing resource allocation network to be trained, and the current general computing resource allocation strategy of the base station output by the general computing resource allocation network is obtained; according to the current general computing resource allocation strategy, the allocation is performed according to the current general computing resource allocation strategy of the base station, the base station executes the computing task assigned to itself, and transmits the computing task assigned to the satellite, the current observation state and the current general computing resource allocation strategy to the satellite, determines the next observation state of the base station, executes the allocated computing task The satellite obtains the current observation state of each base station as the current global state, obtains the current general computing resource allocation strategy of each base station as the global allocation strategy, obtains the next observation state of each base station as the next global state, and transmits the next observation state of each base station back to the corresponding base station respectively; for each base station, the satellite takes the current global state and the global allocation strategy, the next observation state and the total time consumed for executing all computing tasks as input, and updates the parameters of the evaluation network of the base station; determines the allocation strategy loss value of the base station through the evaluation network, and updates the general computing resource allocation network according to the allocation strategy loss value.

[0101] This specification also provides a computer-readable storage medium, which stores a computer program, which can be used to execute the above Figure 1 A base station computing resource allocation method is provided.

[0102] This manual also provides Figure 7 The schematic structure diagram of the electronic device shown in FIG. Figure 7 As mentioned above, at the hardware level, the electronic device includes a processor, an internal bus, a network interface, a memory, and a non-volatile memory, and may also include other hardware required for the business. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to achieve the above Figure 1Of course, in addition to the software implementation, this specification does not exclude other implementations, such as logic devices or a combination of software and hardware, etc., that is, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0103] For the improvement of a technology, it can be clearly distinguished whether it is a hardware improvement (for example, improvement of the circuit structure of diodes, transistors, switches, etc.) or a software improvement (improvement of the method flow). However, with the development of technology, many improvements of the method flow today can be regarded as direct improvements of the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that the improvement of a method flow cannot be implemented with a hardware entity module. For example, a programmable logic device (PLD) (such as a field programmable gate array (FPGA)) is such an integrated circuit whose logical function is determined by the user's programming of the device. Designers can "integrate" a digital system on a PLD by programming themselves, without having to ask chip manufacturers to design and make dedicated integrated circuit chips. Moreover, nowadays, instead of manually making integrated circuit chips, this kind of programming is mostly implemented by "logic compiler" software, which is similar to the software compiler used when developing and writing programs, and the original code before compilation must also be written in a specific programming language, which is called hardware description language (HDL). There is not only one HDL, but many kinds, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also know that it is only necessary to program the method flow slightly in the above-mentioned hardware description languages ​​and program it into the integrated circuit, and then it is easy to obtain the hardware circuit that implements the logic method flow.

[0104] The controller can be implemented in any appropriate manner, for example, the controller can take the form of a microprocessor or processor and a computer-readable medium storing a computer-readable program code (such as software or firmware) that can be executed by the (micro)processor, a logic gate, a switch, an application-specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320. The memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art also know that in addition to implementing the controller in a purely computer-readable program code manner, the controller can be implemented in the form of a logic gate, a switch, an application-specific integrated circuit, a programmable logic controller, and an embedded microcontroller by logically programming the method steps. Therefore, this controller can be considered as a hardware component, and the devices included therein for implementing various functions can also be regarded as structures within the hardware component. Or even, the devices for implementing various functions can be regarded as both software modules for implementing the method and structures within the hardware component.

[0105] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0106] For the convenience of description, the above device is described in various units according to their functions. Of course, when implementing this specification, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0107] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0108] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0109] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0110] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0111] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0112] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0113] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0114] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0115] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems or computer program products. Therefore, this specification may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0116] This specification may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. This specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0117] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0118] The above description is only an embodiment of this specification and is not intended to limit this specification. For those skilled in the art, this specification may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of this specification should be included in the scope of the claims of this application.

Claims

1. A method for allocating computing resources of a base station, characterized in that: The method is applied to a satellite-ground coordination system, the satellite-ground coordination system at least comprising a satellite and a base station, and the method comprises: Initializing the satellite-ground coordination system; Using the satellite to train a computing resource allocation network of the base station; For each base station, in response to the base station receiving the computing task, using the general computing resource allocation network of the base station, and determining the general computing resource allocation strategy of the base station according to the observed state of the base station; The base station executes the computing task received by the base station according to the general computing resource allocation strategy.

2. The method according to claim 1, characterized in that Initializing the satellite-ground coordination system specifically includes: Initialize the system parameters of the satellite-ground coordination system and the observation status of each base station.

3. The method according to claim 2, characterized in that The system parameters include at least one of communication parameters, computing parameters, task parameters, and reinforcement learning parameters.

4. The method according to claim 3, characterized in that The communication parameters include at least the number of base stations, the number of users covered by each base station, and the satellite orbit altitude; the computing parameters include at least the computing load of the edge computing server, the maximum available computing resources of the base station, and the maximum available computing resources of the satellite; the task parameters include at least the size of the task computing amount; the reinforcement learning parameters include at least the maximum epoch length, discount factor, experience replay pool size, batch size, base station computing resource allocation network learning rate, evaluation network learning rate, hidden layer size, evaluation network update coefficient, maximum time slot length, and maximum allowed number of executions.

5. The method according to claim 1, characterized in that The observation state of the base station includes at least the calculation amount of the current calculation task, the calculation amount of the previous round of calculation tasks, the previous round of satellite calculation capacity allocation results, and the previous general calculation resource allocation results.

6. The method according to claim 1, characterized in that The base station executes the computing task received by the base station according to the general computing resource allocation strategy, specifically including: The base station determines, according to the general computing resource allocation strategy, the proportion of computing tasks allocated to users covered by the base station; Determine the edge computing execution strategy for the tasks covered by the base station according to the proportion of the computing tasks allocated; Execute the computing task according to the task edge computing execution strategy.

7. The method according to claim 1, characterized in that The satellite is used to train the computing resource allocation network of the base station, specifically comprising: Determine the current observation status of each base station; For each base station, input the current observation state into a general computing resource allocation network to be trained, and obtain a current general computing resource allocation strategy of the base station output by the general computing resource allocation network; Determine the current computing task allocation ratio of the base station according to the current computing resource allocation strategy; The base station performs the computing task assigned to itself according to the current computing resource allocation strategy of the base station, and transmits the computing task assigned to the satellite, the current observation state and the current computing resource allocation strategy to the satellite, and determines the next observation state of the base station, the time consumption for executing the assigned computing task and the communication delay; The satellite obtains the current observation state of each base station as the current global state, obtains the current computing resource allocation strategy of each base station as the global allocation strategy, obtains the next observation state of each base station as the next global state, and transmits the next observation state of each base station back to the corresponding base station respectively; For each base station, the satellite takes the current global state and the global allocation strategy, the next observation state and the total time consumed to execute all computing tasks as input to update the parameters of the evaluation network of the base station; The allocation strategy loss value of the base station is determined through the evaluation network, and the general resource allocation network is updated according to the allocation strategy loss value.

8. A base station computing resource allocation device, characterized in that: The device is applied to a satellite-ground coordination system, the satellite-ground coordination system at least includes a satellite and a base station, and the device includes: An initialization module, used to initialize the satellite-ground coordination system; A training module, used for training a computing resource allocation network of the base station using the satellite; An allocation module, configured to, for each base station, in response to the base station receiving a computing task, use the general computing resource allocation network of the base station and determine the general computing resource allocation strategy of the base station according to the observed state of the base station; An execution module is used for the base station to execute the computing task received by the base station according to the general computing resource allocation strategy.

9. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the method according to any one of claims 1 to 7 is implemented.

Images