Satellite / cloud computing resource allocation method and apparatus, and computer device and storage medium

Abstract

The present application relates to a satellite / cloud computing resource allocation method and apparatus, and a computer device and a storage medium. The method comprises: acquiring a target computing task and a real-time resource status; inputting the target computing task and the real-time resource status into a resource allocation model; and by means of the resource allocation model, outputting a satellite / cloud resource allocation strategy, the resource allocation model comprising a resource estimation module and a resource optimization module, wherein the resource estimation module is used for determining pre-estimated resources to be occupied for the target computing task, and the resource optimization module is used for determining the satellite / cloud resource allocation strategy on the basis of the real-time resource status and the pre-estimated resources to be occupied. In the present application, inputting a target computing task and a real-time resource status into a resource allocation model, determining, by means of a resource estimation module, pre-estimated resources to be occupied, and in full consideration of a dynamic resource environment, outputting a satellite / cloud resource allocation strategy by means of a resource optimization module and on the basis of the real-time resource status and the pre-estimated resources to be occupied can effectively improve the utilization rate of computing resources.

Description

Star cloud operation resource allocation method and device, computer equipment and storage medium TECHNICAL FIELD

[0001] The present application relates to the technical field of satellite communication, in particular to a star cloud operation resource allocation method and device, computer equipment and storage medium. BACKGROUND

[0002] With the rapid development of cloud computing and artificial intelligence technology, space-air intelligent computing technology plays an increasingly important role in space data processing applications. Space-air intelligent computing can process a large amount of data on-orbit, reducing the dependence on ground transmission, which is crucial for Earth observation, disaster monitoring and other tasks. However, space-air intelligent computing faces problems such as limited computing resources, transmission rate changes in space environment, and unstable ground cloud server load when performing deep learning inference tasks, which pose new challenges to the rational allocation of operation resources.

[0003] In traditional technology, the allocation of star cloud operation resources usually adopts a static computing division strategy. However, due to the dynamic changes of operation resources, the static resource allocation method in traditional technology is difficult to adjust the operation resource allocation of star cloud collaborative inference tasks in real time under unknown dynamic resource environment, resulting in low utilization of operation resources. SUMMARY

[0004] Therefore, it is necessary to provide a star cloud operation resource allocation method, device, computer equipment and storage medium capable of improving the utilization of operation resources to solve the above technical problems.

[0005] In a first aspect, the present application provides a star cloud operation resource allocation method. The method comprises:

[0006] obtaining a target operation task and a real-time resource state;

[0007] inputting the target operation task and the real-time resource state into a resource allocation model, and outputting a star cloud resource allocation strategy from the resource allocation model, wherein the resource allocation model comprises a resource estimation module and a resource optimization module, the resource estimation module is used to determine the pre-occupied resources of the target operation task; the resource optimization module is used to determine the star cloud resource allocation strategy according to the real-time resource state and the pre-occupied resources;

[0008] based on the star cloud resource allocation strategy, the target operation task is allocated to a satellite and / or a ground cloud server for execution.

[0009] In one embodiment, the determination of the pre-occupied resources of the target operation task comprises:

[0010] determining a target neural network corresponding to the target operation task according to the target operation task;

[0011] determining pre-occupied resources of each layer of the target neural network when the target neural network executes the target operation task according to the real-time resource state;

[0012] determining the pre-occupied resources of each layer of the target neural network as pre-occupied resources of the target operation task.

[0013] In one embodiment, the determining the nebula resource allocation strategy according to the real-time resource state and the pre-occupied resources comprises:

[0014] determining Markov decision parameters according to the real-time resource state and the pre-occupied resources;

[0015] determining the nebula resource allocation strategy based on the Markov decision parameters and a deep deterministic policy gradient algorithm.

[0016] In one embodiment, the real-time resource state comprises satellite real-time resource state and cloud server real-time resource state, the Markov decision parameters comprise real-time state space parameters, and the determining the Markov decision parameters according to the real-time resource state and the pre-occupied resources comprises:

[0017] obtaining real-time space environment information and data output quantities of each layer of the target neural network;

[0018] determining the real-time state space parameters according to the real-time space environment information, the data output quantities of each layer of the target neural network, and the real-time resource state.

[0019] In one embodiment, the Markov decision parameters comprise real-time action space parameters, and the determining the Markov decision parameters according to the real-time resource state and the pre-occupied resources comprises:

[0020] determining a real-time satellite set to be allocated according to the satellite real-time resource state;

[0021] determining a set of division points of the target neural network according to the data output quantities of each layer of the target neural network, the real-time resource state, and the pre-occupied resources;

[0022] determining the real-time action space parameters according to the set of division points and the real-time satellite set to be allocated.

[0023] In one embodiment, the determining the nebula resource allocation strategy based on the Markov decision parameters and the deep deterministic policy gradient algorithm comprises:

[0024] determine a resource allocation strategy to be optimized according to the real-time state space parameter, the real-time action space parameter, and a preset reward factor;

[0025] filter the resource allocation strategy to be optimized according to the deep deterministic policy gradient algorithm, and determine the nebula resource allocation strategy.

[0026] In one embodiment, the preset reward factor includes at least one of:

[0027] a reasoning time delay shortening reward factor;

[0028] an energy efficiency reward factor;

[0029] a successful task execution reward factor;

[0030] a satellite resource consumption limitation reward factor;

[0031] a cloud server load limitation reward factor;

[0032] a satellite resource and cloud server resource balance reward factor;

[0033] a data transmission amount limitation reward factor between satellites;

[0034] a fault tolerance reward factor for fast response to environmental changes.

[0035] In one embodiment, the deep deterministic policy gradient algorithm includes an actor network and a critic network, and the filtering of the resource allocation strategy to be optimized according to the deep deterministic policy gradient algorithm and the determination of the nebula resource allocation strategy include:

[0036] the actor network determines a resource allocation strategy with the highest action value in the resource allocation strategy to be optimized as the nebula resource allocation strategy, wherein the action value is determined according to the critic network.

[0037] In a second aspect, the application further provides a nebula computing resource allocation device. The device includes:

[0038] a data acquisition module configured to acquire a target computing task and a real-time resource state;

[0039] a resource allocation module configured to input the target computing task and the real-time resource state into a resource allocation model, and output a nebula resource allocation strategy from the resource allocation model, wherein the resource allocation model includes a resource estimation module and a resource optimization module, the resource estimation module is configured to determine pre-occupied resources of the target computing task, and the resource optimization module is configured to determine the nebula resource allocation strategy according to the real-time resource state and the pre-occupied resources;

[0040] The resource allocation module is further configured to allocate the target operation task to a satellite and / or a ground cloud server for execution based on the nebula resource allocation strategy.

[0041] In a third aspect, the present application provides a computer device. The computer device comprises a memory and a processor. The memory stores a computer program. The processor implements the steps of the nebula operation resource allocation method in any of the first aspect when executing the computer program.

[0042] In a fourth aspect, the present application provides a computer readable storage medium. The computer readable storage medium stores a computer program. The computer program is executed by a processor to implement the steps of the nebula operation resource allocation method in the first aspect.

[0043] The nebula operation resource allocation method, device, computer device and storage medium can obtain a target operation task and a real-time resource state, input the target operation task and the real-time resource state into a resource allocation model, and output a nebula resource allocation strategy through the resource allocation model. The resource allocation model comprises a resource estimation module and a resource optimization module. The resource estimation module is configured to determine pre-occupied resources of the target operation task. The resource optimization module is configured to determine the nebula resource allocation strategy according to the real-time resource state and the pre-occupied resources. The target operation task and the real-time resource state are input into the resource allocation model, the pre-occupied resources are determined through the resource estimation module, and the nebula resource allocation strategy is output through the resource optimization module according to the real-time resource state and the pre-occupied resources under the condition of fully considering a dynamic resource environment, so that the operation resource utilization rate can be effectively improved.

[0044] The details of one or more embodiments of the present application are presented in the following drawings and description to make other features, objects and advantages of the present application more apparent. BRIEF DESCRIPTION OF DRAWINGS

[0045] The accompanying drawings illustrated herein are used to provide further understanding of the present application, and constitute a part of the present application. The illustrative embodiments of the present application and their description serve to explain the present application, and do not constitute improper limitations on the present application. In the drawings:

[0046] FIG. 1 is an application environment diagram of a nebula operation resource allocation method in an embodiment;

[0047] FIG. 2 is a flow diagram of a nebula operation resource allocation method in an embodiment;

[0048] FIG. 3 is a schematic diagram of a resource estimation module determining pre-occupied resources in an embodiment;

[0049] FIG. 4 is an application diagram of a nebula operation resource allocation method in an embodiment;

[0050] FIG. 5 is a schematic diagram of star cloud computing resource allocation in an embodiment;

[0051] FIG. 6 is a structural block diagram of a star cloud computing resource allocation apparatus in an embodiment;

[0052] FIG. 7 is an internal structure diagram of a computer device in an embodiment. DETAILED DESCRIPTION

[0053] In order to make the purposes, technical solutions and advantages of the present application clearer, the present application is further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and not intended to limit the present application.

[0054] Unless otherwise defined, technical terms and scientific terms used in the present application shall have the same meaning as understood by one of ordinary skill in the art to which the present application belongs. In the present application, the terms "one", "a", "an", "the", "these", and similar words do not represent a quantitative limitation, but can be singular or plural. In the present application, the terms "include", "contain", "have" and any variants thereof are intended to cover non-exclusive inclusion; for example, a process, method and system, product or device containing a series of steps or modules (units) are not limited to the listed steps or modules (units), but can include steps or modules (units) not listed, or can include other steps or modules (units) inherent to the process, method, product or device. In the present application, the terms "connected", "connected", "coupled" and similar words are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. In the present application, "multiple" means two or more. The association between the associated objects is described by the term "and / or", which means that there can be three relationships, for example, "A and / or B" can mean that A exists alone, A and B exist together, and B exists alone. In general, the character " / " represents an "or" relationship between the associated objects. In the present application, the terms "first", "second", "third" and the like are only used to distinguish similar objects, and do not represent a specific order of the objects.

[0055] The terms "module", "unit" and the like used below are a combination of software and / or hardware that can achieve a predetermined function. Although the apparatus described in the following embodiments is preferably implemented in hardware, implementation of software or a combination of software and hardware is also possible and contemplated.

[0056] In the field of nebula collaborative reasoning, deep reinforcement learning algorithms provide an adaptive solution. Through continuous interaction between the agent and the environment, it learns the strategy of computational partitioning and task offloading in a changing environment. Since the development of DQN (Deep Q-Network, a combination of deep learning and reinforcement learning algorithm), deep reinforcement learning has been rapidly developing. It not only has the powerful perception and feature extraction ability of deep learning, but also has the powerful control and policy learning ability of reinforcement learning, and performs well in complex environments. Therefore, deep reinforcement learning provides a new direction for solving the problem of nebula collaborative reasoning in space environment. Deep Deterministic Policy Gradient (DDPG) is a classic reinforcement learning algorithm that has been tried for nebula collaborative reasoning problems. Although deep reinforcement learning has shown great potential in many fields, in the space environment, how to effectively apply deep reinforcement learning to solve the difficulties of nebula collaborative reasoning, especially in the face of high-dimensional state space and continuous action space, is still an open problem.

[0057] On the other hand, deep reinforcement learning algorithms need to consider a variety of dynamic factors when dealing with problems in space environment, including but not limited to dynamic changes in space environment, dynamic changes in computing resources, etc. The dynamic changes of these factors put higher requirements on the design and training of algorithms.

[0058] Currently, the research on nebula collaborative reasoning system based on deep reinforcement learning is still in its early stages. How to design a reasoning system that can effectively handle the dynamic nature of space environment, computing resource constraints and complex information transmission conditions is a problem that needs to be solved in this field. In addition, how to build a deep reinforcement learning model that can generalize to unknown environments and has adaptive ability to achieve reasonable allocation of computing resources is also a challenge faced by researchers.

[0059] Based on this, the embodiment of the application provides a star cloud computing resource allocation method. The star cloud computing resource allocation method provided by the embodiment of the application can be applied to an application environment as shown in FIG. 1. Wherein, the satellite 102 communicates with the ground cloud server 104 through the network, and the satellites can also communicate with each other through the network for wireless communication in the space environment. The data storage system can store the data required to be processed by the ground cloud server 104. The data storage system can be integrated in the ground cloud server 104, or can be deployed in the storage module of the satellite 102. The satellite 102 obtains the target computing task and the real-time resource state and sends them to the ground cloud server 104. The ground cloud server 104 inputs the target computing task and the real-time resource state into the resource allocation model, and outputs the star cloud resource allocation strategy through the resource allocation model. Wherein, the resource allocation model includes a resource estimation module and a resource optimization module. The resource estimation module is used to determine the pre-occupied resources of the target computing task. The resource optimization module is used to determine the star cloud resource allocation strategy according to the real-time resource state and the pre-occupied resources. In other embodiments, the ground cloud server can also obtain the target computing task and the real-time resource state, and send the star cloud resource allocation strategy to the satellite 102 after determining the star cloud resource allocation strategy. Wherein, the satellite 102 includes a single satellite or a satellite cluster composed of multiple satellites. The ground cloud server 104 can include a single server or a server cluster composed of multiple servers.

[0060] In the embodiment of the application, as shown in FIG. 2, a star cloud computing resource allocation method is provided. Taking the application scenario in FIG. 1 as an example, the method includes the following steps:

[0061] S201: Obtain the target computing task and the real-time resource state.

[0062] In the embodiment of the application, the star cloud includes satellites and ground cloud servers. The satellites can include a single satellite edge device or a satellite cluster composed of at least two satellite edge devices. The ground cloud servers can include a single server or a ground cloud server cluster composed of at least two servers. The star cloud computing resource allocation includes any one of satellite computing resource allocation, ground cloud server computing resource allocation, or satellite and ground cloud server computing resource collaborative allocation.

[0063] In the embodiment of the application, obtaining the target computing task can include the satellite obtaining an image inference and recognition task in space, or the ground cloud server publishing an image inference and recognition task after receiving the image obtained by the satellite. In other embodiments, the target computing task can also include other computing tasks, and the application does not limit the specific type of the target computing task. In some embodiments, the target computing task can be executed by a target neural network.

[0064] The real-time resource state includes real-time available computing resources of the satellites and / or real-time available computing resources of the ground cloud servers. Specifically, the real-time available resources of the satellites include available computing resources of each satellite, and the real-time available resources of the ground cloud servers can include computing resources of all ground cloud servers.

[0065] S202: inputting the target computing task and the real-time resource state into a resource allocation model, and outputting a star cloud resource allocation strategy through the resource allocation model, wherein the resource allocation model includes a resource estimation module and a resource optimization module, the resource estimation module is configured to determine pre-occupied resources of the target computing task, and the resource optimization module is configured to determine the star cloud resource allocation strategy according to the real-time resource state and the pre-occupied resources.

[0066] S203: distributing the target computing task to the satellites and / or the ground cloud servers for execution based on the star cloud resource allocation strategy.

[0067] In the embodiments of the present application, the resource allocation model is configured to analyze the target computing task and the real-time resource state, and output a star cloud resource allocation strategy. The star cloud resource allocation strategy can include a strategy of splitting the target computing task into multiple sub-target computing tasks according to a target neural network level, and then distributing the multiple sub-target computing tasks to the satellites and / or the ground cloud servers for execution. In other embodiments, the star cloud resource allocation strategy can also include a strategy of distributing the target computing task to the satellites and / or the ground cloud servers for execution as a whole. It can be understood that, due to the differences in the target computing task, the different splitting manners of the target computing task, and the differences in the real-time resource state, there can be one or more star cloud resource pre-allocation strategies. The resource allocation model can filter out one or more strategies with the highest resource utilization rate from the multiple star cloud resource pre-allocation strategies as the star cloud resource allocation strategy.

[0068] In one embodiment, the distributing the target computing task to the satellites and / or the ground cloud servers for execution based on the star cloud resource allocation strategy specifically includes:

[0069] splitting the target computing task into multiple sub-target computing tasks according to a target neural network level, and then distributing the multiple sub-target computing tasks to the satellites and / or the ground cloud servers for execution.

[0070] In another embodiment, the distributing the target computing task to the satellites and / or the ground cloud servers for execution based on the star cloud resource allocation strategy specifically includes:

[0071] distributing the target computing task to the satellites and / or the ground cloud servers for execution as a whole.

[0072] Exemplarily, the target operation task includes at least one of an image inference task and an image recognition task.

[0073] In the embodiments of the present application, the resource allocation model includes a resource estimation module and a resource optimization module. In some embodiments, the resource estimation module is configured to determine pre-occupied resources of a target operation task. The pre-occupied resources of the target operation task can include pre-occupied resources of a target neural network if the target neural network executes the target operation task. In some specific embodiments, the resource estimation module is as shown in FIG. 3. After the satellite and / or the ground cloud server receives a target operation task, the satellite and / or the ground cloud server determines pre-occupied resources of the target operation task by using the resource estimation module.

[0074] In some embodiments, the determination of the pre-occupied resources of the target operation task includes:

[0075] S301: determining a target neural network corresponding to the target operation task according to the target operation task.

[0076] S303: determining pre-occupied resources of each layer of the target neural network when the target neural network executes the target operation task according to the real-time resource state.

[0077] S305: taking the pre-occupied resources of each layer of the target neural network as the pre-occupied resources of the target operation task.

[0078] In the embodiments of the present application, the resource estimation module can determine a target neural network corresponding to a target operation task according to the input target operation task. The resource estimation module can also determine pre-occupied resources of each layer of the target neural network when the target neural network executes the target operation task according to the real-time resource state. Finally, the resource estimation module takes the pre-occupied resources of each layer of the target neural network as the pre-occupied resources of the target operation task.

[0079] In some specific embodiments, the target neural network includes a performance analysis algorithm network based on a statistical regression model of simulation execution data. The target neural network can estimate execution time and other operation resources occupied by different DNN (Deep Neural Network) layers based on the real-time resource state of the satellite and / or the ground cloud server.

[0080] In the embodiments of the present application, the resource optimization module is configured to determine a satellite cloud resource allocation strategy according to the real-time resource state and the pre-occupied resources. In some embodiments, the determination of the satellite cloud resource allocation strategy according to the real-time resource state and the pre-occupied resources includes:

[0081] S401: determining Markov decision parameters according to the real-time resource state and the pre-occupied resources.

[0082] S403: Determine the nebula resource allocation strategy based on the Markov decision parameter and the deep deterministic policy gradient algorithm.

[0083] In the embodiments of the present application, the Markov decision process (MDP) is a mathematical model of sequential decision making, and the Markov decision parameter specifically includes state space parameters, action space parameters, and reward functions, etc. The nebula resource allocation problem such as reasoning calculation division and task offloading can be converted into a Markov decision process, and the nebula resource allocation strategy is determined according to the Markov decision parameter and the deep deterministic policy gradient algorithm (DDPG).

[0084] The application diagram of the nebula computing resource allocation method provided by the embodiments of the present application is shown in FIG. 4. After obtaining the current environment state in real time, that is, obtaining the target computing task and the real-time resource state, the pre-occupied resources such as reasoning delay and transmission delay are calculated by the resource estimation module in the resource allocation model, and then the algorithm reasoning calculation division and the offloading of the target computing task are performed based on the deep deterministic policy gradient network of the Markov decision parameter by the resource optimization module, and the actual allocation scheme is selected from multiple selectable resource allocation schemes as the nebula resource allocation strategy. The dashed lines in FIG. 4 respectively represent multiple selectable neural network division point schemes and selectable resource allocation schemes such as allocating resources to edge devices D1, D k , …D m .

[0085] The nebula computing resource allocation method provided by the embodiments of the present application inputs the target computing task and the real-time resource state into the resource allocation model, determines the pre-occupied resources by the resource estimation module, and outputs the nebula resource allocation strategy based on the real-time resource state and the pre-occupied resources by the resource optimization module under the condition of fully considering the dynamic resource environment, which can effectively improve the utilization rate of computing resources.

[0086] The implementation of determining the Markov decision parameter will be described below by the embodiments of the present application.

[0087] In some embodiments, the real-time resource state includes satellite real-time resource state and cloud server real-time resource state, the Markov decision parameter includes real-time state space parameters, and the determining the Markov decision parameter according to the real-time resource state and the pre-occupied resources includes:

[0088] S501: Obtain real-time space environment information and data output of each layer of the target neural network.

[0089] S503: Determine the real-time state space parameter according to the real-time space environment information, the data output quantity of each layer of the target neural network, and the real-time resource state.

[0090] In the embodiments of the present application, the state space parameter includes static parameters involved in the star cloud resource allocation process. The space environment information includes information transmission rate, electromagnetic interference state, satellite signal strength and other information in the space environment, wherein the information transmission rate specifically can include the communication rate between satellites and the communication rate between the satellite and the ground cloud server.

[0091] The target neural network can be used to perform a target operation task. After determining the target neural network and the parameters of each layer of the target neural network, the data output quantity of each layer of the target neural network can be determined. The data output quantity can include the output data size. It should be noted that the pre-occupied resources of each layer of the target neural network in the embodiments of the present application include the estimated results of the operation resources that each layer of the target neural network may occupy when performing the target operation task; and the data output quantity of each layer of the target neural network includes the output data size generated by each layer of the neural network after processing the input data, which usually includes the dimension (e.g., height, width, and channel number) and data quantity (byte number) of the tensor, etc. The data output quantity is also relatively fixed after the parameters of the target neural network are determined.

[0092] In some specific embodiments, the real-time state space parameter State can be represented as formula (1). State={S|S=(B,SCR,CCR,D)} (1)

[0093] In formula (1), State represents the state space, which is a set of all possible states; S is a specific state in State, which is defined as a four-tuple (B, SCR, CCR, D). Wherein, B represents the transmission rate of the real-time space environment information; SCR=(SCR1, SCR2,... SCR m ) is a sequence with m elements, representing the real-time operation resources of each cloud-native satellite in the current m satellite edge devices; CCR represents the real-time operation resources of the ground cloud server; D=(D input , D1, D2,... D n ) represents the original input data D input and the output data size of each layer of the target neural network.

[0094] The real-time state space parameter provided by the embodiments of the present application is a feasible implementation manner, and in other embodiments, the parameter can not be limited to the above-mentioned parameter. For example, if only the allocation of the target operation task among the satellites is considered, the real-time operation resource of the ground cloud server can not be included in the real-time state space parameter. In some other embodiments, in order to further improve the rationality of resource allocation, other static parameters such as satellite signal strength values can also be added. The above-mentioned determination manner of the real-time state space parameter should not be understood as a limitation of the present application.

[0095] According to the real-time space environment information, the data output of each layer of the target neural network, and the real-time resource state, the embodiments of the present application determine the real-time state space parameter, which can fully consider the real-time static environment information when the star cloud cooperatively executes the target operation task, and effectively improve the resource utilization rate and the rationality of resource allocation.

[0096] In some embodiments, the Markov decision parameter includes a real-time action space parameter, and the determination of the Markov decision parameter according to the real-time resource state and the pre-occupied resource includes:

[0097] S601: determining a real-time satellite set to be allocated according to the satellite real-time resource state.

[0098] S603: determining a division point set of the target neural network according to the data output of each layer of the target neural network, the real-time resource state, and the pre-occupied resource.

[0099] S605: determining the real-time action space parameter according to the division point set and the real-time satellite set to be allocated.

[0100] In the embodiments of the present application, the action space parameter includes the allocation action parameter involved in the star cloud resource allocation process, and the action space parameter includes all optional resource allocation strategies. According to the satellite real-time resource state, the satellite with operation resource margin can be determined as the real-time satellite set to be allocated. According to the data output of each layer of the target neural network, the real-time resource state, and the pre-occupied resource, the division point set of the target neural network can be determined. The division point set of the target neural network is a fine-grained calculation division of the target network level, which can accurately control the resource allocation and the execution of the target operation task.

[0101] The division of the target neural network includes the division of different layers of the target neural network to different computing nodes, such as the specific location of the satellite edge device or the ground cloud server for processing. By selecting these division points, the calculation task of the entire target neural network can be reasonably distributed on different computing resources to optimize the inference efficiency and resource utilization.

[0102] For example, in some embodiments, if the target neural network includes an input layer, a convolutional layer 1, a pooling layer 1, a convolutional layer 2, a pooling layer 2, and a fully connected layer, if P0 is selected as the division point, the input data can be processed on the satellite device to generate the output of the convolutional layer 1, and then the output data is transmitted to the ground cloud server for subsequent processing; if P1 is selected as the division point, it means that the input data, the convolutional layer 1, and the calculation of the pooling layer 1 are all completed on the satellite device, and the output of the pooling layer 1 is transmitted to the ground cloud server for further processing, and so on. In other embodiments, the split sub-tasks can also be unloaded to different satellite devices for execution according to different division points.

[0103] In some embodiments, the real-time action space parameter Action can be represented as formula (2). Action={A|A=(P,SAT)} (2)

[0104] In formula (2), Action represents an action space, which is a set of all possible actions; A is a specific action in Action, which is defined as a binary tuple (P, SAT). Wherein, P=(P0, P1,...P n ) represents a set of division points of the target neural network, and SAT=(SAT1, SAT2,...SAT m ) represents a set of edge devices that have sufficient resources and can perform inference computing tasks at the current time of receiving tasks, i.e., a set of real-time satellites to be allocated. The real-time action space parameter represents that the target computing task is divided in the target neural network through the set of division points, and then the completed divided tasks are unloaded to each satellite device for distributed processing. In other embodiments, the real-time action space parameter can also include a ground cloud server, and the completed divided tasks are unloaded to each satellite device and / or the ground cloud server for distributed processing.

[0105] In the embodiments of the present application, the set of division points (P0, P1,...P n ) of the target neural network is determined according to the data output of each layer of the target neural network, the real-time resource state, and the pre-occupied resources. The specific division point is dynamically adjusted according to the calculation complexity of each layer of the target neural network, the transmission time, and the real-time resource state of the satellite device and the ground cloud server. For each possible calculation division point P nThe division points (i.e., the boundaries of the DNN layers) and the corresponding target operation task offloading strategies are evaluated by a deep deterministic policy gradient deep reinforcement learning network. Through continuous interaction with the environment, the deep deterministic policy gradient network can autonomously learn and optimize the inference calculation division and task offloading strategy, determine the star cloud resource allocation strategy in real time, ensure adaptive selection of the optimal division point in the dynamically changing space environment, and ensure reasonable offloading and efficient execution of the inference calculation task among different satellite devices.

[0106] In the embodiments of the present application, the real-time action space parameters are determined according to the division point set and the real-time satellite set to be allocated. On the one hand, the target operation task can be dispersed to multiple calculation nodes, for example, a satellite device with limited resources processes a smaller layer, and a ground cloud server or a satellite device with sufficient resources processes a larger layer, which can effectively improve the utilization rate of operation resources and the efficiency of task operation. On the other hand, since the data amount output by each network layer is different, selecting a suitable division point can reduce the amount of data transmitted between calculation nodes, thereby reducing transmission delay and bandwidth occupation, and effectively reducing data transmission amount. Furthermore, by dynamically selecting a division point, the overall inference delay can be minimized according to the current calculation resource state and information transmission rate, and the inference efficiency can be improved.

[0107] It should be noted that the real-time action space parameters provided in the embodiments of the present application are a feasible implementation manner, and in other embodiments, the above-mentioned parameters can not be limited, for example, the satellite set to be allocated can also be set according to actual needs. In some other embodiments, in order to further improve the resource utilization rate, other action parameters such as the number of tasks to be allocated at the same time can also be added. The above-mentioned determination method of the real-time action space parameters should not be understood as a limitation of the present application.

[0108] The determination method of the star cloud resource allocation strategy will be described below through the embodiments of the present application.

[0109] In some embodiments, the determination of the star cloud resource allocation strategy based on the Markov decision parameter and the deep deterministic policy gradient algorithm comprises:

[0110] S701: determining a resource allocation strategy to be optimized according to the real-time state space parameter, the real-time action space parameter, and a preset reward factor.

[0111] S703: filtering the resource allocation strategy to be optimized according to the deep deterministic policy gradient algorithm to determine the star cloud resource allocation strategy.

[0112] In this embodiment, based on real-time state space parameters, real-time action space parameters, and a preset reward factor, the Markov decision process can be computed using the deep policy gradient in a deep reinforcement learning algorithm to determine the resource allocation strategy to be optimized. Then, by filtering the resource allocation strategies to be optimized, the nebula resource allocation strategy is determined.

[0113] In some embodiments, the preset reward factors include at least one of the following: inference latency reduction reward factor; energy efficiency reward factor; successful task execution reward factor; satellite resource consumption limit reward factor; cloud server load limit reward factor; satellite resource and cloud server resource balancing reward factor; data transmission volume limit reward factor between satellites; and fault tolerance reward factor for rapid response to environmental changes.

[0114] In some embodiments, the total reward function can be determined based on a preset reward factor, and then the resource allocation strategy to be optimized can be determined based on the total reward function. In some specific embodiments, the total reward function R can be expressed as equation (3):

[0115] Equation (3) includes the inference delay reduction reward factor r. dealy Considering the limited energy of satellites, the energy efficiency reward factor r energy And the reward factor r for successful task execution success The auxiliary reward factors include a satellite resource consumption limit reward factor r to avoid excessive consumption of edge computing resources. edge To avoid excessive load on ground-based cloud servers, the cloud server load limit reward factor r cloud To maintain a balance of computing resources between satellites and ground-based cloud servers, a reward factor r is used to balance satellite resources and cloud server resources. banlance To avoid excessive data transmission between devices in space, the satellite data transmission volume limit should be implemented. (Reward factor r) trans And the fault-tolerant reward factor r for rapid response to environmental changes fault Among them, the auxiliary reward factor is used to reward or punish when the corresponding task requirements are met, thereby improving the efficiency of system reasoning and the ability to handle unexpected events in the environment.

[0116] In equation (3), flag success This indicates that the target computation task has been successfully executed according to the nebula resource allocation strategy. normal and t represent the latency of completing the task using the default inference method and the latency of completing the task using the Nebula computing resource allocation method provided in this application embodiment, respectively. initial and e current These represent the satellite's initial energy at the start of the mission and its current real-time remaining energy, respectively. normal and qedge These represent the number of edge satellites used when performing inference in the default mode and the number of edge satellites used when performing inference using the nebula computing resource allocation method provided in this application embodiment, respectively. normal and l cloud These represent the computational load of the ground cloud server when performing inference using the default method and the computational load of the ground cloud server when performing inference using the nebula computing resource allocation method provided in this application embodiment, respectively. edge and l average Then, d represents the computational load of the edge satellite and the average load of all computational resources when performing inference using the nebula computing resource allocation method provided in this application embodiment. date The data transmission volume between various satellite devices in the space environment is represented by f, where f represents the number of times each device fails within a specific time period. c1, c2, ..., c8 are proportional coefficients that match each reward factor. Different adaptive values ​​are taken to adjust the relative size of the reward factors, thereby adjusting the contribution of different reward factors to the total reward. The total reward function R can include the sum of the above reward factors.

[0117] In some embodiments, the depth-determining policy gradient algorithm includes an actor network and a critic network, and the step of filtering the resource allocation strategy to be optimized and determining the nebula resource allocation strategy according to the depth-determining policy gradient algorithm includes:

[0118] S801: The actor network determines the resource allocation strategy with the highest action value among the resource allocation strategies to be optimized, and uses it as the Nebula resource allocation strategy, wherein the action value is determined according to the critic network.

[0119] In this embodiment, the deep deterministic policy gradient algorithm network structure includes an actor network for decision-making and a critic network for evaluating the decisions. Specifically, the actor network determines the resource allocation strategy with the highest action value among the resource allocation strategies to be optimized, that is, the resource allocation strategy with the highest total reward function, as the nebula resource allocation strategy. The action value is determined by the critic network. The deep deterministic policy gradient algorithm features real-time dynamic learning and can adapt to the changing space environment.

[0120] Based on the deep understanding of the dynamic changes of operation resources in the space environment and the powerful ability of deep reinforcement learning in dealing with complex decision-making problems, the star cloud operation resource allocation method provided by the embodiment has the characteristics of dynamically adapting to the real-time resource state. First, the resource monitoring module is used to obtain the computing resources, load conditions and transmission rates of the satellite edge device and the ground cloud server in real time, so as to accurately determine the real-time resource state. Then, through the deep deterministic policy gradient algorithm in deep reinforcement learning, combined with Markov decision process (MDP), the reasoning calculation division and task offloading strategy in the dynamic environment are optimized, and the star cloud resource allocation strategy is determined.

[0121] In other embodiments, the deep deterministic policy gradient algorithm further includes a target network, the target network including a target actor network and a target critic network, the target actor network having the same structure as the actor network, and the target critic network having the same structure as the critic network.

[0122] In the deep deterministic policy gradient algorithm, the design of the reward function directly affects the updating process of the Actor and Critic networks. In some specific embodiments, the reasoning process of the deep deterministic policy gradient algorithm is as follows:

[0123] (1) Environment interaction: at each time step: the actor network selects an action space parameter a under the real-time state space parameter s, determines a new state space parameter s' after executing the action a, and determines the real-time reward function r = R(s, a, s').

[0124] (2) Store experience: store the experience data (s, a, r, s') in the environment interaction process into the experience replay buffer of the deep deterministic policy gradient.

[0125] (3) Batch update: randomly sample a batch of samples (s i , a i , r i , s i ′) from the experience replay buffer for training.

[0126] (4) Critic network update: the critic network determines the total reward function Q(s, a) of the current action.

[0127] The target Q value y i is determined by the target critic network according to formula (4): y i = r i + γQ′(s i ′, π′(s i ′)) (4)

[0128] In formula (4), γ is a discount factor, and Q' and π' represent policy functions corresponding to the target critic network and the target actor network, respectively. Using the target Q value, the critic network parameters are updated by minimizing the TD error.

[0129] (5) Actor network update: the actor network updates its policy according to the gradient information provided by the critic network, and uses the gradient ascent method to update the actor network parameters, so that the selected action can maximize the total reward function Q value of the critic network.

[0130] (6) Target network soft update: the target network parameters slowly converge to the corresponding actor network and critic network parameters to prevent violent oscillation during training.

[0131] In the embodiments of the present application, the deep deterministic policy gradient algorithm is based on the actor-critic architecture, which selects the optimal action through the actor network, evaluates the action value through the critic network, and ensures the stability and convergence of the training through the slow update of the target network. Through the monitored real-time resource state, the inference calculation and task offloading strategy are dynamically adjusted to cope with the changing external environment, ensuring efficient task execution under different resource states. Not only does it improve task execution efficiency, minimize inference latency and transmission time, but also reduces resource waste and energy consumption, prolonging the service life of satellite devices.

[0132] The star cloud computing resource allocation method provided by the embodiments of the present application has high adaptability and flexibility, and can adaptively cope with the dynamic changes of resource states in space environment, quickly respond to changes in resource states, avoid task failure caused by resource shortage or overload, and improve the success rate of target computing task execution. In addition, the deep deterministic policy gradient algorithm can find the optimal policy in high-dimensional state space and continuous action space, improving the accuracy and reliability of star cloud resource allocation and enhancing the robustness.

[0133] The star cloud computing resource allocation method provided by the embodiments of the present application outputs an approximately optimal star cloud resource allocation strategy for inference calculation division and task offloading selection through a resource allocation model. According to the strategy, as shown in FIG. 5, a specific embodiment divides the inference calculation task according to the division point, and assigns part of the neural network layer task to the satellite for execution and returns the intermediate output data to the ground cloud server, while the other part of the neural network layer task is executed by the ground cloud server, which can effectively improve resource utilization and task execution efficiency.

[0134] It should be understood that although each step in the flowchart involved in the embodiments described above is shown in sequence according to the direction of the arrow, these steps are not necessarily executed in the order indicated by the arrow. Unless explicitly stated herein, there is no strict order limitation for the execution of these steps, and these steps can be executed in other orders. Moreover, at least some of the steps in the flowchart involved in the embodiments described above can include multiple steps or multiple stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily sequential, but can be alternately or alternately executed with at least part of other steps or steps or stages in other steps.

[0135] In the embodiments of the present application, the resource estimation module, the resource optimization module, the resource monitoring module and the storage module can each be one or more processors, controllers or chips with a communication interface capable of implementing a communication protocol, and can also include a memory and related interfaces, a system transmission bus, etc. if necessary; the processor, controller or chip executes program-related code to implement the corresponding functions.

[0136] Based on the same inventive concept, the embodiments of the present application also provide a star cloud computing resource allocation device 900 for implementing the star cloud computing resource allocation method described above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme described in the above method, so the specific limitations in one or more star cloud computing resource allocation device 900 embodiments provided below can refer to the limitations of the star cloud computing resource allocation method described above, which will not be repeated here.

[0137] In one embodiment, as shown in FIG. 6, a star cloud computing resource allocation device 900 is provided, which includes:

[0138] The data acquisition module 901 is configured to acquire a target computing task and a real-time resource state.

[0139] The resource allocation module 902 is configured to input the target computing task and the real-time resource state into a resource allocation model, and output a star cloud resource allocation strategy through the resource allocation model, wherein the resource allocation model includes a resource estimation module and a resource optimization module. The resource estimation module is configured to determine a pre-occupied resource of the target computing task. The resource optimization module is configured to determine the star cloud resource allocation strategy according to the real-time resource state and the pre-occupied resource.

[0140] In some embodiments, the resource allocation module 902 is further configured to determine a target neural network corresponding to the target computing task according to the target computing task; determine pre-occupied resources of each layer of the target neural network when the target neural network executes the target computing task according to the real-time resource state; and determine the pre-occupied resources of each layer of the target neural network as pre-occupied resources of the target computing task.

[0141] In some embodiments, the resource allocation module 902 is further configured to determine Markov decision parameters according to the real-time resource state and the pre-occupied resources; and determine the nebula resource allocation strategy based on the Markov decision parameters and a deep deterministic policy gradient algorithm.

[0142] In some embodiments, the real-time resource state includes a satellite real-time resource state and a cloud server real-time resource state, the Markov decision parameters include real-time state space parameters, and the resource allocation module 902 is further configured to acquire real-time space environment information and data output quantities of each layer of the target neural network; and determine the real-time state space parameters according to the real-time space environment information, the data output quantities of each layer of the target neural network, and the real-time resource state.

[0143] In some embodiments, the Markov decision parameters include real-time action space parameters, and the resource allocation module 902 is further configured to determine a real-time satellite set to be allocated according to the satellite real-time resource state; determine a partition point set of the target neural network according to the data output quantities of each layer of the target neural network, the real-time resource state, and the pre-occupied resources; and determine the real-time action space parameters according to the partition point set and the real-time satellite set to be allocated.

[0144] In some embodiments, the resource allocation module 902 is further configured to determine a resource allocation strategy to be optimized according to the real-time state space parameters, the real-time action space parameters, and a preset reward factor; and determine the nebula resource allocation strategy by screening the resource allocation strategy to be optimized according to the deep deterministic policy gradient algorithm.

[0145] In some embodiments, the preset reward factor includes at least one of the following: an inference time delay shortening reward factor; an energy efficiency reward factor; a successful task execution reward factor; a satellite resource consumption limitation reward factor; a cloud server load limitation reward factor; a satellite resource and cloud server resource balance reward factor; a satellite-to-satellite data transmission volume limitation reward factor; and a fault tolerance reward factor for fast response to environmental changes.

[0146] In some embodiments, the deep deterministic policy gradient algorithm comprises an actor network and a critic network, and the resource allocation module 902 is further configured to determine, by the actor network, a resource allocation strategy with the highest action value in the to-be-optimized resource allocation strategies as the nebula resource allocation strategy, wherein the action value is determined according to the critic network.

[0147] The modules in the nebula computing resource allocation apparatus 900, including the data acquisition module 901 and the resource allocation module 902, can be implemented by software, hardware, or a combination thereof, in whole or in part. The modules can be embedded in or independent of a processor in a computer device in hardware form, or stored in a memory in the computer device in software form, so as to be called and executed by the processor to perform the operations corresponding to the modules.

[0148] In one embodiment, a computer device, which can be a terminal, is provided, and an internal structure diagram of the computer device can be as shown in FIG. 7. The computer device includes a processor, a memory, a communication interface, a display screen, and an input device connected through a system bus. The processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for running the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is configured to perform wired or wireless communication with an external terminal. The wireless communication can be achieved by WIFI, mobile cellular network, NFC (near field communication), or other technologies. The computer program is executed by the processor to implement a nebula computing resource allocation method. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer overlaid on the display screen, or a key, trackball, or touchpad arranged on the shell of the computer device, or an external keyboard, touchpad, or mouse, etc.

[0149] Those skilled in the art can understand that the structure shown in FIG. 7 is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. Specifically, the computer device can include more or fewer components than those shown in the figure, or combine certain components, or have a different component arrangement.

[0150] In one embodiment, a computer device is provided, including a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of the nebula computing resource allocation method according to any one of the above embodiments.

[0151] In one embodiment, a computer readable storage medium is provided, and the computer readable storage medium has stored thereon a computer program. The computer program is executed by a processor to implement the steps of the nebula computing resource allocation method according to any one of the above embodiments.

[0152] In one embodiment, a computer program product is provided, and the computer program product includes a computer program. The computer program is executed by a processor to implement the steps of the nebula computing resource allocation method according to any one of the above embodiments.

[0153] It should be noted that the user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties.

[0154] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when the computer program is executed, the processes of the above-mentioned embodiments of the methods can be included. Any reference to memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (Read-Only Memory, ROM), magnetic tape, floppy disk, flash memory, optical storage, high-density embedded non-volatile memory, resistive memory (ReRAM), magnetoresistive random access memory (Magnetoresistive Random Access Memory, MRAM), ferroelectric memory (Ferroelectric Random Access Memory, FRAM), phase change memory (Phase Change Memory, PCM), graphene memory, etc. Volatile memory can include random access memory (Random Access Memory, RAM) or external cache memory, etc. As an illustration but not limitation, the RAM can be in various forms, such as static random access memory (Static Random Access Memory, SRAM) or dynamic random access memory (Dynamic Random Access Memory, DRAM), etc. The database involved in the embodiments provided in the present application can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a block chain, etc., without being limited thereto. The processor involved in the embodiments provided in the present application can be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., without being limited thereto.

[0155] Any combination of the technical features of the above embodiments can be made. In order to make the description simple, all possible combinations of the technical features in the above embodiments are not described, however, as long as the combination of the technical features does not exist contradictory, it should be considered as the scope of the present application.

[0156] The above embodiments only express several implementation manners of the present application, and the description is more specific and detailed, but it should not be understood as a limitation on the scope of the patent of the present application. It should be pointed out that for ordinary skilled in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which are within the scope of protection of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. A nebular computing resource allocation method, characterized by, The method comprises: obtaining a target operation task and a real-time resource state; inputting the target operation task and the real-time resource state into a resource allocation model, and outputting a nebula resource allocation strategy through the resource allocation model, wherein the resource allocation model comprises a resource estimation module and a resource optimization module, the resource estimation module is used to determine pre-occupied resources of the target operation task, and the resource optimization module is used to determine the nebula resource allocation strategy according to the real-time resource state and the pre-occupied resources; based on the nebula resource allocation strategy, the target operation task is allocated to a satellite and / or a ground cloud server for execution.

2. The method of claim 1, wherein, The determination of the pre-occupied resources of the target operation task comprises: determining a target neural network corresponding to the target operation task according to the target operation task; determining pre-occupied resources of each layer of the target neural network when the target neural network executes the target operation task according to the real-time resource state; taking the pre-occupied resources of each layer of the target neural network as the pre-occupied resources of the target operation task.

3. The method of claim 2, wherein, The determination of the nebula resource allocation strategy according to the real-time resource state and the pre-occupied resources comprises: determining Markov decision parameters according to the real-time resource state and the pre-occupied resources; determining the nebula resource allocation strategy based on the Markov decision parameters and a deep deterministic policy gradient algorithm.

4. The method of claim 3, wherein, The real-time resource state comprises a satellite real-time resource state and a cloud server real-time resource state, the Markov decision parameters comprise real-time state space parameters, and the determination of the Markov decision parameters according to the real-time resource state and the pre-occupied resources comprises: obtaining real-time space environment information and data output quantities of each layer of the target neural network; determining the real-time state space parameters according to the real-time space environment information, the data output quantities of each layer of the target neural network and the real-time resource state.

5. The method of claim 4, wherein, The Markov decision parameters comprise real-time action space parameters, and the determination of the Markov decision parameters according to the real-time resource state and the pre-occupied resources comprises: determining a real-time satellite set to be allocated according to the satellite real-time resource state; determining a partition point set of the target neural network according to the data output quantities of each layer of the target neural network, the real-time resource state and the pre-occupied resources; determining the real-time action space parameters according to the partition point set and the real-time satellite set to be allocated.

6. The method of claim 5, wherein, The determination of the nebula resource allocation strategy based on the Markov decision parameters and the deep deterministic policy gradient algorithm comprises: determining a to-be-optimized resource allocation strategy according to the real-time state space parameters, the real-time action space parameters and a preset reward factor; determining the nebula resource allocation strategy according to the deep deterministic policy gradient algorithm screening the to-be-optimized resource allocation strategy.

7. The method of claim 6, wherein, The preset reward factor comprises at least one of the following: an inference time delay shortening reward factor; an energy efficiency reward factor; a successful task execution reward factor; a satellite resource consumption limitation reward factor; a cloud server load limitation reward factor; A satellite resource and cloud server resource balanced reward factor; A data transmission volume between satellites limited reward factor; A fault tolerance reward factor for fast response to environmental changes.

8. The method of claim 6, wherein, The deep deterministic policy gradient algorithm includes an actor network and a critic network, the resource allocation strategy to be optimized is filtered according to the deep deterministic policy gradient algorithm, and the star cloud resource allocation strategy is determined. The actor network determines a resource allocation strategy with the highest action value in the resource allocation strategy to be optimized as the star cloud resource allocation strategy, wherein the action value is determined according to the critic network.

9. A Nebula computing resource allocation apparatus, characterized by comprising: The device comprises: a data acquisition module configured to acquire a target operation task and a real-time resource state; a resource allocation module configured to input the target operation task and the real-time resource state into a resource allocation model, and output a star cloud resource allocation strategy through the resource allocation model, wherein the resource allocation model comprises a resource estimation module and a resource optimization module, the resource estimation module is configured to determine pre-occupied resources of the target operation task, and the resource optimization module is configured to determine the star cloud resource allocation strategy according to the real-time resource state and the pre-occupied resources; the resource allocation module is further configured to allocate the target operation task to a satellite and / or a ground cloud server for execution based on the star cloud resource allocation strategy. 10.A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the computer device is configured to perform the method according to any one of claims 1-9. The processor executes the computer program to realize the steps of the method in any one of claims 1 to 8.

11. A computer readable storage medium having stored thereon a computer program, characterized in that The computer program is executed by the processor to realize the steps of the method in any one of claims 1 to 8.

Images