LEO multi-model parameter dynamic reasoning strategy

Abstract

The invention provides an LEO multi-model parameter dynamic reasoning strategy aiming at the characteristics that low earth orbit (LEO) is limited in calculation, limited in energy, unstable in supply and variable in task demand, so that the reasoning efficiency is optimized, and the task response capability of a system is improved. The strategy is based on a Markov Decision Process (MDP), and in combination with deep reinforcement learning (DQN), dynamic adaptation of parameters of a satellite inference model is realized. By introducing a multi-model switching and parameter adjusting mechanism, the method achieves effective balance among reasoning precision, calculation energy consumption and response time delay. In a simulation experiment, compared with a traditional fixed mode, the strategy provided by the invention has the advantages that the QoE (Quality of Experience) is improved by about 25.2%, compared with a random mode, the QoE is improved by about 63.9%, the reasoning precision reaches 76.5%, and meanwhile, the calculation energy consumption and the response time delay are respectively maintained at 2.3 J and 2.1 ms.

Description

Technical Field

[0001] The present invention relates to the field of low-orbit satellite network communications, and in particular to a LEO multi-model parameter dynamic reasoning strategy. Background Art

[0002] Low-orbit satellite networks (LEO Satellite Networks, LSNs) provide strong support for the realization of 6G global seamless coverage. At the same time, artificial intelligence (AI) is widely used in many fields.

[0003] In this context, LSNs are gradually transitioning from their initial basic communication functions to traditional data collection, and eventually to more intelligent real-time data processing and analysis. This transition relies on deploying inference models on satellites, enabling them to process collected data in a timely manner in orbit. However, due to the various limitations of LSNs and their dynamically changing topological structures, achieving efficient, accurate, and adaptive inference has become an extremely challenging task.

[0004] Specifically, LSNs carry inference models for real-time computing, and they face the following problems: First, computing resources are limited and cannot support complex deep learning models. Second, energy is limited and the supply is unstable. LSNs alternately pass through the sunlit area and the shadow area. Their energy is heavily dependent on solar energy and is accompanied by randomness. Third, fixed configurations are difficult to cope with dynamic mission requirements. With the increase in mission complexity and changes in resource constraints, fixed configurations cannot flexibly respond to the diverse mission requirements that arise during satellite operation.

[0005] To solve the above problems, the industry has conducted extensive research. In response to the problem of limited computing, some literature deploys deep neural networks for satellite earth observation and uses distributed edge computing to address the limitations of satellite computing capabilities. Some literature proposes a lightweight inference solution. This solution reduces the depth of the neural network and the network parameters of the model, while ensuring accuracy, reducing the inference time to 3.26ms and achieving an accuracy of 93.56%.

[0006] In response to the problem of energy limitation and unstable supply, a paper proposed a lightweight image data processing architecture, which maintains spatial information by adding position encoding and uses a multi-head self-attention mechanism to capture global features, significantly reducing computing energy consumption while maintaining high reasoning accuracy. A paper proposed a high-efficiency satellite image recognition framework, which achieves ultra-low power target recognition by integrating a lightweight adapter and taking advantage of pre-trained models.

[0007] In order to solve the problem that fixed configurations are difficult to cope with dynamic task requirements, some literature uses deep reinforcement learning to optimize channel allocation for energy efficiency in satellite IoT, and introduces an adaptive resource management mechanism to alleviate the problem of limited satellite computing power and power resources. Some literature proposes an adaptive streaming framework for real-time video analysis (LVA) in a (Low Earth Orbit, LEO) satellite network environment, which dynamically and adaptively adjusts the network and video configuration based on the Transformer predictor and optimizer.

[0008] However, existing research mainly focuses on static configuration, ignoring the adaptive adjustment capability under dynamic tasks and resource constraints. In addition, existing research lacks comprehensive consideration of user satisfaction. Therefore, achieving this goal faces the following challenges: it is necessary to reduce computing energy consumption and response latency while ensuring reasoning accuracy to balance model accuracy and resource consumption; the model must adapt to the dynamic changes of resources and tasks; there is a lack of a user satisfaction indicator that comprehensively considers accuracy and latency. Therefore, in order to maintain efficient reasoning capabilities in a dynamic environment, it is urgent to dynamically adjust the parameter configuration of the reasoning model according to real-time conditions to improve the adaptability of the model. Summary of the invention

[0009] In response to the technical problems pointed out in the above background technology, the present invention proposes a LEO multi-model parameter dynamic reasoning strategy, which is based on Markov Decision Process (MDP) and combined with deep reinforcement learning (Deep Q Network, DQN) to achieve dynamic adaptation of satellite reasoning model parameters to optimize reasoning efficiency and improve the system's mission responsiveness.

[0010] The technical solution for achieving the purpose of the present invention is:

[0011] A LEO multi-model parameter dynamic inference strategy, comprising:

[0012] (1) Building a satellite network model

[0013] The image inference model is used to capture images and perform inference tasks; the orbit equation of the LEO satellite is expressed as:

[0014] r(t)=(R+h)[1-e·cos(α(t))] (1)

[0015] Where R is the radius of the Earth, h is the orbital altitude, e is the orbital eccentricity, and α(t) is the true anomaly, which is the angle between the current position of the satellite and the perigee;

[0016] The orbital period of a LEO satellite is:

[0017]

[0018] Where G is the gravitational constant, M e is the mass of the Earth;

[0019] (2) Building a satellite energy model

[0020] Assume that the instantaneous solar radiation power received by the LEO satellite solar panel per square meter is P r (t), which is defined as follows:

[0021]

[0022] Among them, P max is the maximum power received by the solar panel per square meter, α is the proportion of the illuminated area during the orbital period, and σ controls the width of the radiation intensity distribution;

[0023] Assume the initial battery energy of the satellite is The maximum battery capacity is The minimum tolerable energy level is

[0024] The orbital period is divided into N equal time periods, each of which is separated by a time interval of Δt; the time period sequence is represented by T = {t1,···,t i ,···,t N}, where t i Represents any time period;

[0025] If t i In the sunlit area, that is, 1≤i≤αN, the energy captured by the LEO satellite is:

[0026]

[0027] Where η is the solar energy conversion efficiency and D is the area of ​​the solar panel;

[0028] A fixed battery charging energy mechanism is used, that is, the battery is replenished with the same average energy in each time period; the battery i The charging energy during this period is:

[0029]

[0030] In t i The available energy for image inference during this period is:

[0031]

[0032] When the satellite is in the shadow area, that is, αN<i≤N, the cyclic average energy mechanism is adopted; after deducting the inference energy, the average value of the next time period is recalculated; this iterative process will continue until the last time period in the shadow area;

[0033] In t i The available energy for image inference during this period is:

[0034]

[0035] Among them, E v represents the energy consumption in any time period from αN to i-1;

[0036] The sequence of available energy for image inference during the entire orbital period is

[0037] (3) Building a satellite inference model

[0038] The set of CNN models with different parameters for the deployment of M LEO satellites is Where m j Represents an arbitrary CNN model; at each time slot t i An optimal model will be pre-selected, expressed as

[0039] For the best model selected The number of neural network layers is L, l represents any layer of the neural network (l = 1, 2, ..., L);

[0040] Time slot t i Using model m j The performance set for reasoning about the kth image is:

[0041] P I ={p ijk ,t ijk ,y ijk} , 1≤i≤N,1≤j≤M (8)

[0042] Among them, p ijk is the inference power of the image, t ijk is the inference time of the image, y ijk is the inference label of the picture;

[0043] Time slot t i Select model m j The inference energy consumption is:

[0044]

[0045] If the inference label y ijk With the actual label xijk If they are the same, it is counted as a correct inference; then in time slot t i Select the inference model m i The inference accuracy is:

[0046]

[0047] Among them, 1(x) is an indicator function, indicating whether the condition is met, 1 if it is met, and 0 otherwise;

[0048] (4) Define tasks in different time slots t i The distribution function of time slot t i The number of images to be processed follows a Poisson distribution as follows:

[0049]

[0050] Among them, n i Indicates time slot t i The number of pictures, λ i Indicates time slot t i The average number of pictures;

[0051] Time slot t i The set of images to be processed in is represented as ik is the time slot t i Any picture number of ;

[0052] (5) Define users in different time slots t i Relative preference for inference accuracy and inference latency:

[0053] ω1(t i )+ω2(t i )=1 (12)

[0054] Among them, ω1(t i ) represents the weight of inference accuracy, ω2(t i ) represents the weight of the inference delay;

[0055] User preference function and the number of images n i When the number of pictures is larger, users pay more attention to the inference delay; when the number of pictures is smaller, users pay more attention to the inference accuracy. By defining the weight of the inference accuracy in different time slots, the weight of the inference delay can be calculated by formula (13).

[0056]

[0057] Among them, ρ is a tuning parameter, which indicates the sensitivity of preference to the number of pictures;

[0058] Adopting QoE i(Quality of Experience) indicators include the following two key factors: inference accuracy and inference latency;

[0059]

[0060] Among them, ω1(t i )+ω2(t i )=1 means that in each time slot t i In terms of the user's relative preference for inference accuracy and inference latency;

[0061] Each t i The user needs are different. In order to maximize the i User Quality of Experience (QoE) i ,The number of network layers and batch size need to be dynamically adjusted to meet different requirements;

[0062] Maximize each t i User satisfaction QoE i ; The inference energy consumption does not exceed the available energy of the satellite, and the number of network layers and batch size are within the feasible domain:

[0063]

[0064] L min ≤L i ≤L max

[0065] B min ≤B i ≤B max (19)

[0066] (6) Sunlight Area Modeling: Deterministic Markov Decision Process (Deterministic MDP)

[0067] The energy state transfer function is used to replace the energy state transfer probability. The deterministic Markov decision process consists of a tuple (S, A, T, r, γ);

[0068] is a state space where the state elements Indicates time slot t i Available energy of satellite, state element I i Indicates time slot t i The distribution of tasks;

[0069] is the action space, which is the selection object of the adaptive strategy;

[0070] T={(T i E ,T iI )} is the state transfer function, where T i E represents the energy state transfer function, T i I Represents the task state transfer function;

[0071] The energy state transfer function is:

[0072]

[0073] (7) Modeling of Earth's Shadow Region: Markov Decision Process (MDP)

[0074] The Markov decision process consists of the tuple (S, A, P, r, γ), is a state space where the state elements Indicates time slot t i Available energy of satellite, state element I i Indicates time slot t i The distribution of tasks;

[0075] is the action space, which is the selection object of the adaptive strategy;

[0076] P={(P i E ,P i I )} is the state transition probability, where P i E represents the energy state transition probability, P i I represents the task state transition probability; the state transition probability describes the system from state s at a certain moment t Transfer to the next state s t+1 The probability of

[0077] The energy state transition probability is:

[0078]

[0079] The sum of the probabilities of each row is 1. i E (s n →s n ) is defined as follows:

[0080]

[0081] The task state transition probability is:

[0082]

[0083] r(ti ,m j ,l) is the reward function, which means that in time slot t i The CNN model m is adopted j The reward obtained after the l-th layer of the neural network is as follows:

[0084]

[0085] Among them, δ1 is the weight coefficient of accuracy energy efficiency, and δ2 is the deviation coefficient of minimum accuracy.

[0086] γ is the discount factor, which represents the discount rate of long-term rewards and is set between 0 and 1. The higher the setting, the better the expected performance of long-term planning.

[0087] Furthermore,

[0088] In order to quantify the accuracy and energy consumption, the satellite reasoning model in step (3) defines an accuracy-energy efficiency (AEE) index, which represents the reasoning accuracy per unit energy consumption, in units of accuracy per joule (Accuracy / Joule); this index is used in the initial decision model in each time slot:

[0089]

[0090] After the initial model selection, each time slot needs to decide the optimal number of network layers corresponding to its model; the definition steps are as follows:

[0091] First, define the relationship between the inference accuracy A and the number of neural network layers l:

[0092]

[0093] Among them, α1 and β1 are model parameters, which determine the growth rate and initial accuracy; α2 and β2 determine the maximum value of accuracy and the shape of decline, l mid is the number of layers where the accuracy reaches the peak;

[0094] Secondly, define the relationship between the inference delay and the number of neural network layers:

[0095]

[0096] Where T is the inference delay (unit: s), which represents the time superposition of the model from input to output across all neural network layers; P eff is the effective computing performance of the hardware (unit: FLOPs / s); L conv is the number of standard convolutional layers (conv); S out,n is the spatial size of the output feature map of the nth conv layer; F conv,nis the product of the number of input and output channels of the nth conv layer; K n is the convolution kernel size of the nth conv layer; L dw is the number of depthwise separable convolutional layers (dw); S out,m The spatial size of the feature map output by the mth dw layer; C in,m is the number of input channels of the mth dw layer; F dw,m The product of the number of input and output channels of the mth dw layer; K m The convolution kernel size of the mth dw convolution layer; L fc is the number of fully connected layers (fc); F fc,p is the product of the number of input and output nodes of the pth fc layer;

[0097] Finally, the relationship between inference energy consumption and the number of neural network layers is defined as:

[0098]

[0099] Where E is the inference energy consumption (unit: Joule); L is the number of layers of the neural network; e c is the energy consumption per floating point operation (unit: Joule / FLOP); e t is the energy consumption per byte of data access (unit: Joule / Byte); B d is the number of bytes of each data (FP32, FP16, INT8); S out,k is the spatial size of the k-th layer output feature map; γ k is the calculation coefficient of the kth layer; W k is the number of weight parameters of the kth layer; A in,k and A out,k is the number of input and output activation elements of the kth layer.

[0100] Compared with the prior art, the main contributions of the present invention are as follows:

[0101] First, a user satisfaction index based on QoE is proposed, which comprehensively considers reasoning accuracy and latency. Secondly, an adaptive reasoning framework based on MDP is constructed to achieve dynamic adjustment of parameters, namely the number of network layers. Finally, based on the existing reasoning framework, the strategy proposed in this invention improves the total QoE index by about 63.9% compared with the random mode and by about 25.2% compared with the fixed mode. Its reasoning accuracy reaches 76.5%, the highest among all strategies, and while maintaining the reasoning delay at 2.1ms, it controls the computing energy consumption at 2.3J, reflecting good comprehensive performance advantages and resource efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0102] Figure 1A satellite network model diagram according to an embodiment of the present invention;

[0103] Figure 2 A satellite energy conversion diagram according to an embodiment of the present invention;

[0104] Figure 3 A satellite reasoning model diagram according to an embodiment of the present invention;

[0105] Figure 4 A schematic diagram of determining the number of network layers of a CNN model during the satellite operation phase according to an embodiment of the present invention;

[0106] Figure 5 This is a convergence trend diagram of the total reward during the DQN training process of an embodiment of the present invention;

[0107] Figure 6 A statistical diagram of the selection distribution of the number of network layers of the model according to an embodiment of the present invention;

[0108] Figure 7 This is a performance diagram of a comparative benchmark strategy according to an embodiment of the present invention;

[0109] Figure 8 It is a schematic diagram of convergence analysis and robustness test of an embodiment of the present invention. DETAILED DESCRIPTION

[0110] The present invention is further described below in conjunction with the accompanying drawings and embodiments, but the contents of the present invention are not limited thereto.

[0111] Example:

[0112] A LEO multi-model parameter dynamic inference strategy, comprising:

[0113] 1. Building a system model

[0114] 1. Build a satellite network model

[0115] like Figure 1 As shown in FIG. 1 , a LEO satellite with multiple image inference models deployed alternately passes through the sunlit area and the earth's shadow area in its orbit. During operation, it uses the image inference models to capture images and perform inference tasks.

[0116] The orbit equation of a LEO satellite is expressed as:

[0117] r(t)=(R+h)[1-e·cos(α(t))] (1)

[0118] Among them, R is the radius of the earth, h is the orbit altitude, e is the orbit eccentricity, and α(t) is the true anomaly, that is, the angle between the current position of the satellite and the perigee.

[0119] The orbital period of a LEO satellite is:

[0120]

[0121] Where G is the gravitational constant, M e The mass of the earth.

[0122] 2. Building a satellite energy model

[0123] Figure 2 The different energy sources of satellites in the sunlit area and the shadow area, as well as their respective energy distribution mechanisms, are described. In the sunlit area, solar radiation energy is used as the energy source. In the shadow area, solar radiation energy is zero, and the satellite relies on its battery as energy.

[0124] The satellite is in the sunlit area. Assume that the instantaneous solar radiation power received by the LEO satellite solar panel per square meter is P r (t), which is defined as follows:

[0125]

[0126] Among them, P max is the maximum power received by the solar panel per square meter, α is the proportion of the illuminated area during the orbital period, and σ controls the width of the radiation intensity distribution.

[0127] Assume the initial battery energy of the satellite is The maximum battery capacity is The minimum tolerable energy level is Before entering the shadow region, LEO satellites must charge their batteries to To ensure there is enough energy available.

[0128] The orbital period is divided into N equal time periods, each of which is separated by a time interval of Δt. The time period sequence is represented by T = {t1,···,t i ,···,t N}, where t i Represents any time period.

[0129] If t i In the sunlit area, that is, 1≤i≤αN, the energy captured by the LEO satellite is:

[0130]

[0131] Where η is the solar energy conversion efficiency and D is the area of ​​the solar panel.

[0132] A fixed battery charging energy mechanism is used, that is, the same average energy is added to the battery in each time period. i The charging energy during this period is:

[0133]

[0134] In t i The available energy for image inference during this period is:

[0135]

[0136] When the satellite is in the shadow area, that is, αN<i≤N, the cyclic average energy mechanism is used. Initially, the energy of the time period is averaged to determine the available energy for image inference. After deducting the inference energy, the average value of the next time period is recalculated. This iterative process will continue until the last time period in the shadow area.

[0137] In t i The available energy for image inference during this period is:

[0138]

[0139] Among them, E v Represents the energy consumption of any time period from αN to i-1.

[0140] The sequence of available energy for image inference during the entire orbital period is

[0141] 3. Build a satellite inference model

[0142] like Figure 3 As shown in Figure 2, the number of LEO satellite deployments with different parameters of CNN models is M, and the set is Where m j Represents an arbitrary CNN model. In each time slot t i An optimal model will be pre-selected, expressed as

[0143] For the best model selected The number of neural network layers is L, and l represents any layer of the neural network (l=1,2,...,L).

[0144] Time slot t i Using model m j The performance set for reasoning about the kth image is:

[0145] P I ={p ijk ,t ijk ,y ijk} , 1≤i≤N,1≤j≤M (8)

[0146] Among them, p ijk is the inference power of the image, t ijk is the inference time of the image, y ijk is the inferred label of the image.

[0147] Time slot t i Select model m j The inference energy consumption is:

[0148]

[0149] If the inference label y ijk With the actual label x ijk If they are the same, it is counted as a correct inference. i Select the inference model m i The inference accuracy is:

[0150]

[0151] Among them, 1(x) is an indicator function, indicating whether the condition is met, 1 if it is met, and 0 otherwise.

[0152] To quantify accuracy and energy consumption, we first define the accuracy-energy efficiency (AEE) index, which represents the inference accuracy per unit of energy consumption, with the unit of accuracy per joule (Accuracy / Joule). This metric is used in the initial decision model at each time slot as follows:

[0153]

[0154] After the initial model selection, each time slot needs to decide the optimal number of network layers corresponding to its model, and the following relational expression is defined.

[0155] First, define the relationship between the inference accuracy A and the number of neural network layers l:

[0156]

[0157] Among them, α1 and β1 are model parameters, which determine the growth rate and initial accuracy. α2 and β2 determine the maximum value of accuracy and the shape of decline. mid is the number of layers where the accuracy reaches its peak.

[0158] Secondly, define the relationship between the inference delay and the number of neural network layers:

[0159]

[0160] Where T is the inference delay (unit: s), which represents the time superposition of the model from input to output across all neural network layers; P eff is the effective computing performance of the hardware (unit: FLOPs / s); L conv is the number of standard convolutional layers (conv); S out,nis the spatial size of the output feature map of the nth conv layer; F conv,n is the product of the number of input and output channels of the nth conv layer; K n is the convolution kernel size of the nth conv layer; L dw is the number of depthwise separable convolutional layers (dw); S out,m The spatial size of the feature map output by the mth dw layer; C in,m is the number of input channels of the mth dw layer; F dw,m The product of the number of input and output channels of the mth dw layer; K m The convolution kernel size of the mth dw convolution layer; L fc is the number of fully connected layers (fc); F fc,p is the product of the number of input and output nodes of the pth fc layer.

[0161] Finally, the relationship between inference energy consumption and the number of neural network layers is defined as:

[0162]

[0163] Where E is the inference energy consumption (unit: Joule); L is the number of layers of the neural network; e c is the energy consumption per floating point operation (unit: Joule / FLOP); e t is the energy consumption per byte of data access (unit: Joule / Byte); B d is the number of bytes of each data (FP32, FP16, INT8); S out,k is the spatial size of the k-th layer output feature map; γ k is the calculation coefficient of the kth layer; W k is the number of weight parameters of the kth layer; A in,k and A out,k is the number of input and output activation elements of the kth layer.

[0164] 2. Problem Modeling

[0165] Some tasks may require higher reasoning accuracy, which requires deeper network layers, but deeper networks mean higher computational load and energy consumption; in other tasks, in order to reduce latency or save resources, it may be necessary to reduce the number of network layers or adjust the batch size. In addition, the satellite's computing resources, power status, communication conditions, and different mission scenarios may all have an impact on the optimal configuration of the reasoning model.

[0166] First, define the tasks in different time slots t i The distribution function of time slot t i The number of images to be processed follows a Poisson distribution as follows:

[0167]

[0168] Among them, n i Indicates time slot t i The number of pictures, λ i Indicates time slot t i The average number of images.

[0169] Time slot t i The set of images to be processed in is represented as ik is the time slot t i Any picture number.

[0170] Secondly, define the users in different time slots t i The relative preference for inference accuracy and inference latency.

[0171] ω1(t i )+ω2(t i )=1 (16)

[0172] Among them, ω1(t i ) represents the weight of inference accuracy, ω2(t i ) represents the weight of the inference delay.

[0173] User preference function and the number of images n i When the number of pictures is larger, users pay more attention to the inference delay; when the number of pictures is smaller, users pay more attention to the inference accuracy. That is, the fewer the number of pictures, the higher the requirement for inference accuracy. Only the weight of inference accuracy in different time slots is defined, and the weight of inference delay can be calculated by formula 17.

[0174]

[0175] Among them, ρ is a tuning parameter, which indicates the sensitivity of preference to the number of images.

[0176] In order to measure each time slot t i User satisfaction with satellite reasoning systems, this study uses QoE i The Quality of Experience (QoE) indicator includes the following two key factors: inference accuracy and inference latency.

[0177]

[0178] Among them, ω1(t i )+ω2(t i )=1 means that in each time slot t i Inference accuracy and latency, users have relative preferences for inference accuracy and latency.

[0179] Each ti The user requirements are different: some users may be more concerned about inference accuracy, while others may be more concerned about inference latency. In order to maximize the i User Quality of Experience (QoE) i ,The number of network layers and batch size need to be dynamically adjusted to meet different requirements.

[0180] Objective function: maximize each t i User satisfaction QoE i Constraints: The inference energy consumption does not exceed the available energy of the satellite. The number of network layers and batch size are within the feasible domain.

[0181]

[0182] L min ≤L i ≤L max

[0183] B min ≤B i ≤B max

[0184] 3. Problem Solving

[0185] For such dynamic optimization and stochastic optimization problems, the Markov decision process can be used for mathematical modeling and solution.

[0186] Markov Decision Process (MDP) can capture the dynamic changes of satellite systems in different time slots, including the dynamic changes of power, the number of tasks, and changes in user preferences. Reasoning accuracy and latency are mutually constrained goals. The reward function can comprehensively consider different goals and achieve optimization by adjusting the number of network layers. MDP can well handle the impact of system decisions on future states, especially when satellite resources are limited. The decision in the current time slot will directly affect the available resources in the subsequent time slots.

[0187] Due to the different ways in which the sunlit area and the shadow area obtain energy, they are modeled as a deterministic Markov decision process and a Markov decision process, respectively.

[0188] ●Sunlight area modeling: Deterministic Markov decision process (Deterministic MDP)

[0189] Although the energy acquisition is relatively certain in the sunny area, there are still dynamic changes and uncertainties in the system, such as changes in task load and fluctuations in user demand, which makes it meaningful to use MDP modeling. For the determined energy acquisition in the sunny area, this example uses the energy state transfer function instead of the energy state transfer probability.

[0190] The deterministic Markov decision process consists of the tuple (S, A, T, r, γ).

[0191] is a state space where the state elements Indicates time slot t i Available energy of satellite, state element I i Indicates time slot t i distribution of tasks.

[0192] It is the action space, which is the selection object of the adaptive strategy.

[0193] T={(T i E ,T i I )} is the state transfer function, where T i E represents the energy state transfer function, T i I Represents the task state transition function.

[0194] The energy state transfer function is:

[0195]

[0196] Modeling the Earth's Shadow: Markov Decision Process (MDP)

[0197] Unlike the sunlit area, the energy in the shadow area is limited and completely relies on battery energy storage. Therefore, the challenge facing the system is how to dynamically adjust the inference model to maximize user satisfaction under limited energy conditions.

[0198] Since the states and resources in satellite reasoning tasks are dynamically changing, and the decisions made in each time slot will affect the future states, this type of problem is naturally suitable for modeling using Markov decision processes (MDPs).

[0199] The Markov decision process consists of the tuple (S, A, P, r, γ).

[0200] is a state space where the state elements Indicates time slot t i Available energy of satellite, state element I i Indicates time slot t i distribution of tasks.

[0201] It is the action space, which is the selection object of the adaptive strategy.

[0202] P={(P i E ,Pi I )} is the state transition probability, where P i E represents the energy state transition probability, P i I represents the task state transition probability. The state transition probability describes the system from state s at a certain moment. t Transfer to the next state s t+1 probability.

[0203] The energy state transition probability is:

[0204]

[0205] The sum of the probabilities of each row is 1. i E (s n →s n ) is defined as follows:

[0206]

[0207] The task state transition probability is:

[0208]

[0209] r(t i ,m j ,l) is the reward function, which means that in time slot t i The CNN model m is adopted j The reward obtained after the l-th layer of the neural network is as follows:

[0210]

[0211] Among them, δ1 is the weight coefficient of accuracy energy efficiency, and δ2 is the deviation coefficient of minimum accuracy.

[0212] γ is the discount factor, which represents the discount rate of long-term rewards and is set between 0 and 1. The higher the setting, the better the expected performance of long-term planning.

[0213] 4. Simulation Test

[0214] 1. Simulation parameters

[0215] Table 1 shows a simulation scenario involving a low earth orbit (LEO) satellite at an altitude of 500 km. The satellite has an orbital period of 5670 seconds divided into 30 time slots. In addition, the satellite is equipped with 100 square meters of solar panels with an efficiency of 0.25.

[0216] Table 1LEO satellite network parameters

[0217]

[0218]

[0219] Table 2 describes the CNN models and computational parameters. Three CNN models are deployed on LEO satellites. These models have low computational complexity and are suitable for resource-constrained environments. In addition, they show strong competitiveness and complementarity in different usage scenarios.

[0220] Table 2 CNN model and calculation parameters

[0221]

[0222] Table 3 shows the Top-1 accuracy of ResNet, MobileNetV2, and MobileNetV3 on the ImageNet dataset. On NVIDIA Jetson AGX Orin, the batch size is 1 and the resolution multiplier is 1. The performance at different network layers is shown in the following table.

[0223] Table 3 Performance of different model network layers

[0224]

[0225]

[0226] 2. Simulation results and analysis

[0227] Figure 4 The relationship between the available energy in different time slots and the selected CNN model network layer strategy (marked by different models) during the entire orbital cycle is shown. The red step curve on the left represents the dynamic change of available energy over time, while the blue curve on the right reflects the specific deep model (ResNet or MobileNet series and its different α parameters) used in each time slot through different marks, thus reflecting the decision-making process of the number of network layers based on QoE.

[0228] In the early stage of the track (where the time slot is smaller on the left), the available energy is low, but gradually increases over time. In this stage, the strategy chooses more lightweight models (such as MobileNet variants) to obtain reasonable QoE performance at low energy consumption. As the time slot progresses, the available energy increases significantly in the "sunlit region". At this time, the increase in energy consumption caused by selecting higher layers or more complex models (such as ResNet-101, ResNet-152) can be supported by high energy conditions, thereby ensuring higher accuracy without causing energy shortages. When entering the "shadow region", the available energy begins to decrease, and the choice of deep models needs to be weighed again. The strategy will tend to reduce the complexity of the model (select a lighter network) to reduce energy consumption, and maintain the best QoE as much as possible under the available energy constraint.

[0229] Figure 5 This figure reflects the process and results of using DQN (Deep Q-Network) to dynamically decide the number of model layers in satellite on-orbit reasoning tasks in this research work. In the figure, the horizontal axis is the number of training rounds (Episode), and the vertical axis is the total reward (Total Reward) obtained in this round. The reward function comprehensively considers the reasoning accuracy, latency, and energy consumption constraints. Through the DQN algorithm, the agent learns how to choose the most appropriate model complexity (number of network layers and structure) under a given state (including available energy, current task requirements, time slots and lighting environment) in continuous interaction, thereby maximizing the cumulative reward in the long run.

[0230] The total reward is low and fluctuates greatly in the early stage, reflecting the lack of experience of the agent in the initial stage and the need to explore different model configurations and strategies. In the middle and late stages, as the agent continuously optimizes the strategy through experience replay and Q-value update, the total reward tends to gradually increase and gradually approach a stable value, showing the adaptation and convergence of the strategy to the environment.

[0231] Figure 6The experimental results show that the performance of different models in terms of selection frequency reflects the trade-off between performance and resource consumption. As the depth of the ResNet series increases, the selection frequency increases overall, but medium-depth models (such as ResNet-34 and ResNet-152) are more attractive. In the MobileNet series, the selection frequency of MobileNetV3 is better than that of MobileNetV2 overall, especially MobileNetV3_α=0.75, which achieves the best balance between performance and efficiency. It is the model with the highest selection frequency in the final experimental stage, showing a significant advantage. In resource-constrained scenarios, MobileNetV2_α=0.25 and MobileNetV3_α=0.25 also have high application value. Overall, the experimental results clearly reflect the importance of model lightweight and high efficiency in practical tasks.

[0232] from Figure 7 It can be seen that different strategies have significant differences in performance indicators. Compared with random strategies and fixed model strategies, the MDP-DQN strategy performs best in all key performance indicators, especially in terms of average reward (AvgReward) and average accuracy (Avg Accuracy), which reach 82.3 and 76.5 respectively, significantly better than other strategies. At the same time, the MDP-DQN strategy also maintains a low level in average latency (Avg Latency) and average energy consumption (Avg Energy), which are 2.1ms and 2.3J respectively, reflecting its advantages in high efficiency and energy saving. This shows that the MDP-DQN strategy can effectively balance task performance and resource overhead in a resource-constrained environment, significantly improve the optimization effect of the overall system, and provide a better solution for dynamic task allocation and model selection.

[0233] from Figure 8 It can be seen that in 1000 training rounds, the total reward of the algorithm shows a gradual upward trend with the increase in the number of training rounds, and the fluctuation gradually decreases. This shows that the algorithm has good convergence during the training process and can effectively learn the optimal strategy. The average reward curve shown in the figure rises steadily, while the standard deviation area gradually shrinks, which further verifies the robustness of the algorithm, indicating that the model has stable performance in multiple experiments and has high adaptability and reliability. This result proves that the proposed method can effectively optimize dynamic tasks in complex environments and has strong generalization ability.

[0234] V. Conclusion

[0235] This paper combines Markov decision process (MDP) with deep reinforcement learning (DQN) to optimize the efficiency of computing resource utilization while ensuring the accuracy and latency requirements of task response. Experimental results show that this strategy has significant advantages in solving resource allocation problems in complex dynamic environments, and can effectively balance reasoning accuracy, response latency and computing energy consumption, thereby improving the overall performance of satellite mission systems.

Claims

1. A LEO multi-model parameter dynamic inference strategy, characterized by: include: (1) Building a satellite network model The image inference model is used to capture images and perform inference tasks; the orbit equation of the LEO satellite is expressed as: r(t)=(R+h)[1-e·cos(α(t))] (1) Where R is the radius of the Earth, h is the orbital altitude, e is the orbital eccentricity, and α(t) is the true anomaly, which is the angle between the current position of the satellite and the perigee; The orbital period of a LEO satellite is: Where G is the gravitational constant, M e is the mass of the Earth; (2) Building a satellite energy model Assume that the instantaneous solar radiation power received by the LEO satellite solar panel per square meter is P r (t), which is defined as follows: Among them, P max is the maximum power received by the solar panel per square meter, α is the proportion of the illuminated area during the orbital period, and σ controls the width of the radiation intensity distribution; Assume the initial battery energy of the satellite is The maximum battery capacity is The minimum tolerable energy level is The orbital period is divided into N equal time periods, each of which is separated by a time interval of Δt; the time period sequence is represented by T = {t1,···,t i ,···,t N }, where t i Represents any time period; If t i In the sunlit area, that is, 1≤i≤αN, the energy captured by the LEO satellite is: Where η is the solar energy conversion efficiency and D is the area of ​​the solar panel; A fixed battery charging energy mechanism is used, that is, the same average energy is added to the battery in each time period; the battery i The charging energy during this period is: In t i The available energy for image inference during this period is: When the satellite is in the shadow area, that is, αN<i≤N, the cyclic average energy mechanism is adopted; after deducting the inference energy, the average value of the next time period is recalculated; this iterative process will continue until the last time period in the shadow area; In t i The available energy for image inference during this period is: Among them, E v represents the energy consumption in any time period from αN to i-1; The sequence of available energy for image inference during the entire orbital period is (3) Building a satellite inference model The set of CNN models with different parameters for the deployment of M LEO satellites is Where m j Represents an arbitrary CNN model; at each time slot t i An optimal model will be pre-selected, expressed as For the best model selected The number of neural network layers is L, l represents any layer of the neural network (l = 1, 2, ..., L); Time slot t i Using model m j The performance set for reasoning the kth image is: P I ={p ijk ,t ijk ,y ijk },1≤i≤N,1≤j≤M (8) Among them, p ijk is the inference power of the image, t ijk is the inference time of the image, y ijk is the inference label of the picture; Time slot t i Select model m j The inference energy consumption is: If the inference label y ijk With the actual label x ijk If they are the same, it is counted as a correct inference; then in time slot t i Select the inference model m i The inference accuracy is: Among them, 1(x) is an indicator function, indicating whether the condition is met, 1 if it is met, and 0 otherwise; (4) Define tasks in different time slots t i The distribution function of time slot t i The number of images to be processed follows a Poisson distribution as follows: Among them, n i Indicates time slot t i The number of pictures, λ i Indicates time slot t i The average number of pictures; Time slot t i The set of images to be processed in is represented as ik is the time slot t i Any picture number of ; (5) Define users in different time slots t i Relative preference for inference accuracy and inference latency: ω1(t i )+ω2(t i )=1 (12) Among them, ω1(t i ) represents the weight of inference accuracy, ω2(t i ) represents the weight of the inference delay; User preference function and the number of images n i When the number of images is large, users pay more attention to the inference delay; when the number of images is small, users pay more attention to the inference accuracy. The weight of the inference accuracy in different time slots is defined, and the weight of the inference delay can be calculated by formula (13): Among them, ρ is a tuning parameter, which indicates the sensitivity of preference to the number of pictures; Adopting QoE i Indicators, including the following two key factors: inference accuracy and inference latency; Among them, ω1(t i )+ω2(t i )=1 means that in each time slot t i In terms of the user's relative preference for inference accuracy and inference latency; Each t i The user needs are different. In order to maximize the i User Quality of Experience (QoE) i ,The number of network layers and batch size need to be dynamically adjusted to meet different requirements; Maximize each t i User satisfaction QoE i ; The inference energy consumption does not exceed the available energy of the satellite, and the number of network layers and batch size are within the feasible domain: L min ≤L i ≤L max B min ≤B i ≤B max (19) (6) Modeling of the sunlit area: The deterministic Markov decision process uses the energy state transfer function to replace the energy state transfer probability. The deterministic Markov decision process consists of a tuple (S, A, T, r, γ); is a state space where the state elements Indicates time slot t i Available energy of satellite, state element I i Indicates time slot t i The distribution of tasks; is the action space, which is the selection object of the adaptive strategy; T={(T i E ,T i I )} is the state transfer function, where T i E represents the energy state transfer function, T i I Represents the task state transfer function; The energy state transfer function is: (7) Modeling of Earth's Shadow Region: Markov Decision Process The Markov decision process consists of the tuple (S, A, P, r, γ), is a state space where the state elements Indicates time slot t i Available energy of satellite, state element I i Indicates time slot t i The distribution of tasks; is the action space, which is the selection object of the adaptive strategy; P={(P i E ,P i I )} is the state transition probability, where P i E represents the energy state transition probability, P i I represents the task state transition probability; the state transition probability describes the system from state s at a certain moment t Transfer to the next state s t+1 The probability of The energy state transition probability is: The sum of the probabilities of each row is 1. i E (s n →s n ) is defined as follows: The task state transition probability is: r(t i ,m j ,l) is the reward function, which means that in time slot t i The CNN model m is adopted j The reward obtained after the l-th layer of the neural network is as follows: Among them, δ1 is the weight coefficient of accuracy energy efficiency, and δ2 is the deviation coefficient of minimum accuracy; γ is the discount factor, which represents the discount rate of long-term rewards and is set between 0 and 1. The higher the setting, the better the expected performance of long-term planning.

2. The LEO multi-model parameter dynamic reasoning strategy according to claim 1 is characterized by: In order to quantify the accuracy and energy consumption, the accuracy-energy efficiency index is defined, which represents the inference accuracy per unit of energy consumption, and the unit is accuracy per joule (Accuracy / Joule); this index is used in the initial decision model in each time slot: After the initial model selection, each time slot needs to decide the optimal number of network layers corresponding to its model; the definition steps are as follows: First, define the relationship between the inference accuracy A and the number of neural network layers l: Among them, α1 and β1 are model parameters, which determine the growth rate and initial accuracy; α2 and β2 determine the maximum value of accuracy and the shape of decline, l mid is the number of layers where the accuracy reaches the peak; Secondly, define the relationship between the inference delay and the number of neural network layers: Where T is the inference delay (unit: s), which represents the time superposition of the model from input to output across all neural network layers; P eff is the effective computing performance of the hardware (unit: FLOPs / s); L conv is the number of standard convolutional layers (conv); S out,n is the spatial size of the output feature map of the nth conv layer; F conv,n is the product of the number of input and output channels of the nth conv layer; K n is the convolution kernel size of the nth conv layer; L dw is the number of depthwise separable convolutional layers (dw); S out,m The spatial size of the feature map output by the mth dw layer; C in,m is the number of input channels of the mth dw layer; F dw,m The product of the number of input and output channels of the mth dw layer; K m The convolution kernel size of the mth dw convolution layer; L fc is the number of fully connected layers (fc); F fc,p is the product of the number of input and output nodes of the pth fc layer; Finally, the relationship between inference energy consumption and the number of neural network layers is defined as: Where E is the inference energy consumption (unit: Joule); L is the number of layers of the neural network; e c is the energy consumption per floating point operation (unit: Joule / FLOP); e t is the energy consumption per byte of data access (unit: Joule / Byte); B d is the number of bytes of each data (FP32, FP16, INT8); S out,k is the spatial size of the output feature map of the kth layer; γ k is the calculation coefficient of the kth layer; W k is the number of weight parameters of the kth layer; A in,k and A out,k is the number of input and output activation elements of the kth layer.

Images