A distributed inference method in a spaceborne edge computing system

Abstract

The present application relates to the field of satellite network, specifically relates to a kind of distributed inference method in on-board edge computing system.The present application is mainly aimed at on-board inference computing task.First, select the satellite participating in distributed inference;Then establish neural network model cutting optimization problem, determine which neural network layer is included in inference step according to the optimal decision variable obtained by solving;Finally, the multiple inference steps formed after cutting neural network model are placed on the selected satellite one by one.The present application can significantly improve the speed of on-orbit intelligent inference, and efficiently utilize the computing resources of multiple satellites.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of satellite networks, in particular to a distributed inference method in a satellite-borne edge computing system. BACKGROUND

[0002] The satellite-borne edge computing system can provide computing and storage services for users on site by deploying computing and storage resources on satellites, and typical services include intelligent inference on data generated by user terminals.

[0003] However, considering that the computing and storage resources on a single satellite are relatively limited, it may not be able to meet the resource requirements of the computing task, so it is necessary to split the computing task and reasonably deploy the split sub-tasks (select appropriate satellites to execute the split sub-tasks).

[0004] Taking target recognition on images as an example, if the data source continuously generates pictures to be processed, a single satellite may not be able to continuously complete the target recognition task. For example, when the picture generation rate is 100 pictures / second, and the processing rate of a single satellite for pictures is 20 pictures / second, as time goes on, a large number of pictures to be processed will appear. Therefore, multiple satellites are needed for distributed collaborative inference to improve the inference speed. SUMMARY

[0005] The present application addresses the above problems by splitting the inference process and executing different steps of the entire inference process on different satellites,

[0006] The technical solution adopted by the present application is as follows:

[0007] A distributed inference method in a satellite-borne edge computing system, comprising the following steps:

[0008] (1) selecting N chosen satellites participating in distributed inference; wherein N chosen is a configurable even parameter;

[0009] (2) establishing a neural network model cutting optimization problem, and determining which neural network layers are included in each of the N chosen inference steps according to the optimal decision variables obtained by solving;

[0010] (3) placing the N chosen inference steps formed after cutting the neural network model on the N chosen satellites in sequence.

[0011] Further, in step (1), the principle for selecting satellites is that there is an inter-satellite link between the two satellites selected in sequence, and there is an inter-satellite link between the last selected satellite and the first satellite.

[0012] Further, the inference step in step (2) is:

[0013] Merging some adjacent layers of the deep neural network to form an inference step, or a single layer as an inference step; wherein each inference step only contains some layers that are continuous and adjacent, and does not include layers that are not continuous and adjacent.

[0014] Further, the specific process of step (2) is:

[0015] Optimizing the model cutting problem, the decision variables in the optimization problem include matrix variable Δ = {δ i,j} i∈I,j∈J and matrix variable Γ = {γ i,j} i∈I,j∈J ; δ i,j takes the value of 0 or 1, δ i,j = 1 represents that the jth layer is deployed on the ith satellite, that is, the jth layer is divided into the ith step, δ i,j = 0 represents that the jth layer is not deployed on the ith satellite, that is, the jth layer is not divided into the ith step; γ i,j takes the value of 0 or 1, γ i,j = 1 represents that the intermediate result generated by the jth layer on the ith satellite is sent to the selected next satellite, γ i,j = 0 represents that the intermediate result generated by the jth layer on the ith satellite is not sent to the selected next satellite.

[0016] Define various variables, including: I represents the set {1, 2, …, N chosen}, J represents the layer number set in the deep neural network model, that is, the set {1, 2, …, L max}, L max represents the number of neural network layers; c i represents the operation amount that the ith satellite can complete per second, l j represents the operation amount required by the jth layer in the deep neural network, then the processing time required for processing the jth layer on the ith satellite is equal to s j represents the intermediate result data amount generated by the jth layer, b i represents the link rate between the ith satellite and the (i+1)th satellite, represents the signal propagation delay between the ith satellite and the (i+1)th satellite, then if the intermediate result of the jth layer is sent from the ith satellite to the (i+1)th satellite, the transmission delay is calculated as

[0017] By using the method of performing data operation while transmitting the intermediate result, the total delay generated by the ith satellite is calculated as The maximum value of the transmission delay and the processing time length;

[0018] Then the neural network model cutting problem is optimized modeling:

[0019]

[0020]

[0021]

[0022]

[0023]

[0024]

[0025]

[0026]

[0027]

[0028]

[0029]

[0030] After solving the above optimization problem, according to the optimal delta value obtained by solving, the specific scheme of cutting the neural network model is determined, and it is determined which neural network layers are included in each inference step.

[0031] The advantages of the present application compared with the prior art are: it can significantly improve the speed of on-orbit intelligent inference, and efficiently utilize the computing resources of multiple satellites. BRIEF DESCRIPTION OF DRAWINGS

[0032] Figure 1 It is a deep neural network diagram of the present application.

[0033] Figure 2 It is a deep neural network step cutting diagram of the present application.

[0034] Figure 3 It is a satellite selection and step placement diagram in distributed inference of the present application. DETAILED DESCRIPTION

[0035] The present application will be further explained and described below in conjunction with the accompanying drawings.

[0036] As Figure 1As shown, a deep neural network is generally composed of multiple layers, such as convolutional layers, fully connected layers, pooling layers, etc. Each layer completes certain data calculation. The inference process of a deep neural network can be abstracted as follows: data is input to the 1st layer, the data is calculated by the operators in the 1st layer to generate the result processed by the 1st layer, then the result processed by the 1st layer (intermediate result) is passed to the 2nd layer, the result processed by the 1st layer is calculated by the operators in the 2nd layer to generate the result processed by the 2nd layer. By analogy, until the final result is generated by the nth layer.

[0037] The present application proposes to "merge" part of adjacent layers of a deep neural network to form an inference "step". Of course, the special case of a single layer as a step is also allowed. Figure 2 For example, assuming that the deep neural network inference process in Figure 1 needs to be cut into 4 steps, then one possible cutting scheme of the present embodiment is as follows:

[0038] Step 1 consists of the 1st layer; Step 2 consists of the 2nd and 3rd layers; Step 3 consists of the 4th to nth-2 layers; Step 4 consists of the nth-1 and nth layers.

[0039] In order to avoid additional data transmission, in the process of cutting the model and forming the steps, each step only contains a number of consecutive adjacent layers and does not include non-consecutive adjacent layers, such as not dividing the 2nd and 4th layers into a step and dividing the 3rd layer into another step.

[0040] A distributed inference method in a satellite-borne edge computing system of the present application, specifically comprising the following steps:

[0041] (1) Select N chosen satellites participating in distributed inference;

[0042] Number the orbits in the constellation from 1 to N orbit , number the satellites in each orbit from 1 to N sat , and number all the satellites from 1 to N orbit ×N sat . Specifically, the number of the nth orbit satellite in the nth sat orbit is (n orbit -1) × N orbit +n sat . For example, if the number of orbits is 8 and the number of satellites in each orbit is 20, then the number of the 3rd satellite in the 2nd orbit is (2-1) × 20 + 3 = 23.

[0043] The principle of selecting the satellites is that the two satellites selected in sequence have inter-satellite links, and the last selected satellite has an inter-satellite link with the first satellite. This embodiment takes selecting four satellites as an example to introduce the specific scheme of selecting the satellites. If the nth orbit satellite in the nth sat orbit is the data source satellite (i.e. the satellite at which the reasoning process starts), there are the following eight better schemes of selecting four satellites:

[0044] 1) selecting the four satellites numbered (n orbit -1)×N orbit +n sat , n orbit ×N orbit +n sat , n orbit ×N orbit +n sat +1, (n orbit -1)×N orbit +n sat +1 in sequence;

[0045] 2) selecting the four satellites numbered (n orbit -1)×N orbit +n sat , (n orbit -1)×N orbit +n sat +1, n orbit ×N orbit +n sat +1, n orbit ×N orbit +n sat in sequence;

[0046] 3) selecting the four satellites numbered (n orbit -1)×N orbit +n sat , n orbit ×N orbit +n sat , n orbit ×N orbit +n sat -1, (n orbit -1)×N orbit +n sat -1 in sequence;

[0047] 4) selecting the four satellites numbered (n orbit -1)×N orbit +n sat , (n orbit -1)×N orbit +n sat -1, n orbit ×Norbit + n sat - 1, n orbit × N orbit + n sat 4 satellites numbered (n

[0048] 5) 4 satellites numbered (n orbit - 1) × N orbit + n sat , (n orbit - 2) × N orbit + n sat , (n orbit - 2) × N orbit + n sat + 1, (n orbit - 1) × N orbit + n sat + 1 in turn;

[0049] 6) 4 satellites numbered (n orbit - 1) × N orbit + n sat , (n orbit - 1) × N orbit + n sat + 1, (n orbit - 2) × N orbit + n sat + 1, (n orbit - 2) × N orbit + n sat in turn;

[0050] 7) 4 satellites numbered (n orbit - 1) × N orbit + n sat , (n orbit - 2) × N orbit + n sat , (n orbit - 2) × N orbit + n sat - 1, (n orbit - 1) × N orbit + n sat - 1 in turn;

[0051] 8) 4 satellites numbered (n orbit - 1) × N orbit + n sat , (n orbit - 1) × N orbit + n sat - 1, (n orbit - 2) × N orbit + n sat - 1, (n orbit- 2) x N orbit + n sat 4 satellites.

[0052] As a special case, if the orbit number of the selected satellite is greater than N orbit , the orbit number is changed to 1; if the orbit number of the selected satellite is less than 1, the orbit number is changed to N orbit ; if the in-orbit number of the selected satellite is greater than N sat , the in-orbit number is changed to 1; if the in-orbit number of the selected satellite is less than 1, the in-orbit number is changed to N sat .

[0053] (2) After N chosen (N chosen is an even number greater than 1 and less than or equal to the number of layers of the model) satellites participating in distributed reasoning have been selected, the model cutting problem is modeled, and by solving the model cutting problem, it is determined how the model is cut into N chosen steps, so that the N chosen steps formed after cutting are placed on the N chosen satellites in turn, realizing distributed reasoning.

[0054] First, the meaning of the set I used in modeling is explained: I represents the set {1, 2, …, N chosen}, and the elements in I represent the serial numbers of each satellite after N chosen satellites have been selected. For example, after satellite 22, satellite 23, satellite 43, and satellite 42 are selected in turn, the first satellite in the model cutting algorithm is satellite 22, the second satellite is satellite 23, the third satellite is satellite 43, and the fourth satellite is satellite 42.

[0055] The optimization variables include the matrix variable Δ = {δ i,j} i∈I,j∈J and the matrix variable Γ = {γ i,j} i∈I,j∈J . δ i,j can take values of 0 or 1, δ i,j = 1 represents deploying the jth layer on the ith satellite (that is, dividing the jth layer into the ith step), and δ i,j = 0 represents not deploying the jth layer on the ith satellite (that is, not dividing the jth layer into the ith step). Here, i can be any element in the set I, and j can be any element in the set J, which represents the layer number set in the deep neural network model, that is, the set {1, 2, …, L max}, and L max represents the number of neural network layers. γ i,j can take values of 0 or 1, γ i,j= 1 represents that the intermediate result generated by the i-th satellite at the j-th layer is sent to the selected next satellite, and γ i,j = 0 represents that the intermediate result generated by the i-th satellite at the j-th layer will not be sent to the selected next satellite.

[0056] Let c i represent the operation amount that the i-th satellite can complete per second, and let l j represent the operation amount required for the j-th layer in the deep neural network to be processed, then the processing time required for processing the j-th layer on the i-th satellite is equal to

[0057] Let s j represent the intermediate result data amount generated by the j-th layer, and let b i represent the link rate between the i-th satellite and the i+1-th satellite, represent the signal propagation delay between the i-th satellite and the i+1-th satellite. According to the above definition, if the i-th satellite transmits the intermediate result of the j-th layer to the i+1-th satellite, the transmission delay can be calculated as

[0058] By using the method of performing data operation at the same time as transmitting the intermediate result, the total delay generated by the i-th satellite can be calculated as that is, the maximum of the transmission delay and the processing time.

[0059] After completing the above definition, the model cutting problem can be optimized as follows:

[0060]

[0061]

[0062]

[0063]

[0064]

[0065]

[0066]

[0067]

[0068]

[0069]

[0070]

[0071] Explanation:

[0072] Equation (1) is the optimization objective, and the optimization variables are Δ and Γ;

[0073] Equation (2) is a type of constraint condition, indicating that for any i satisfying i ∈ I and any j satisfying j ∈ J, γ i,j and δ i,j need to satisfy (2);

[0074] Equation (3) is a type of constraint condition, indicating that for any i satisfying i ∈ I and i + 1 ∈ I, and for any j satisfying j ∈ J and j + 1 ∈ J, γ i,j and δ i+1,j+1 need to satisfy (3);

[0075] Equation (4) is a type of constraint condition, indicating that for any i satisfying i ∈ I and i + 1 ∈ I, and for any j satisfying j ∈ J and j + 1 ∈ J, γ i,j , δ i,j and δ i+1,j+1 need to satisfy (4);

[0076] Equation (5) is a type of constraint condition, indicating that for any j satisfying j ∈ J, the partial elements of Δ need to satisfy (5);

[0077] Equation (6) is a type of constraint condition, indicating that for any j satisfying j ∈ J, the partial elements of Γ need to satisfy (6);

[0078] Equation (7) is a type of constraint condition, indicating that for any i satisfying i ∈ I, and for any j1 ∈ J satisfying 1 ≤ j1 < j2 and any i2 ∈ J, the partial elements of Δ need to satisfy (7);

[0079] Equation (8) is a type of constraint condition, indicating that for any i satisfying i ∈ I, and for any j satisfying j ∈ J, δ i,j need to satisfy (8);

[0080] Equation (9) is a type of constraint condition, indicating that for any i satisfying i ∈ I, and for any j satisfying j ∈ J, γ i,j need to satisfy (9);

[0081] Equation (10) is a type of constraint condition, indicating that for any i satisfying i ∈ I, the partial elements of Δ need to satisfy (10);

[0082] Equation (11) is a type of constraint condition, indicating that for any i satisfying i ∈ I, the partial elements of Γ need to satisfy (11);

[0083] The explanation of the above optimization problem is as follows:

[0084] 1) Formula (1) means that by solving for the optimal values ​​of Δ and Γ, the time of one reasoning process is minimized;

[0085] 2) By satisfying constraints (2), (3), and (4), the value constraint relationship between the Δ element and the Γ element is ensured, that is, only δ i,j and δ i+1,j+1 When both are equal to 1, γ i,j Only equals 1;

[0086] 3) By satisfying constraints (5) and (6), ensure that each layer of the neural network can only be placed on one satellite;

[0087] 4) By satisfying the constraint (7), ensure that only a number of consecutive adjacent layers are deployed on each satellite (i.e., each step contains only a number of consecutive adjacent layers);

[0088] 5) Restrictions (8) and (9) specify the range of values ​​for the Δ and Γ elements, respectively;

[0089] 6) Constraints (10) and (11) achieve linearization of the optimization objective.

[0090] (3) The N generated after cutting the neural network model chosen The reasoning steps are sequentially placed in N. chosen On a satellite.

[0091] After selecting the satellites, the various steps involved in creating the cutting model can be sequentially placed onto the selected satellites. Figure 3 For example, suppose there are 20 satellites in each orbit of this constellation. The figure shows the first three satellites in orbits 1, 2, and 3, for a total of nine satellites. Assuming satellite 22 is the data source satellite, and taking four satellites as an example, step 1 can be placed in satellite 22, step 2 in satellite 23, step 3 in satellite 43, and step 4 in satellite 42.

Claims

1. A distributed inference method in a spaceborne edge computing system, characterized in that, Includes the following steps: (1) Select N chosen N satellites participate in distributed inference; among them, N chosen It is a configurable even-numbered parameter; (2) Establish a neural network model to segment the optimization problem, and determine N based on the optimal decision variables obtained from the solution. chosen Which neural network layers are included in each inference step? (3) The N generated after cutting the neural network model chosen The reasoning steps are sequentially placed in N. chosen On one satellite; The reasoning steps in step (2) are as follows: Some adjacent layers of a deep neural network are merged to form an inference step, or a single layer is used as an inference step; each inference step contains only a number of consecutive adjacent layers and does not include non-consecutive adjacent layers. The specific method for establishing the neural network model to cut the optimization problem in step (2) is as follows: Decision variables in optimization problems include matrix variables Δ={δ i,j } i∈I,j∈J And matrix variable Γ={γ i,j } i∈I,j∈J ;δ i,j The value can be 0 or 1, δ i,j =1 means that the j-th layer is deployed on the i-th satellite, that is, the j-th layer is assigned to the i-th step, δ i,j =0 means that the j-th layer is not deployed on the i-th satellite, that is, the j-th layer is not assigned to the i-th step; γ i,j The value is either 0 or 1, γ i,j =1 means that the i-th satellite sends the intermediate results generated in the j-th layer to the next selected satellite, γ i,j =0 means that the intermediate results generated by the j-th layer are not sent by the i-th satellite to the next selected satellite; Define the variables, including: I represents the set {1,2,…,N}. chosen }, where J represents the set of layer numbers in the deep neural network model, i.e., the set {1,2,…,L}. max }, L max c represents the number of layers in a neural network. i l represents the amount of computation that the i-th satellite can complete per second. j The amount of computation required to process the j-th layer in a deep neural network is represented by the processing time required to process the j-th layer on the i-th satellite, which is equal to the processing time required to process the j-th layer on the i-th satellite. s j b represents the amount of intermediate result data generated at layer j. i This represents the link rate between the i-th satellite and the (i+1)-th satellite. Let represent the signal propagation delay between the i-th satellite and the (i+1)-th satellite. If the i-th satellite sends the intermediate result of the j-th layer to the (i+1)-th satellite, the transmission delay is calculated as follows: By using a method that performs data processing while transmitting intermediate results, the total delay generated by the i-th satellite is calculated as follows: That is, the maximum value between transmission delay and processing time; Then, the neural network model segmentation problem is modeled for optimization: After solving the above optimization problem, based on the optimal Δ value obtained, determine the specific scheme for cutting the neural network model and identify which neural network layers are included in each inference step.

2. The distributed inference method in a spaceborne edge computing system according to claim 1, characterized in that, In step (1): the principle for selecting satellites is that the two satellites selected in sequence have an inter-satellite link, and the last selected satellite has an inter-satellite link with the first satellite.

Images