Inference decision-making method based on model division and transfer learning

Abstract

The invention relates to the technical field of mobile communication, in particular to a model division and transfer learning-based reasoning decision-making method, which comprises the following steps of S1, deploying a DNN model and generating a state value, inputting the state value and a training set into the DNN model, if the DNN model needs to be redeployed, loading neural network parameters from the DNN model which is deployed last time, and if the DNN model needs to be redeployed, loading neural network parameters from the DNN model which is deployed last time; initializing full connection layer parameters, inputting the full connection layer parameters into a coding network, and inputting model parameters into a model cache; s2, sampling training data from the model cache, and calculating an accumulated advantage, a loss function value and a gradient descent update parameter; s3, training is performed according to the calculation result until a preset training round is reached, and then training is completed; and S4, inputting the test set into the training model to obtain a decision result. According to the method, time is shortened through transfer learning, computing resources are saved, the high-cost effect of training all parameters from the beginning is avoided, model division and transfer learning are combined, and downtime is shortened.

Description

Technical Field

[0001] The present invention relates to the field of mobile communication technology, and in particular to an inference decision method, device and storage medium based on model partitioning and transfer learning. Background Art

[0002] With the continuous development and advancement of mobile communication technology, machine learning is no longer limited to traditional cloud data centers. This trend has led deep neural network (DNNs) model reasoning technology to a new level, providing broader possibilities for the application of many mobile intelligent systems, which cover a wide range of fields including but not limited to drone technology, medical Internet of Things applications and autonomous driving technology.

[0003] However, with the rapid growth of mobile intelligent applications, mobile intelligent systems are facing the challenges of large-scale computationally intensive and real-time reasoning tasks, which require a large amount of data processing and decision making in a very short time, but mobile devices are usually limited by hardware performance, which makes it difficult for them to meet these highly complex computing requirements because it involves performing complex machine learning tasks on resource-constrained mobile devices.

[0004] Existing edge collaborative reasoning systems encounter serious performance issues when switching across geographic services or networks. Specifically, current methods have the following shortcomings: Performance issues: These systems encounter significant performance issues when switching networks or encountering severe interference, which may lead to service unreliability and task timeouts; Service processing mechanism defects: Although current containerized computing frameworks are able to handle some distributed tasks, these systems lack flexible service processing mechanisms, especially between geographically dispersed edge nodes; Poor resource utilization: The challenge of efficient resource utilization has not been fully addressed, which means that existing methods still need to be improved in resource management. Summary of the invention

[0005] The purpose of the present invention is to solve the shortcomings of the prior art and provide a reasoning decision method based on model partitioning and transfer learning, comprising the following steps:

[0006] S1: deploy the DNN model and generate a state value, input the state value and the training set into the DNN model, determine whether to redeploy the DNN model, and if so, load the neural network parameters from the last deployed DNN model, initialize the fully connected layer parameters and input them into the encoding network, and input the model parameters including the neural network parameters and the fully connected layer parameters into the model cache;

[0007] S2: sampling training data from the model cache, and calculating cumulative advantage, loss function value and gradient descent update parameters according to the training data;

[0008] S3: Perform training according to the accumulated advantage, the loss function value and the gradient descent update parameter until the training is completed after reaching a preset training round, and save the network parameters of the training model when the training is completed. If the training is not completed, return to step S2;

[0009] S4: Input the test set into the training model to obtain a decision result.

[0010] Preferably, in step S1, deploying the DNN model includes:

[0011] S11: Through the observation space O n Get each mobile device observation, where the observation space O n for and The Cartesian product of three subsets, where The computing conditions of the mobile device, including the type of DNN model, data to be inferred, preprocessing delay, and the interruption status of the previous slot, Communication related information including signal strength, location, movement and belonging edge node, edge node calculation conditions including node type and current occupancy level;

[0012] S12: encoding the mobile device observation by an encoder of a neural block to obtain an encoded observation, wherein the neural block includes two fully connected layers using a LeakyReLU activation function;

[0013] S13: inputting the coded observation into a memory drive module, and outputting a read result and a new memory result, wherein the memory drive module includes a memory block for recording and maintaining the most recent observation, a read block for receiving the coded observation, and a write block for receiving the old memory result;

[0014] S14: After the head module splices the encoded observation, the read result and the new memory result, a sub-decision is generated through probability sampling, and the critic network generates the state value V(o t ,m t ) to perform a training process, wherein the head module includes a fully connected layer using a LeakyReLU activation function and a fully connected layer using a Softmax activation function.

[0015] Preferably, in step S13, the step of inputting the coding observation into a memory driving module and outputting the coding observation result and the old memory result further comprises:

[0016] In the set time period, the reading block obtains the reading result h according to the encoding observation and the latest observation of the memory block. n,L :

[0017]

[0018] in, represents the Hadamard product, W n,1 ,W n,2 is the learning weight in the read block, g n,t is the coded observation of the set time period, m n,t is the most recent observation for the set time period;

[0019] During the set time period, the write block generates a new memory result based on the encoding observation and the latest observation of the memory block. And update the memory block according to the new memory result, the new memory result The calculation formula is as follows:

[0020]

[0021] Among them, W n,3 ,W n,4 ,W n,5 are the learning weights in the write block.

[0022] Preferably, in step S14, after the head module splices the encoding observation, the reading result and the new memory result, a sub-decision is generated by probability sampling, further comprising:

[0023] Aggregate the memory results of all mobile devices through the average formula to form the new memory content of each mobile device, and then generate sub-decisions through probability sampling;

[0024] The average formula is as follows:

[0025]

[0026] Among them, the To remember the results.

[0027] Preferably, in step S1, the step of inputting the state value and the training set into the DNN model and determining whether to redeploy the DNN model further comprises:

[0028] For the current observation of mobile device n in time period t, through the reward function r n,t Make a decision, reward function r n,t The calculation formula is as follows:

[0029]

[0030] in, is the local processing time, that is, the entire task is completely processed locally, is the pipeline time of device n in time period t;

[0031] Among them, for the prefetch mechanism, if the current slot is less than τ+1, the strategy of each mobile device is to process completely locally, and the function r n is a continuous function, the maximum value of the decision value is 1. The minimum decision value is -4. when When , the decision value is 0.

[0032] Preferably, in step S2, calculating the loss function value according to the training data further includes:

[0033] The small batch samples in the training set are input into the DNN model, and the estimated value of the cumulative advantage of the action is obtained according to the state value. The proximal strategy optimization objective function obtains the loss function value according to the estimated value and the training set, as shown below:

[0034]

[0035] Among them, E t [·] is the expected value from a mini-batch of samples, clip((∈)) is a pruning function with hyperparameters ∈, π θ is a policy with training parameters θ, θ old is the parameter before training update, ent((·)) represents the entropy reward to encourage exploration, ρ is a hyperparameter, is an estimate of the cumulative advantage of an action in a relative quantitative time period t, smooth(L1(·)) is the smooth L1 loss function, is the cumulative reward starting from time period t.

[0036] Preferably, obtaining an estimated value of the cumulative advantage of the action according to the state value further comprises:

[0037] An estimate of the cumulative advantage of actions over a period of time t based on the relative quantification of the state values The calculation formula is as follows:

[0038]

[0039] Among them, γλ is a hyperparameter, and λ∈[0,1], V(·) is the state value function during training, that is, the critic network, r t+i is the reward obtained in time period t+i, o t+iis the observation obtained in time period t+i, m t+i It is the memory content obtained in time period t+i.

[0040] Preferably, in step S4, the test set is input into the training model to obtain a decision result, further comprising:

[0041] After the test set is input into the training model, an action decision is obtained. The action decision A n for The Cartesian product of three subsets, where Including migration decisions of edge nodes, comprising sub-decisions of the training model partitioning decision, Contains the decision of data compression, which is used to indicate that the transmitted data is compressed according to the compression ratio.

[0042] Based on the same concept, the present invention also provides a computer device, including a memory and a processor, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the processor executes the steps of the reasoning and decision-making method based on model partitioning and transfer learning as described in any one of the embodiments.

[0043] Based on the same concept, the present invention also provides a storage medium storing computer-readable instructions, which, when executed by one or more processors, enables one or more processors to perform the steps of the reasoning and decision-making method based on model partitioning and transfer learning as described in any one of the embodiments.

[0044] Compared with the prior art, the present invention has the following beneficial effects:

[0045] 1) The present invention accelerates the process of redeploying the DNN model by loading the neural network parameters from the last deployed DNN model, significantly shortening the time, saving computing resources, avoiding the high cost of training all parameters from scratch, and improving the adaptability and stability of the network model in the new environment, thereby reducing the overall training cost, and by retaining the previously trained knowledge, enhancing the generalization ability of the model in a variety of practical application scenarios, further improving the efficiency and performance of the deep learning framework in edge task offloading, and optimizing the reasoning offloading of mobile devices through fair resource allocation and pre-fetch service migration;

[0046] 2) The present invention combines model partitioning and transfer learning to form a joint optimization problem, which helps to reduce downtime and provide continuous service support for mobile devices during service migration. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the following detailed description of the preferred embodiment.The drawings are only for the purpose of illustrating the preferred embodiments and are not to be construed as limiting the invention.

[0048] Figure 1 It is a flow chart of the reasoning and decision-making method based on model partitioning and transfer learning of the present invention;

[0049] Figure 2 It is a training flow chart of the present invention;

[0050] Figure 3 is a schematic diagram of a neural block of the present invention;

[0051] Figure 4 It is a schematic diagram of vertical partitioning of the present invention. DETAILED DESCRIPTION

[0052] In order to make the purpose, technical scheme and advantages of the present invention clearer, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. Obviously, the described embodiments are part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of this application.

[0053] Those skilled in the art will appreciate that, unless otherwise stated, the singular forms "a", "an", and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present invention refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.

[0054] Example 1

[0055] In order to meet the challenges of large-scale computationally intensive and real-time demanding inference tasks, mobile edge computing technology provides an innovative solution, namely device edge collaborative inference. This approach allows the DNN model to be divided into multiple small tasks, which are then intelligently distributed between mobile devices and edge computing infrastructure to achieve balanced computing loads and better utilize mobile devices and edge resources, improving the performance and efficiency of mobile intelligent applications. This not only reduces the burden on mobile devices, but also makes real-time decision-making and data processing more feasible, thereby promoting the further development and popularization of mobile intelligent systems.

[0056] Considering that some environmental information is difficult to obtain immediately, is uncertain, or is restricted due to privacy issues, the system state is incomplete in the real world. To meet this challenge, this problem is solved through the Markov decision process (POMDP). Then, the corresponding state, action, and reward space are defined, and PPO is combined with a memory mechanism to determine the decision.

[0057] The present invention designs a group of edge nodes M = {1, 2, ..., M} and a group of mobile devices N = {1, 2, ..., N}, and each edge node is equipped with a base station to provide computing resources through a mobile wireless link. In addition, some edge nodes are directly connected through optical fibers, and time is divided into a series of moments T = {1, 2, ..., T}.

[0058] See also Figure 1 and Figure 2 As shown, the reasoning decision method based on model partitioning and transfer learning provided in this embodiment includes the following steps:

[0059] S1: Deploy the DNN model and generate state values, input the state values ​​and training sets into the DNN model, determine whether to redeploy the DNN model, and if so, load the neural network parameters from the last deployed DNN model, initialize the fully connected layer parameters and input them into the encoding network, and input the model parameters including the neural network parameters and the fully connected layer parameters into the model cache to achieve transfer learning.

[0060] Preferably, in step S1, deploying a DNN model includes:

[0061] S11: Through the observation space O n Get observations for each mobile device The observation space O n for and The Cartesian product of three subsets, where The computing conditions of the mobile device, including the type of DNN model, data to be inferred, preprocessing delay, and the interruption status of the previous slot, Communication related information including signal strength, location, movement and belonging edge node, The edge node calculation conditions including the node type and the current occupancy level are used to implement the model partitioning. Specifically, in this embodiment, the classification information is discretized using one-hot encoding and the continuous dimensions are normalized.

[0062] See also Figure 3As shown, since mobile devices compete for computing resources in the system, this embodiment adopts an independent learning setting. By designing a memory mechanism to promote the exchange of hidden information between mobile devices, a neural network is designed to generate two sub-actions for each device, and a multi-head mechanism is used to obtain three sub-decisions respectively.

[0063] S12: Encode the observation of the mobile device through the encoder of the neural block to obtain the encoded observation, wherein the neural block includes two fully connected layers using the LeakyReLU activation function. Specifically, in this embodiment, the number of neurons in each fully connected layer is 128.

[0064] S13: Input the encoded observation to the memory driving module, and output the reading result and the new memory result, wherein the memory driving module includes a memory block for recording and maintaining the most recent observation (the memory block is a memory vector, and the dimension of this vector is set to 256), a reading block for receiving the encoded observation, and a writing block for receiving the old memory result. Specifically, in this embodiment, a memory driving module is designed after the encoder to further improve the perception ability.

[0065] Preferably, in step S13, the coding observation is input into the memory driving module, and the coding observation result and the old memory result are output, further comprising:

[0066] In the set time period, the reading block obtains the reading result h according to the encoding observation and the latest observation of the memory block. n,L :

[0067]

[0068] in, represents the Hadamard product, W n,1 ,W n,2 is the learned weight in the read block, g n,t is the coded observation for the set time period, m n,t is the most recent observation for the set time period;

[0069] During the set time period, the write block generates a new memory result based on the encoding observation and the most recent observation of the memory block. And update the memory block according to the new memory result. The calculation formula is as follows:

[0070]

[0071] Among them, W n,3 ,W n,4 ,W n,5 are the learned weights written into the block.

[0072] S14: After the head module concatenates the encoded observations, read results, and new memory results, it generates sub-decisions through probability sampling, and the critic network generates the state value V(o t ,m t ) for training process, wherein the head module includes a fully connected layer using a LeakyReLU activation function and a fully connected layer using a Softmax activation function. Specifically, in this embodiment, three heads are used to generate different types of decisions, each head consists of two fully connected layers, each layer has 128 neurons, the activation function of the first layer is LeakyReLU, and the second layer is Softmax. Specifically, in this embodiment, the splicing block in the head module splices the encoded observation, the reading result, and the new memory result to generate a sub-decision through probability sampling, and the critic network generates a state value V(o t ,m t ).

[0073] Preferably, in step S14, after the head module splices the coded observation, the reading result and the new memory result, a sub-decision is generated by probability sampling, further comprising:

[0074] Aggregate the memory results of all mobile devices through the average formula to form the new memory content of each mobile device, and then generate sub-decisions through probability sampling;

[0075] The average formula is as follows:

[0076]

[0077] in, To memorize the results, specifically, in this embodiment, all mobile devices' understanding of the environment is aggregated to form new memory content of each mobile device.

[0078] The transfer learning method based on policy reuse redeploys the model when the environment changes, but only initializes some parameters, while the other parameters follow the parameters obtained from previous training. This method aims to reduce the training time and resource consumption in the new environment by maintaining the continuity of existing knowledge, while improving the adaptability of the model in the new environment. In the proposed deep learning framework for edge task offloading, only the fully connected layer used to encode the input in the neural network is initialized during redeployment, while the remaining parameters follow the parameters obtained from previous training. This method retains the knowledge of the feature extraction part such as the convolution layer by selectively initializing part of the model parameters, thereby speeding up the redeployment process. Specifically, when the edge device environment changes, such as due to differences in network conditions or device performance, the framework only reinitializes the parameters of the fully connected layer. This is because the fully connected layer is usually responsible for the specific classification or regression operation of the task, and its parameters are more susceptible to environmental changes, while the parameters of the convolution layer are mainly used for the extraction of general features and have good cross-environment adaptability. Through this policy reuse method, we can quickly deploy the model in the new edge environment, saving computing resources and ensuring the effectiveness and accuracy of the model.

[0079] Preferably, in step S1, the state value and the training set are input into the DNN model, and determining whether to redeploy the DNN model further includes:

[0080] For the current observation of mobile device n in time period t, through the reward function r n,t Make a decision, reward function r n,t The calculation formula is as follows:

[0081]

[0082] in, is the local processing time, that is, the entire task is completely processed locally, is the pipeline time of device n in time period t;

[0083] Among them, for the prefetch mechanism, if the current slot is less than τ+1, the strategy of each mobile device is to process completely locally, and the function r n is a continuous function, the maximum value of the decision value is 1. The minimum decision value is -4. when When , the decision value is 0.

[0084] S2: Sample training data from the model cache, calculate the cumulative advantage, loss function value and gradient descent update parameters based on the training data.

[0085] Proximal Policy Optimization (PPO) is a policy gradient based deep reinforcement learning framework with the advantages of superior performance, fast implementation and easy debugging.

[0086] Preferably, in step S2, calculating the loss function value according to the training data further includes:

[0087] The small batch samples in the training set are input into the DNN model, and the estimated value of the cumulative advantage of the action is obtained according to the state value. The proximal strategy optimization objective function obtains the loss function value according to the estimated value and the training set, as shown below:

[0088]

[0089] Among them, E t [·] is the expected value from a mini-batch of samples, clip((∈)) is a pruning function with hyperparameters ∈, π θ is a policy with training parameters θ, θ old is the parameter before training update, ent((·)) represents the entropy reward to encourage exploration, ρ is a hyperparameter, is an estimate of the cumulative advantage of an action in a relative quantitative time period t, smooth(L1(·)) is the smooth L1 loss function, is the cumulative reward starting from time period t.

[0090] Preferably, obtaining an estimated value of the cumulative advantage of the action according to the state value further includes:

[0091] An estimate of the cumulative advantage of actions over a period of time t based on the relative quantification of state values The calculation formula is as follows:

[0092]

[0093] Among them, γλ is a hyperparameter, and λ∈[0,1], V(·) is the state value function during training, that is, the critic network, r t+i is the reward obtained in time period t+i, o t+i is the observation obtained in time period t+i, m t+i It is the memory content obtained in time period t+i.

[0094] S3: Perform training according to the accumulated advantage, loss function value and gradient descent update parameters until the training is completed after reaching the preset training rounds, and save the network parameters of the training model when the training is completed. If the training is not completed, return to step S2;

[0095] S4: Input the test set into the training model to obtain the decision result.

[0096] Preferably, in step S4, the test set is input into the training model to obtain a decision result, further comprising:

[0097] After inputting the test set into the training model, the action decision is obtained. Action decision A n for The Cartesian product of three subsets, where Including migration decisions of edge nodes, Contains sub-decisions of the training model partitioning decision, Contains the decision of data compression, which is used to indicate that the transmitted data is compressed according to the compression ratio.

[0098] Preferably, assuming that the device environment state remains unchanged within a moment and the mobile device has an inference task to process, the task can be offloaded to the edge node within a moment, where the inference offloading includes three processing steps: local processing, data transmission and remote processing.

[0099] A practical way for service providers (such as Alicloud) to manage computing resources (CPU, GPU, and memory) for AI services is to divide and package the resources into different service settings in proportion. The computing capacity of edge node m is represented as Measured in floating point operations per second (FLOPS). The maximum and minimum computing power of service instances that a provider can create.

[0100] Each device n that occupies the computing power of node m can obtain a node with computing power C m (ω m,t ), where the occupancy level of edge node m in time period t is ω m,t ∈N + . Edge nodes can provide up to For service instances, we design a pre-fetch service migration setting, which occupies the resources of subsequent edges before migration. In this case, mobile devices will occupy the computing power of multiple edge nodes at the same time.

[0101] refer to Figure 4 As shown in Figure 2, a vertical partition is set to split the inference task into offloadable computational workloads to achieve model partitioning. It is assumed that each mobile device only generates one type of DNN inference task, considering that each device executes a specific application at a time.

[0102] Each mobile device generates only one type of DNN inference task, considering that each device executes a specific application at a time. At time t, the feasible partitioning solutions for mobile device n can be represented as a set P n,t ={1,2,…,P n,t}, where Pn,t ∈N + is the number of partition solutions in the set, given a point P n,t ∈P n,t For device n at time t, the time required to execute the local part is denoted as t n (p {n,t} ).

[0103] Preferably, according to each observation, the mobile device needs to take action for each time, and the action space A n It consists of three subsets, namely and Contains all edge nodes and default selections, including migration operations to corresponding nodes and no operation, contains all model partition solutions and the default selection, which means selecting the corresponding partition points and keeping the past partition solutions, Contains all data compression solutions. Note that we consider typical compression thresholds as possible decisions and keep the previously selected partitioning solution when the partition point is selected.

[0104] in, Contains data compression decisions, which are used to indicate that the transmitted data is compressed according to the compression ratio. This embodiment performs compression through pruning, which is a model lightweight technology that compresses the model to a very small size with a slight but non-negligible performance loss. Similar to the data distribution of model parameters, a considerable portion of parameter values ​​in the intermediate results are close to zero. Therefore, the absolute values ​​in the intermediate results that are less than the threshold are set to 0. Use bitmap arrays and data arrays to store processed data, and represent the original data size of the intermediate results as D' n (p n,t ),in For a mobile device n and time t, choose η n,t ∈ζ n As the threshold of pruning compression, the compressed data size is expressed as D n (p n,t ,η n,t ), which is a non-increasing continuous function.

[0105] In order to measure the impact of quantization on the inference results, a function is defined Indicates whether the compressed inference result is different from the original result. If the event is true, the function value is 1; otherwise, it is 0. In order to punish the negative impact of such events, we regard these tasks as overtime tasks, and the corresponding pipeline time is expressed as but The value of is a multiple of the average local processing time of the inference task, and the default value is 3.

[0106] Example 2

[0107] In some embodiments of the present application, a computer device is also provided, including a memory and a processor, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the processor performs the steps of the reasoning and decision-making method based on model partitioning and transfer learning as described in any one of Example 1.

[0108] The present invention also provides a storage medium storing computer-readable instructions, which, when executed by one or more processors, enables the one or more processors to perform the steps of the reasoning and decision-making method based on model partitioning and transfer learning as described in any one of Example 1.

[0109] It is understandable that, for the aforementioned reasoning and decision-making methods based on model partitioning and transfer learning, if they are all implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention is essentially or part of the contribution to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer server, or a network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program codes.

[0110] Computer readable storage media may include data signals propagated in baseband or as part of a carrier wave, wherein readable program codes are carried. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The readable storage medium may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, device, or device. The program codes contained on the readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination thereof.

[0111] The above is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should also be regarded as the protection scope of the present invention.

Claims

1. The reasoning decision method based on model partitioning and transfer learning is characterized by: The following steps are involved: S1: deploy the DNN model and generate a state value, input the state value and the training set into the DNN model, determine whether to redeploy the DNN model, and if so, load the neural network parameters from the last deployed DNN model, initialize the fully connected layer parameters and input them into the encoding network, and input the model parameters including the neural network parameters and the fully connected layer parameters into the model cache; S2: sampling training data from the model cache, and calculating cumulative advantage, loss function value and gradient descent update parameters according to the training data; S3: Perform training according to the accumulated advantage, the loss function value and the gradient descent update parameter until the training is completed after reaching a preset training round, and save the network parameters of the training model when the training is completed. If the training is not completed, return to step S2; S4: After inputting the test set into the training model, a decision result is obtained.

2. The reasoning and decision-making method based on model partitioning and transfer learning according to claim 1, characterized in that: In step S1, the deployment of the DNN model includes: S11: Through the observation space O n Get each mobile device observation, where the observation space O n for and The Cartesian product of three subsets, where The computing conditions of the mobile device, including the type of DNN model, data to be inferred, preprocessing delay, and the interruption status of the previous slot, Communication related information including signal strength, location, movement and belonging edge node, edge node calculation conditions including node type and current occupancy level; S12: encoding the mobile device observation by an encoder of a neural block to obtain an encoded observation, wherein the neural block includes two fully connected layers using a LeakyReLU activation function; S13: inputting the coded observation into a memory drive module, and outputting a read result and a new memory result, wherein the memory drive module includes a memory block for recording and maintaining the most recent observation, a read block for receiving the coded observation, and a write block for receiving the old memory result; S14: After the head module splices the encoded observation, the read result and the new memory result, a sub-decision is generated through probability sampling, and the critic network generates the state value V(o t ,m t ) to perform a training process, wherein the head module includes a fully connected layer using a LeakyReLU activation function and a fully connected layer using a Softmax activation function.

3. The reasoning and decision-making method based on model partitioning and transfer learning according to claim 2, characterized in that: In step S13, the coding observation is input into the memory driving module, and the coding observation result and the old memory result are output, further comprising: In the set time period, the reading block obtains the reading result h according to the encoding observation and the latest observation of the memory block. n,L : in, represents the Hadamard product, W n,1 ,W n,2 is the learning weight in the read block, g n,t is the coded observation for the set time period, m n,t is the most recent observation for the set time period; During the set time period, the write block generates a new memory result based on the encoding observation and the latest observation of the memory block. And update the memory block according to the new memory result, the new memory result The calculation formula is as follows: Among them, W n,3 ,W n,4 ,W n,5 are the learning weights in the write block.

4. The reasoning and decision-making method based on model partitioning and transfer learning according to claim 3 is characterized in that: In step S14, after the head module splices the coded observation, the read result and the new memory result, a sub-decision is generated by probability sampling, further comprising: Aggregate the memory results of all mobile devices through the average formula to form the new memory content of each mobile device, and then generate sub-decisions through probability sampling; The average formula is as follows: Among them, the To remember the results.

5. The reasoning and decision-making method based on model partitioning and transfer learning according to claim 4, characterized in that: In step S1, the state value and the training set are input into the DNN model, and it is determined whether to redeploy the DNN model, further comprising: For the current observation of mobile device n in time period t, through the reward function r n,t Make a decision, reward function r n,t The calculation formula is as follows: in, is the local processing time, that is, the entire task is completely processed locally, is the pipeline time of device n in time period t; Among them, for the prefetch mechanism, if the current slot is less than τ+1, the strategy of each mobile device is to process completely locally, and the function r n is a continuous function, the maximum value of the decision value is 1. The minimum decision value is -4. when When , the decision value is 0.

6. The reasoning and decision-making method based on model partitioning and transfer learning according to claim 5, characterized in that: In step S2, the loss function value is calculated according to the training data, further comprising: The small batch samples in the training set are input into the DNN model, and the estimated value of the cumulative advantage of the action is obtained according to the state value. The proximal strategy optimization objective function obtains the loss function value according to the estimated value and the training set, as shown below: Among them, E t [·] is the expected value from a mini-batch of samples, clip((∈)) is a pruning function with hyperparameters ∈, π θ is a policy with training parameters θ, θ old is the parameter before training update, ent((·)) represents the entropy reward to encourage exploration, ρ is a hyperparameter, is an estimate of the cumulative advantage of actions in a relative quantitative time period t, amoot(L1(·)) is a smoothed L1 loss function, is the cumulative reward starting from time period t.

7. The reasoning and decision-making method based on model partitioning and transfer learning according to claim 6, characterized in that: Obtaining an estimated value of the cumulative advantage of the action according to the state value further includes: An estimate of the cumulative advantage of actions over a period of time t based on the relative quantification of the state values The calculation formula is as follows: Among them, γλ is a hyperparameter, and λ∈[0,1], V(·) is the state value function during training, that is, the critic network, r t+i is the reward obtained in time period t+i, o t+i is the observation obtained in time period t+i, m t+i It is the memory content obtained in time period t+i.

8. The reasoning and decision-making method based on model partitioning and transfer learning according to claim 7, characterized in that: In step S4, the test set is input into the training model to obtain a decision result, which further includes: After the test set is input into the training model, an action decision is obtained. The action decision A n for The Cartesian product of three subsets, where Including migration decisions of edge nodes, comprising sub-decisions of the training model partitioning decision, Contains the decision of data compression, which is used to indicate that the transmitted data is compressed according to the compression ratio.

9. A computer device, characterized in that: It includes a memory and a processor, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the processor executes the steps of the reasoning and decision-making method based on model partitioning and transfer learning as described in any one of claims 1 to 8.

10. A storage medium storing computer-readable instructions, characterized in that: When the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the reasoning and decision-making method based on model partitioning and transfer learning as described in any one of claims 1 to 8.

Images