Distributed deduction method and system
By allocating deep neural network deduction tasks between edge devices and cloud servers, the problems of high resource consumption, large latency and low prediction accuracy when performing machine learning tasks under wireless connection edge devices are solved, and an efficient and flexible distributed deduction method under dynamic environmental conditions is realized.
Patent Information
- Application Number
- CN202280098278.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2022-09-06
- Publication Date
- 2025-03-07
AI Technical Summary
The prior art faces the problems of high resource consumption, large latency and low prediction accuracy when performing machine learning tasks on wirelessly connected edge devices, especially under dynamic channel conditions.
The distributed deduction method is used to divide the deep neural network into local strata and remote strata. Through the wireless connection between mobile devices and server devices, it dynamically decides whether to perform local deduction or remote deduction, and provides the final prediction result through the combination of multiple DNN deduction functions.
It realizes balancing prediction accuracy, latency and energy consumption under dynamic environmental conditions, is suitable for various machine learning tasks, and improves the flexibility and practicality of distributed systems.
Smart Images
Figure CN119585743A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates generally to the field of machine learning, and to a distributed inference method and system over a wireless connection. Background Art
[0002] Machine learning (ML) and deep learning (DL) have achieved breakthrough progress in various challenging applications such as image classification, computer vision, and natural language processing.
[0003] However, DL models deployed to support ML tasks are often resource-intensive, and large DL models can easily consume energy and time. Therefore, today's standard practice is to rely on cloud computing to provide these services. For example, when an edge device needs to perform an ML task (e.g., inference, prediction) on a data sample, the edge device sends the sample to a cloud server, where it is processed by a powerful large DL model, and the output of the model is then returned to the edge device.
[0004] The emerging need to perform ML tasks on edge devices wirelessly connected to the cloud poses new challenges due to the dynamic nature of wireless channels and raises the question of whether the inference should be performed in the cloud (remote inference) or on the mobile device (local inference). Local inference on the mobile device avoids the round-trip wireless transmission to the cloud, thus achieving low latency. However, it is impractical to deploy large models in energy-constrained devices (the battery will also drain quickly), so local inference may reduce prediction accuracy. Remote inference on the cloud server can support significantly larger DL models, but at the cost of increased response latency. This becomes particularly important in wireless networks due to the dynamic nature of channel conditions. Not only is the inference time random and can be long, but the transmission can also be very energy-intensive if the channel conditions are not good. As both approaches have obvious drawbacks, distributed inference has emerged as a promising intermediate and flexible approach, in which the Deep Neural Network (DNN) is split into a local layer on the edge device and a remote layer on the cloud server. An important point is that the device has the ability to provide output predictions based on itself, which is called local inference.
[0005] During the inference process, for each new data sample x, it is necessary to decide whether the prediction should be offloaded to the cloud using the full DNN (local and remote layers), or whether the local hardware is sufficient to provide good results in local inference. If local inference is chosen, only the local model is executed, and the final inference result is the local prediction; no bits are sent to the cloud server over the wireless channel, and it is not known what might happen if all DNN layers are used. In other cases, the sample first passes through the local layer, and then the output of one of the DNN layers of the sample is sent to the cloud server over the wireless channel, where it is used as the input of the DNN layer in the cloud. The remote inference results of the complete DNN are returned to the edge device again over the wireless channel.
[0006] Despite the flexibility and practicality of this setup, there are still some key issues that need to be addressed. First, the DNN should be trained to provide good / relevant outputs in all its inferences. Since the device and the cloud server can be far apart, both centralized and distributed training can be used. Second, the edge device needs to make an offloading decision for each sample, which can only rely on the local layers and their outputs of the edge device. Third, the offloading decision must be application-agnostic so that it can handle a variety of tasks, etc. One problem that arises in the distributed DNN setting is that the standard way to characterize the quality of a local inference (and therefore decide whether a remote layer should be used) is its uncertainty. Nonetheless, the concept of prediction entropy (using probability masses of different classes to represent uncertainty) is applicable to image classification, but not to time series problems, or regression problems in general. Summary of the invention
[0007] The present disclosure aims to overcome these and other disadvantages.
[0008] The above and other objects are achieved by the features of the independent claims. Other implementations are apparent from the dependent claims, the description and the drawings.
[0009] According to a first aspect, a distributed deduction method is provided. The method includes: processing data samples according to a first deep neural network (DNN) deduction function of a mobile device; starting local deduction and / or remote deduction according to a second DNN deduction function of the mobile device; forwarding the processed data samples according to the started local deduction and / or remote deduction; determining the local deduction result of the started local deduction according to the third DNN deduction function of the mobile device and the processed data sample; determining the remote deduction result of the started remote deduction according to the fourth DNN deduction function of the server device and the processed data sample; and providing the final result of the started local deduction and / or remote deduction according to the fifth deduction function.
[0010] First, in the default power-saving mode, the mobile device receives observations from its environment that help the mobile device understand its surroundings. These observations are critical because distributed inference involves wireless communication between the mobile device and the server device, and this wireless communication exhibits randomness and unpredictability. As a result, the mobile device can perform actions that strike a balance between prediction accuracy, experienced delay / latency, and / or energy consumption. Therefore, the distributed inference approach addresses the problem of dynamic environmental conditions.
[0011] Second, in an alternative precision mode, more (local and remote) forecasts are available, and these can be combined so that the final forecast is in principle better than the two initial forecasts. Thus, a distributed inference approach can improve forecast accuracy.
[0012] Third, the “offloading” of the mobile device 2 is task-agnostic. Since no problem-specific aspects are used, the logic and overall architecture are able to solve any given ML task. Therefore, the distributed inference approach is applicable to a wide variety of ML problems.
[0013] As used herein, “deep neural network” may refer to an artificial neural network architecture that includes multiple layers between the input and output layers and is applicable to a broader family of machine learning methods known as deep learning.
[0014] As used herein, an "inference function" may refer to a function that predicts an output data sample based on an input data sample and based on a model of the relationship between the input data sample and the output data sample. With respect to a DNN, an inference function may refer to the inference phase of the DNN, in which predictions may be inferred based on the input data sample and the DNN-based model of the relationship between the input data sample and the output data sample. The DNN-based model is typically acquired in a previous training phase of the DNN.
[0015] As used herein, "local deduction" may refer to calling a deduction function in an edge device (e.g., a mobile device) that has strict limitations in processing power and energy storage capabilities. The result of the local deduction may be referred to as a "local deduction result."
[0016] As used herein, "remote deduction" may refer to calling a deduction function of a remote device (e.g., a server device) that has fewer strict limitations on processing power and energy storage capabilities than an edge device. The results of the remote deduction may be referred to as "remote deduction results."
[0017] As used herein, “distributed gaming” may refer to a combination of local gaming and remote gaming. The results of distributed gaming may be referred to as “final gaming results”, which depend on the “local gaming results” of local gaming and the “remote gaming results” of remote gaming.
[0018] In one possible implementation, starting the local deduction and / or the remote deduction may include: if a high-precision indication (of the deduction) is set, providing a corresponding local deduction indication and a remote deduction indication; if a high-precision indication is not set, providing a corresponding local deduction indication and a remote deduction indication according to the state of the second DNN deduction function, the data sample and the environment of the mobile device.
[0019] In a possible implementation manner, the data sample may include one of the following: a digital image, and a part of a time series.
[0020] As used herein, a “digital image” may refer to a finite set of digital values (e.g., picture elements or “pixels”) arranged in a fixed number of rows and columns, each of which represents the brightness of a given color at any particular point within the arrangement (e.g., the intersection of a row and column).
[0021] As used in this article, "time series" may refer to a series of data samples listed in chronological order.
[0022] In a possible implementation, the state of the environment of the mobile device may include one or more of the following: a channel measurement result of a wireless connection between the mobile device and the server device; and a load indication of the distributed deduction.
[0023] As used herein, a "wireless connection" may refer to at least one-way coded information transmission via wireless / radio communication. For example, a wireless connection may include a 4G or 5G cellular connection or a Wi-Fi / WLAN connection.
[0024] In a possible implementation manner, the channel measurement result may include a channel quality indicator.
[0025] As used herein, a “channel quality indicator” may be an indication of the quality of a communication channel, such as a signal-to-noise ratio (SNR), a signal-to-interference plus noise ratio (SINR), a bit error rate (BER), etc.
[0026] In a possible implementation, the load indication may include one or more of the following: a queue length of a data sample queue of the mobile device, wherein the queue includes the data sample; a status of the server device. For example, the status of the server device may include an indication that the server device successfully (unsuccessfully) remotely simulates the remote simulation result, an indication that the server device 3 makes a positive (ACK) or negative (NACK) confirmation in response to data packet transmission via a wireless connection, an indication that the mobile device 2 or the server device 3 has timed out data packet transmission via a wireless connection, and the like.
[0027] In one possible implementation, forwarding the processed data sample may include: if the local deduction indication is set, forwarding the processed data sample to the third DNN deduction function of the mobile device.
[0028] In a possible implementation, forwarding the processed data sample may include: if the remote deduction indication is set, sending the processed data sample to the server device via a wireless connection.
[0029] In a possible implementation, providing the final result may include: if the high-precision indication is not set and if the local deduction indication is set, providing the final result of the initiated local deduction according to the fifth deduction function and the local deduction result of the local deduction; if the high-precision indication is not set and if the remote deduction indication is set, providing the final result of the initiated remote deduction according to the fifth deduction function and the remote deduction result of the initiated remote deduction received from the server device through the wireless connection; or if the high-precision indication is set, providing the final results of the initiated local deduction and remote deduction according to the fifth deduction function, the local deduction result of the initiated local deduction, and the remote deduction result of the initiated remote deduction received from the server device through the wireless connection.
[0030] In one possible implementation, the first, third, and fourth DNN inference functions may respectively include convolutional neural networks or fully connected feedforward neural networks.
[0031] As used herein, a "feedforward neural network" may refer to an artificial neural network architecture that typically includes a recurrent feedforward structure of one or more layers in which the connections between nodes do not form a cycle.
[0032] As used herein, "convolutional neural network (CNN)" may refer to a feed-forward neural network that is most commonly used to analyze digital images and typically includes a recurrent structure of one or more convolutional layers followed by pooling layers. The convolutional layers extract features from data samples (e.g., two-dimensional digital images), while the pooling layers reduce the redundancy of the extracted features by merging the outputs of a cluster of neurons in one layer into a single neuron in the next layer.
[0033] In a possible implementation, the method may further include: performing joint training of the first, third, and fourth DNN inference functions based on supervised learning.
[0034] In one possible implementation, executing the joint training of the first, third, and fourth DNN inference functions may include: calculating the joint loss of the initiated local inference and remote inference based on the target result associated with the data sample, the local inference result, and the remote inference result; and updating the corresponding weight parameters of the first, third, and fourth DNN inference functions based on the backpropagation technique and the joint loss.
[0035] In a possible implementation, calculating the joint loss may include: calculating the joint loss of the initiated local deduction and remote deduction according to the target result associated with the data sample and an average value of the local deduction result and the remote deduction result.
[0036] In one possible implementation, calculating the joint loss may include: calculating a first loss of the local deduction result of the initiated local deduction based on the weight parameters of the first DNN deduction function, the weight parameters of the third DNN deduction function, the target result associated with the data sample, and the local deduction result; calculating a second loss of the remote deduction result of the initiated remote deduction based on the weight parameters of the first DNN deduction function, the weight parameters of the fourth DNN deduction function, the target result associated with the data sample, and the remote deduction result; and calculating the joint loss of the initiated local deduction and remote deduction based on the first loss and the second loss.
[0037] In a possible implementation, calculating the joint loss may include: calculating the joint loss according to a linear combination of a first loss and a second loss, wherein the linear combination may be based on a loss weight of the first loss and a loss weight of the second loss.
[0038] In one possible implementation, calculating the joint loss may include: calculating the joint loss based on a mean squared error (MSE) loss function or a cross entropy loss function, the local deduction result, the remote deduction result, and the target result associated with the data sample.
[0039] In one possible implementation, the second DNN deduction function may include a policy deep Q-network (DQN), a target DQN, and a state transition memory. The policy DQN and the target DQN may be used to provide a corresponding first Q value associated with a local deduction and a corresponding second Q value associated with a remote deduction. The target DQN may also be used to copy weight parameters from the policy DQN every K training steps.
[0040] In a possible implementation, the method may further include: performing training of the second DNN deduction function according to reinforcement learning (RL).
[0041] In one possible implementation, executing the training of the second DNN deduction function may include: determining the first Q value and the second Q value of the policy DQN according to the state of the environment of the mobile device before performing the operation; starting the local deduction or the remote deduction according to the smaller one of the determined first Q value and the determined second Q value; observing one or more of the following: the delay from starting to process the data sample to completing the determination of the final result, the penalty for the queue length of the data sample queue, the first loss of the local deduction result of the started local deduction, and the started remote deduction. the second loss of the remote deduction result; determining the cost associated with the initiated local deduction or remote deduction; filling the state transition memory with a tuple, wherein the tuple includes the state before performing the operation, the operation, the cost associated with the operation, and the state of the environment of the mobile device after performing the operation; extracting a batch of training samples from the state transition memory; updating the weight parameters of the second DNN deduction function according to the back-propagation technique and the prediction errors of the policy DQN and the target DQN of the second DNN deduction function for the extracted batch of training samples.
[0042] In one possible implementation, starting the local deduction or the remote deduction may include: if the first Q value of the strategy DQN associated with the local deduction is less than or equal to the second Q value of the strategy DQN associated with the remote deduction, starting the local deduction; if the first Q value of the strategy DQN associated with the local deduction is greater than the second Q value of the strategy DQN associated with the remote deduction, starting the remote deduction.
[0043] In one possible implementation, determining the cost associated with the initiated local deduction or remote deduction may include: if the local deduction is initiated, determining the cost based on the sum of the first loss, the delay, and the penalty; if the remote deduction is initiated and if the change in state is related to unsuccessful remote deduction of the remote deduction result by the server device, determining the cost based on the sum of the delay and the penalty; if the remote deduction is initiated and if the change in state is related to successful remote deduction of the remote deduction result by the server device, determining the cost based on the sum of the second loss, the delay, and the penalty.
[0044] In a possible implementation, during the life cycle of the data sample, the state of the environment may change once or multiple times.
[0045] In one possible implementation, the fifth inference function may include a fifth DNN inference function, and the fifth DNN inference function includes a convolutional neural network or a fully connected feedforward neural network.
[0046] In a possible implementation, the method may further include: performing training of the fifth DNN deduction function based on supervised learning.
[0047] In one possible implementation, executing the training of the fifth DNN inference function may include: calculating the loss based on the weight parameters of the fifth DNN inference function, the target result associated with the data sample, and the final result; and updating the weight parameters of the fifth DNN inference function based on the backpropagation technique and the loss.
[0048] In a possible implementation manner, calculating the loss may include: calculating the loss according to the MSE loss function, the target result, and the final result.
[0049] In a possible implementation, calculating the loss may include: calculating the loss according to a cross entropy loss function, the target result, and the final result.
[0050] According to a second aspect, a distributed deduction system is provided, wherein the system comprises a mobile device and a server device, and is configured to execute the method according to any one of the above claims.
[0051] According to a third aspect, a computer program is provided, wherein the computer program comprises program code, and when the program code is executed on a computer, the computer program is used to execute the method according to the first aspect or any one of its implementations. BRIEF DESCRIPTION OF THE DRAWINGS
[0052] The above aspects and implementations are explained below with reference to the accompanying drawings, wherein the same or similar reference numerals represent the same or similar elements.
[0053] The drawings should be considered schematic representations, and the elements shown in the drawings are not necessarily shown to scale. Rather, various elements are shown so that their function and general purpose are apparent to those skilled in the art.
[0054] Figure 1 A block diagram of a distributed deduction system 2 provided by the present disclosure is shown;
[0055] Figures 2 to 6 Shows Figure 1 Exemplary implementations of various deduction functions of the system in;
[0056] Figure 7 A flow chart of a distributed deduction method provided by the present disclosure is shown;
[0057] Figure 8 Shown in more detail Figure 7 Step 15 of the method;
[0058] Fig. 9 Shown in more detail Figure 7 Step 16 of the method;
[0059] Fig.10 Shown in more detail Figure 7 Step 19 of the method;
[0060] Fig.11 Shown in more detail Figure 7 Step 11 of the method;
[0061] Fig.12 Shown in more detail Figure 7 Step 12 of the method;
[0062] Fig.13 Shown in more detail Figure 7 Step 13 of the method. DETAILED DESCRIPTION
[0063] In the following description, reference is made to the accompanying drawings that form a part of the present disclosure, which illustrate specific aspects of the implementation of the present disclosure or specific aspects of the implementation of the present disclosure by way of illustration. It is understood that the implementation of the present disclosure can be used in other aspects and include structural changes or logical changes that are not depicted in the accompanying drawings. Therefore, the following detailed description should not be understood in a restrictive sense, and the scope of the present disclosure is limited by the appended claims.
[0064] For example, it is understood that the disclosure related to describing a method may also be applicable to a corresponding device or system for performing the method, and vice versa. For example, if one or more specific method steps are described, the corresponding device may include one or more units (e.g., functional units) to perform the described one or more method steps (e.g., one unit performs one or more steps, or multiple units perform one or more steps of multiple steps respectively), even if such one or more units are not explicitly described or illustrated in the drawings. On the other hand, for example, if a specific device is described based on one or more units (e.g., functional units), the corresponding method may include a step to perform the function of one or more units (e.g., one step performs the function of one or more units, or multiple steps perform the function of one or more units of multiple units respectively), even if such one or more steps are not explicitly described or illustrated in the drawings. In addition, it is understood that, unless otherwise explicitly stated, the features of the various exemplary implementations and / or aspects described herein may be combined with each other.
[0065] Figure 1 The block diagram of the distributed deduction system 2 and 3 provided by the present disclosure is shown. Figures 2 to 6 Shows Figure 1 An exemplary implementation of various deduction functions 21, 22, 24, 25, 31 of the systems 2 and 3.
[0066] Figure 1 Divided into three parts: input (on the left), mobile device 2 (in the middle) and server device 3 (on the right).
[0067] The input comprises a data sample x, a state S of the environment of the mobile device 2 and an optional inferred high-precision indication m.
[0068] Data sample x represents an input data sample for which mobile device 2 needs to return a prediction. State S can be any set of observed measurements, such as (i) (e.g., lossy) channel state, (ii) device buffer state (e.g., number of data samples x in the queue), and (iii) server state (e.g., very busy). Indication m can indicate high precision of the derivation when set (m=1), or low precision of the derivation when not set (m=0).
[0069] The systems 2 and 3 include a mobile device 2 and a server device 3, and are used to execute the method 1 described in the first aspect (see below). Figure 7 ) or any implementation thereof.
[0070] The mobile device 2 includes a first deep neural network (DNN) to deduce a function F 0 21, the function is used to process 14 the data sample x into the processed data sample z. The first DNN deduces the function F 0 21 can include convolutional neural networks or fully connected feedforward neural networks (see Figure 3 ).
[0071] The data sample x may include one of the following: a digital image, and a part of a time series.
[0072] The state S of the environment of the mobile device 2 mentioned above may include one or more of the following: a channel measurement result of the wireless connection 4 between the mobile device 2 and the server device 3, for example, a channel quality indication c∈{1,…,C}; a load indication of distributed deduction, for example, one or more of the following: a queue length q∈{0,1,…,K} of a data sample queue of the mobile device 2 including the data sample x; a state of the server device 3. During the life cycle of the data sample x, the state S=[c,q] of the environment may change once or multiple times.
[0073] The mobile device 2 also includes a second DNN derivation function F C 22, the function is used to start 15 local deduction and / or remote deduction according to the data sample x, the state S of the environment of the mobile device 2 and the optional high-precision indication m of the deduction to be started. The second DNN deduction function F C 22 can include in particular the strategy DQN (see Figure 2 ), target DQN (same Figure 2 ) and state transition memory B C (not shown). The policy DQN and the target DQN may be used to provide a corresponding first Q-value Q associated with the local deduction 1 (x, S) and the corresponding second Q value Q associated with the remote derivation 2 (x,S). The policy DQN and target DQN usually correspond to each other. However, the target DQN can also be used to copy the weight parameters θ from the policy DQN every K training steps C For example, the weight parameter θ of the target DQN C It can be kept constant over K=1000 training steps of policy DQN.
[0074] The mobile device 2 also includes a forwarding function 23, which is used to initiate a local deduction (R 1 =1) and / or remote simulation (R 2 =1), forwarding 16 the processed data sample z to the third DNN derivation function F of the mobile device 2 1 24 (for local deduction) and / or the fourth DNN deduction function F of the server device 3 2 31 (for remote deduction). In other words, the second DNN deduction function F C 22 is used to decide where the forwarding function 23 sends the processed data sample z. The forwarding function 23 may comprise a logic circuit in particular.
[0075] In energy-saving mode (m=0, default mode), the second DNN deduces function F C 22 can call local deduction (R 1 =1) or remote deduction (R 2 =1). In precision mode (m=1), the second DNN deduction function F is bypassed. C 22, and at the same time call the local deduction (R 1 =1) and long-range simulation (R 2 =1).
[0076] The mobile device 2 also includes a third DNN derivation function F 1 24, this function is used to determine the local deduction (R 1 =1) of the local deduction result y 1 The third DNN deduces the function F 1 24 can include convolutional neural networks or fully connected feedforward neural networks (see Figure 4 ).
[0077] The server device 3 includes a fourth DNN derivation function F 2 31, this function is used to determine the long-range deduction (R 2 =1) of the remote deduction result y 2 The fourth DNN deduced function F 2 31 In particular, it may include a convolutional neural network or a fully connected feedforward neural network (see Figure 5 ).
[0078] The mobile device 2 also includes a fifth derivation function F G 25, the function may include a fifth DNN deduction function F G 25, the fifth DNN deduces function F G 25 for local deduction based on the startup (R 1=1) of the local deduction result y 1 and / or initiated remote gaming (R 2 =1) of the remote deduction result y 2 , providing the final results of the local and / or remote games initiated at 19 final The fifth DNN deduced function F G 25 can include convolutional neural networks or fully connected feedforward neural networks (see Figure 6 ).
[0079] Therefore, in the energy-saving mode (m=0), the fifth (DNN) deduces the function F G 25 can be used to receive a prediction (e.g., local derivation result y 1 And the remote deduction result y 2 and outputs the prediction as the final result y final In precision mode (m=1), the fifth (DNN) deduces the function F G 25 Receives two predictions and applies a trainable function to provide the final result y final .
[0080] Figure 7 A flow chart of the distributed deduction method 1 provided in the present disclosure is shown.
[0081] Method 1 may include executing 11 first, third, and fourth DNN derivation functions F according to supervised learning 0 21. F 1 24. F 2 31 joint training. Since the first and third DNNs deduce function F 0 21. F 1 24 constitutes a part of the mobile device 2, and the fourth DNN deduces the function F 2 31 constitutes a part of the server device 2, so that centralized training can be used (indicated by the dashed horizontal arrows between the steps marked as 11 on the mobile device 2 and the server device 3; accordingly, the model is trained on the server device 3 and θ is obtained 0 ,θ 1 ,θ 2 Afterwards, the weight parameter θ is shared with mobile device 2 0 ,θ 1 ), distributed training may also be used (also indicated by the dashed horizontal arrow between the steps labeled 11 on the mobile device 2 and the server device 3), because the mobile device 2 and the server device 3 may be far apart.
[0082] The term “joint training” used in this article may refer to the process of binding the first, third, and fourth DNN deduction functions F 0 21. F1 24. F 2 31 training methods.
[0083] Method 1 may further include executing 12 a second DNN deducing function F according to reinforcement learning (RL). C 22 training.
[0084] Method 1 may further include executing 13 a fifth DNN deducing function F according to supervised learning again. G 25 training.
[0085] At this point, the training phase of method 1 (and systems 2 and 3) ends and the deduction phase begins.
[0086] Method 1 also includes deducing a function F according to a first deep neural network (DNN) of mobile device 2 0 21, process 14 data samples x.
[0087] Method 1 also includes inferring function F according to a second DNN of mobile device 2 C 22. Initiate 15 local and / or remote simulations.
[0088] The method 1 further comprises forwarding 16 the processed data sample z according to the initiated local and / or remote deduction.
[0089] Forwarding 16 may include forwarding 161 the processed data sample z in the mobile device 2 to the third DNN derivation function F 1 twenty four.
[0090] If so, the method 1 further includes inferring the function F according to the third DNN of the mobile device 2 1 24 and the processed data sample z, determine the local deduction result y of the local deduction started at 17 1 .
[0091] However, forwarding 16 may additionally or alternatively comprise sending 162 the processed data samples z to the server device 3 via the wireless connection 4 , as indicated by the dashed horizontal arrow from step 16 on the mobile device 2 to the server device 3 .
[0092] If so, the method 1 further comprises inferring the function F according to the fourth DNN of the server device 3 2 31 and the processed data sample z, determine the remote deduction result y of the remote deduction initiated by 18 2 .
[0093] Method 1 also includes the following step: G25. Local simulation result y of the local simulation started 1 And / or the remote game results of the initiated remote game 2 , providing the final results of the local and / or remote games initiated at 19 final .
[0094] Figure 8 Shown in more detail Figure 7 Follow the steps in step 15 of Method 1.
[0095] According to this improvement, starting 15 local and / or remote game play may include: if a high-precision indication m is set, providing 151 a correspondingly set local game play indication R 1 and long-range deduction instructions R 2 ; If the high-precision indication m is not set, the function F is deduced according to the second DNN C 22. The data sample x and the state S of the environment of the mobile device 2 provide 152 a corresponding local deduction indication R 1 and long-range deduction instructions R 2 .
[0096] Fig. 9 Shown in more detail Figure 7 Follow the steps in step 16 of Method 1.
[0097] According to this improvement, forwarding 16 the processed data sample z may include: if the local derivation indication R is set 1 , forwarding 161 the processed data sample z to the third DNN deduction function F of the mobile device 2 1 twenty four.
[0098] Alternatively or additionally, forwarding 16 the processed data sample z may include: if the remote derivation indication R is set 2 , the processed data sample z is sent 162 to the server device 3 via the wireless connection 4.
[0099] Fig.10 Shown in more detail Figure 7 Follow the steps in step 19 of Method 1.
[0100] According to this improvement, the final result y is provided. final May include: If the high precision indicator m is not set, and if the local derivation indicator R is set 1 , according to the fifth deduction function F G 25 and the local deduction result y of the local deduction 1 , providing the final result y of the local deduction started at 191 final ; If the high-precision indication m is not set, and if the long-range deduction indication R is set 2, according to the fifth deduction function F G 25 and the remote deduction result y of the initiated remote deduction received from the server device 3 via the wireless connection 4 2 , providing the final results of the long-distance game started by 192 final ; or if the high-precision indication m is set, according to the fifth deduction function F G 25. Local simulation result y of the local simulation started 1 and the remote deduction result y of the initiated remote deduction received from the server device 3 via the wireless connection 4 2 , providing the final results of the local and remote games started by 193 final .
[0101] Fig.11 Shown in more detail Figure 7 Follow the steps in Method 1, step 11.
[0102] According to this improvement, the first, third and fourth DNN deduction functions F are executed. 0 21. F 1 24. F 2 The joint training of 31 may include: according to the target result y associated with the data sample x, the local deduction result y 1 And the remote deduction result y 2 , calculate the joint loss of local and remote games started by 111 L = L(y,y 1 ,y 2 ); Based on the back propagation technique and the joint loss L, update 112 the first, third, and fourth DNN deduction functions F 0 21. F 1 24. F 2 The corresponding weight parameter θ of 31 0 ,θ 1 ,θ 2 .
[0103] The target result y represents the actual expected output (label).
[0104] According to further improvements, the joint loss L = L(y,y 1 ,y 2 ) may include: according to the target result y associated with the data sample x and the local deduction result y 1 And the remote deduction result y 2 The average Calculate the joint loss of the local game and the remote game started at 1111
[0105] Alternatively, the joint loss L = L(y,y1 ,y 2 ) may include: deducing a function F according to the first DNN 0 The weight parameter θ of 21 0 , the third DNN deduction function F 1 The weight parameter θ is 24 1 , the target result y associated with the data sample x and the local deduction result y 1 , calculate the local deduction result y of the local deduction started at 1112 1 The first loss L 1 =L(y,y 1 θ 0 ,θ 1 );According to the first DNN deduced function F 0 The weight parameter θ of 21 0 、The fourth DNN deduction function F 2 The weight parameter θ of 31 2 , the target result y associated with the data sample x, and the remote deduction result y 2 , calculate the remote deduction result y of the remote deduction started at 1113 2 The second loss L 2 =L(y,y 2 θ 0 ,θ 2 ); According to the first loss L 1 and the second loss L 2 , calculate the joint loss of the local game and the remote game started at 1114 L = f(L 1 ,L 2 ).
[0106] According to the further improvement, the joint loss L = L(y,y 1 ,y 2 ) may include: according to the first loss L 1 and the second loss L 2 Linear combination of 11141, calculate the joint loss L = w 1 L 1 +w 2 L 2 , where this linear combination is based on the first loss L 1 The loss weight w 1 and the second loss L 2 The loss weight w 2 For example, given some known loss weights w for each play 1 、w 2 And the deduced function F 0 、F 1 、F 2 The fixed weight parameter θ0 ,θ 1 ,θ 2 , each loss L 1 , L 2 The total loss is constructed in an additive manner (which can then be used for backpropagation) as follows: L = w 1 L(y,y 1 θ 0 ,θ 1 )+w 2 L(y,y 2 θ 0 ,θ 2 ).
[0107] It is worth noting that if w 1 = 0, then only θ is trained 0 ,θ 2 (Untrained θ 1 ), then this is a traditional DNN, which only generates remote deduction results y on server device 3 2 ; Technically, local deduction can still return local deduction results y 1 , that is, predictions, but these predictions are meaningless. In other extreme cases, that is, w 2 = 0, then only θ 2 It is never trained, and only local inferences return meaningful predictions.
[0108] Calculating the joint loss 1114 may also include: calculating the joint loss 1114 according to the MSE loss function or the cross entropy loss function, the local deduction result y 1 、Remote deduction results y 2 and the target result y associated with the data sample x, calculate 11142 the joint loss L = L(y,y 1 ,y 2 ). For example, the function L is chosen to be related to the ML task one is interested in (e.g., MSE loss function for regression).
[0109] Fig.12 Shown in more detail Figure 7 Follow the steps in step 12 of method 1.
[0110] According to this improvement, the second DNN deduction function F is executed 12 C The training of 22 may include: determining 121 a first Q value Q of the strategy DQN according to the state S = [c, q] of the environment of the mobile device 2 before the operation is performed, 1 (x,S) and the second Q value Q 2 (x,S); according to the determined first Q value Q 1 (x,S) and the determined second Q value Q 2The smaller one of (x, S) starts 122 local deduction (R = [R 1 ,R 2 ]=[1,0]) or long-range deduction (R=[R 1 ,R 2 ]=[0,1]); observe 123 one or more of the following: from the start of processing data sample x to the completion of determining the final result y final The delay d, the penalty g(q) for the queue length q of the data sample queue, and the local deduction result y of the local deduction started 1 The first loss L(y,y 1 θ 1 ) and the remote game result y of the remote game started 2 The second loss L(y,y 2 θ 2 ); determine 124 and start the local deduction (R 1 =1) or remote deduction (R 2 =1) associated cost; use the tuple {[x,S],R,cost,[x,S ′ ]} Fill 125 state transition memory B C , where the tuple {[x,S],R,cost,[x,S ′ ]} includes the state S = [c, q] before the operation is performed, the operation R, the cost associated with the operation R, and the state S′ = [c′, q′] of the environment of the mobile device 2 after the operation R is performed; from the state conversion memory B C 126 training samples are extracted from the training set; according to the back propagation technique and the second DNN deduction function F C The strategy DQN and target DQN of 22 update the prediction error of a batch of training samples by 127 and the second DNN deduction function F C The weight parameter θ is 22 C .
[0111] According to a further improvement, initiating 122 the local game or the remote game may include: if the first Q-value Q associated with the local game of the policy DQN 1 (x, S) is less than or equal to the second Q-value Q associated with the remote game of policy DQN 2 (x,S), start 1221 local deduction (R 1 =1); if the first Q-value Q associated with the local deduction of the policy DQN 1 (x,S) is greater than the second Q-value Q associated with the remote deduction of the policy DQN 2 (x,S), start 1222 remote deduction (R 2 =1).
[0112] According to a further refinement, determining 124 a cost associated with initiating a local game or a remote game may include: if a local game (R 1 =1), according to the first loss L(y,y 1 θ 1 ), the sum of delay d and penalty g(q), determines the cost 1241(cost=L(y,y 1 θ 1 )+d+g(q)); if the remote simulation (R 2 =1), and if the change in state is consistent with the remote deduction result y of the server device 3 2 The remote deduction is unsuccessful. According to the sum of the delay d and the penalty g(q), the cost 1242 (cost = d + g(q)) is determined. If the remote deduction (R 2 =1), and if the change in state is consistent with the remote deduction result y of the server device 3 2 The success of the long-range deduction is related to the second loss L(y,y 2 θ 2 ), the sum of delay d and penalty g(q), determines the cost 1243 (cost = L(y,y 2 θ 2 )+d+g(q)).
[0113] The key contribution of this disclosure includes the second DNN inference function F C 22, the training method of this function will be explained in this section. This function is used in the energy saving mode (m=0, default mode) of the mobile device 2. The training is obviously independent of the user-defined signal m, so the second DNN deduces the function F C The input data of 22 are data samples x and the state S of the environment of the mobile device 2.
[0114] According to the RL-based training method, the first, third, and fourth DNNs deduce the function F 0 21. F 1 24. F 2 31 constitutes part of the environment. The arriving data samples x are processed in a First Come First Served (FCFS) manner. In addition, at time t, the channel quality indication c∈{1,…,C} and the queue length q∈{0,1,…,K} of the data sample queue of mobile device 2 are observed; these constitute the state S=[c,q] of the environment of mobile device 2.
[0115] As mentioned above, the second DNN deduces the function F C22 can be represented as a DQN because (a) the state space ([x, S], where x can be an image) is very large and (b) the action space is small (only two possible actions). Therefore, the RL agent (of the RL training method) includes an experience buffer B C and two DNNs: target DNN and policy DNN. RL state is a tuple [x,S], and operation is “local execution” (R 1 =1) and Remote Execution (R 2 =1). Therefore, in the deduction of the strategy DQN, we will see two Q values Q 1 (x,S),Q 2 (x,S), choose the smaller one (e.g., the one with smaller cost). Therefore, if Q 1 (x,S)>Q 2 (x,S), then R=[R 1 ,R 2 ]=[0,1]; otherwise, R=[R 1 ,R 2 ]=[1,0].
[0116] During the RL-based training process, the environment will change. When mobile device 2 sends a data packet (e.g., processed data sample z) to server device 3 for processing (i.e., requesting prediction y from server device 3), 2 ), one of the following may occur:
[0117] (a) The packet is lost; or the server device 3 is busy; or the server device 3 responds with an ACK (successful receipt of packet z) but returns a prediction y 2 too late, e.g. exceeding an acceptable timeout period (in all these cases, the timeout flag is triggered);
[0118] (b) the server device 3 erroneously receives a data packet sent by the mobile device 2 (the server sends a NACK to the device), or the mobile device 2 erroneously receives a response from the server device 3 (e.g., by performing a parity check), in which case an error flag is triggered;
[0119] (c) Server device 3 successfully receives z, sends an ACK to the device, and server device 3's prediction arrives at mobile device 2 on time (e.g., before timeout).
[0120] In cases (a) and (b), mobile device 2 needs to make a new decision, that is, how to continue processing data sample x when the environmental state S = [c, q] (q increases, c changes) may be different; in case (c), mobile device 2 successfully receives the remote deduction result from server device 3.
[0121] The reward is determined as follows: When any of the above events occurs, it is observed that from the beginning of processing data sample x to the completion of determining the final result y final The delay d and the penalty g(q) for the queue length q of the data sample queue. When the prediction y (of the derivation i that is required to provide the prediction) i When available, the corresponding loss L(y,y i ). The operator also receives data samples x and environment state S (RL state is [x, S]), and outputs operation R = [R 1 ,R 2 ], receives the result of its operation, i.e., the value cost, and then transitions to the new environment state [x′, S′]. State transition memory B C Store RL tuples {[x,S],R,cost,[x ′ ,S ′ ]}.
[0122] The cost of operation R depends on the same conditions:
[0123] -When R = [1,0], get the local prediction y 1 ,cost=L(y,y 1 )+d+f(q)
[0124] - When R = [0, 1], request remote prediction y from server device 3 2 , the following events may occur:
[0125] ○ "Timeout" or "Error Flag" or "NACK", for example, in the above cases (a) and (b): cost = d + f(q)
[0126] ○ "Server Prediction 2 Successful return": cost = L(y,y 2 )+d+f(q)
[0127] A given data sample x may go through multiple states S of the environment until the data sample is finally processed (which means that the environment can change during the life cycle of the data sample x).
[0128] After each state transition S→S′, the state transition memory B C A batch of samples are extracted from the dataset and the second DNN deduction function F is updated according to the standard back-propagation technique. C The weight parameter θ is 22 C .
[0129] New data samples x can be obtained from the data sample queue to repeat the RL-based training process.
[0130] The training process is usually not performed online: as mentioned above, the reward consists of L(y,y i )(where i=1 or 2)); therefore, measuring the loss requires knowing the target value y (ground truth). In the case of image classification, this cannot be achieved in an online manner, as this would imply knowing the true labels of new, unseen images during training, which is unrealistic. Therefore, in the case of image classification, training is performed in an offline manner, and during testing, the DQN does not observe the loss and therefore does not update it. However, when considering time series related tasks, online training can also be performed. In such tasks, given some past time samples, some prediction is made for the next time instance. After making the prediction, the true value of the predicted variable is available, so the loss can be measured and stored in the state transition memory B C And can perform online DQN training.
[0131] Fig.13 Shown in more detail Figure 7 Follow the steps in Method 1, step 13.
[0132] According to the improvement, the fifth DNN deduction function F is executed in 13 G The training of 25 may include: deducing the function F according to the fifth DNN G The weight parameter θ is 25 G , the target result y associated with the data sample x, and the final result y final , calculate 131 loss L G =L(y,y final θ G ); Based on the back-propagation technique and the loss L G , update 132 fifth DNN deduction function F 0 The weight parameter θ of 21 G .
[0133] According to a further improvement, calculating 131 the loss may include: according to the MSE loss function, the target result y and the final result y final , calculate 1311 loss
[0134] Alternatively, calculating 131 the loss may include: according to the cross entropy loss function, the target result y and the final result y final , calculate 1312 loss L = CrossEntropy(y,y final ).
[0135] In precision mode (m=1), the fifth DNN deduces function F G 25 Receive two predictions y from two derivations 1 ,y2 , these predictions are used to train F G To get the final result y using these two predictions final .
[0136] In form, the final prediction is y final =F G (y 1 ,y 2 ), for a given set of weight parameters θ G , the measured loss is Loss = L(y,y final θ G ).
[0137] For example, time series tracking might involve Image classification may involve L(y,y final θ G )=CrossEntropy(y,y final )=CrossEntropy(y,F G (y 1 ,y 2 )).
[0138] The present disclosure has been described in conjunction with various implementations as examples and implementations. However, based on a study of the drawings, the present disclosure, and the independent claims, those skilled in the art will be able to understand and implement other variations when implementing the claimed subject matter. In the claims and the specification, the word "comprising" does not exclude other elements or steps, and the quantifier "one" or "an" does not exclude a plurality. A single element or other unit may satisfy the functions of several entities or items described in the claims. The listing of certain measures in mutually different dependent claims does not mean that the combination of these measures cannot be used in an advantageous implementation. A computer program may be stored / distributed in a suitable medium, for example, an optical storage medium or solid-state medium provided together with other hardware or as part of other hardware, and may also be distributed in other forms (for example, via the Internet or other wired or wireless communication systems).
Claims
1. A distributed deduction method (1), the method (1) comprising: Processing (14) a data sample (x) according to a first deep neural network DNN deducing function (F0 21) of the mobile device (2); According to the second DNN derivation function (F C 22), start (15) local and / or remote simulations; forwarding (16) processed data samples (z) according to the initiated local simulation and / or remote simulation; Determining (17) a local deduction result (y1) of the initiated local deduction according to the third DNN deduction function (F1 24) of the mobile device (2) and the processed data sample (z); Determining (18) a remote deduction result (y2) of the initiated remote deduction according to the fourth DNN deduction function (F2 31) of the server device (3) and the processed data sample (z); According to the fifth deduction function (F G 25), providing the final result of the local simulation and / or remote simulation initiated in (19) (y final ).
2. The method (1) according to claim 1, wherein: Initiating (15) the local simulation and / or the remote simulation includes: If the high-precision indication (m) is set, providing (151) the correspondingly set local deduction indication (R1) and remote deduction indication (R2); If the high-precision indication (m) is not set, according to the second DNN deduction function (F C 22), the data sample (x) and the state (S) of the environment of the mobile device (2), providing (152) the corresponding local deduction indication (R1) and the remote deduction indication (R2).
3. The method (1) according to claim 2, wherein: The data sample (x) comprises one of: a digital image, and a portion of a time series; 4. The method (1) according to claim 2 or 3, wherein the state (S) of the environment of the mobile device (2) comprises one or more of the following: a channel measurement result of a wireless connection (4) between the mobile device (2) and the server device (3); and An indication of a load of the distributed game.
5. The method (1) according to claim 4, wherein the channel measurement result comprises a channel quality indicator (c∈{1, ..., C}).
6. The method (1) according to claim 4 or 5, wherein the load indication comprises one or more of the following: The queue length (q∈{0,1,…,K}) of the data sample queue of the mobile device (2) is, The queue includes the data samples (x); and The status of the server device (3).
7. The method (1) according to any one of the preceding claims, wherein: Forwarding (16) the processed data sample (z) comprises: If the local deduction indication (R1) is set, the processed data sample (z) is forwarded (161) to the third DNN deduction function (F1 24) of the mobile device (2).
8. The method (1) according to any one of the preceding claims, wherein: Forwarding (16) the processed data sample (z) comprises: If the remote deduction indication (R2) is set, the processed data sample (z) is sent (162) to the server device (3) via the wireless connection (4).
9. The method (1) according to any one of the preceding claims, wherein: Provide the final result (y) described in (19) final )include: If the high precision indication (m) is not set, and if the local derivation indication (R1) is set, according to the fifth derivation function (F G 25) and the local deduction result (y1) of the local deduction, providing (191) the final result (y1) of the initiated local deduction final ); If the high precision indication (m) is not set, and if the long range deduction indication (R2) is set, according to the fifth deduction function (F G 25) and the remote deduction result (y2) of the initiated remote deduction received from the server device (3) via the wireless connection (4), providing (192) the final result (y2) of the initiated remote deduction final );or If the high-precision indication (m) is set, according to the fifth derivation function (F G 25), the local deduction result (y1) of the initiated local deduction and the remote deduction result (y2) of the initiated remote deduction received from the server device (3) via the wireless connection (4), providing (193) the final result (y2) of the initiated local deduction and remote deduction final ).
10. According to the method (1) described in any one of the above claims, the first, third and fourth DNN inference functions (F021, F1 24, F2 31) respectively include convolutional neural networks or fully connected feedforward neural networks.
11. The method (1) according to claim 10, further comprising: According to supervised learning, perform (11) joint training of the first, third and fourth DNN inference functions (F0 21, F1 24, F2 31).
12. The method (1) according to claim 11, wherein: Executing (11) the joint training of the first, third, and fourth DNN deduction functions (F0 21, F1 24, F2 31) includes: Calculating (111) a joint loss (L=L(y, y1, y2)) of the initiated local and remote deductions based on the target result (y) associated with the data sample (x), the local deduction result (y1), and the remote deduction result (y2); According to the back-propagation technique and the joint loss (L), the corresponding weight parameters (θ0, θ1, θ2) of the first, third and fourth DNN inference functions (F0 21, F1 24, F2 31) are updated (112).
13. The method (1) according to claim 12, wherein: Calculating (111) the joint loss (L=L(y, y1, y2)) includes: According to the target result (y) associated with the data sample (x) and the average value of the local deduction result (y1) and the remote deduction result (y2) Calculate (1111) the joint loss of the initiated local game and remote game 14. The method (1) according to claim 12, wherein: Calculating (111) the joint loss (L=L(y, y1, y2)) includes: Calculating (1112) a first loss (L1=L(y,y1;θ0,θ1)) of the local deduction result (y1) of the initiated local deduction according to the weight parameter (θ0) of the first DNN deduction function (F0 21), the weight parameter (θ1) of the third DNN deduction function (F1 24), the target result (y) associated with the data sample (x), and the local deduction result (y1); Calculating (1113) a second loss (L2=L(y,y2;θ0,θ2)) of the remote deduction result (y2) of the initiated remote deduction according to the weight parameter (θ0) of the first DNN deduction function (F0 21), the weight parameter (θ2) of the fourth DNN deduction function (F231), the target result (y) associated with the data sample (x), and the remote deduction result (y2); The joint loss (L=f(L1, L2)) of the initiated local and remote games is calculated (1114) based on the first loss (L1) and the second loss (L2).
15. The method (1) according to claim 14, wherein: Calculating (1114) the joint loss (L = L (y, y1, y2)) includes: The joint loss (L=w1L1+w2L2) is calculated (11141) based on a linear combination of the first loss (L1) and the second loss (L2), wherein the linear combination is based on a loss weight (w1) of the first loss (L1) and a loss weight (w2) of the second loss (L2).
16. The method (1) according to any one of claims 12 to 15, wherein: Calculating (1114) the combined loss includes: The joint loss (L=L(y,y1,y2)) is calculated (11142) based on the mean square error (MSE) loss function or the cross entropy loss function, the local deduction result (y1), the remote deduction result (y2) and the target result (y) associated with the data sample (x).
17. The method (1) according to any one of claims 12 to 16, wherein the second DNN deduces a function (F C 22) includes a policy deep Q network DQN, a target DQN, and a state transition memory (B C ), the policy DQN and the target DQN are used to provide a respective first Q value (Q1(x, S)) associated with the local deduction and a respective second Q value (Q2(x, S)) associated with the remote deduction, and the target DQN is further used to copy the weight parameters (θ) from the policy DQN every K training steps C ).
18. The method (1) according to claim 17, further comprising: According to reinforcement learning RL, execute (12) the second DNN deduction function (F C 22) training.
19. The method (1) according to claim 18, wherein: Execute (12) the second DNN deduction function (F C 22) The training includes: Determining (121) the first Q value (Q1(x,S)) and the second Q value (Q2(x,S)) of the strategy DQN according to the state (S=[c,q]) of the environment of the mobile device (2) before performing the operation; Initiating (122) the local derivation (R=[1,0]) or the remote derivation (R=[0,1]) according to the smaller one of the determined first Q value (Q1(x,S)) and the determined second Q value (Q2(x,S)); Observe (123) one or more of the following: From the start of processing the data sample (x) to the completion of determining the final result (y final ), a penalty (g(q)) for the queue length (q) of the data sample queue, the first loss (L(y, y1; θ1)) of the local deduction result (y1) of the initiated local deduction, and the second loss (L(y, y2; θ2)) of the remote game result (y2) of the initiated remote game; determining (124) a cost associated with the initiated local game (R=[1,0]) or remote game (R=[0,1]); The state transition memory (B) is filled (125) with the tuple {[x, S], R, cost, [x, S′]} C ), wherein the tuple {[x, S], R, cost, [x, S′]} comprises the state before performing the operation (S=[c, q]), the operation (R), the cost (cost) associated with the operation (R), and the state of the environment of the mobile device (2) after performing the operation (S′=[c′, q′]); From the state conversion memory (B C ) extract (126) a batch of training samples; According to the back propagation technology and the second DNN deduction function (F C 22) of the strategy DQN and the target DQN update (127) the second DNN deduction function (F C 22) of the weight parameter (θ C ).
20. The method (1) according to claim 19, wherein: Initiating (122) the local game or the remote game includes: If the first Q-value (Q1(x, S)) of the policy DQN associated with the local game is less than or equal to the second Q-value (Q2(x, S)) of the policy DQN associated with the remote game, start (1221) the local game (R=[1, 0]); If the first Q-value (Q1(x,S)) of the policy DQN associated with the local game is greater than the second Q-value (Q2(x,S)) of the policy DQN associated with the remote game, the remote game (R=[0,1]) is initiated (1222).
21. The method (1) according to claim 19 or 20, wherein: Determining (124) the cost associated with the initiated local game or remote game includes: If the local deduction is activated (R=[1,0]), the cost (cost=L(y,y1;θ1)+d+g(q)) is determined (1241) based on the sum of the first loss (L(y,y1;θ1)), the delay (d) and the penalty (g(q)); If the remote deduction is initiated (R=[0,1]), and if the change in state is related to unsuccessful remote deduction of the remote deduction result (y2) by the server device (3), determining (1242) the cost (cost=d+g(q)) based on the sum of the delay (d) and the penalty (g(q)); If the remote deduction is started (R=[0,1]), and if the change in state is related to the success of the remote deduction of the remote deduction result (y2) by the server device (3), the cost (cost=L(y,y2;θ2)+d+g(q)) is determined (1243) based on the sum of the second loss (L(y,y2;θ2)), the delay (d) and the penalty (g(q)).
22. The method (1) according to any one of claims 19 to 21, wherein the state (S = [c, q]) of the environment changes once or more during the life cycle of the data sample (x).
23. The method (1) according to any one of claims 12 to 22, wherein the fifth deduction function (F G 25) includes the fifth DNN deduction function (F G 25), the fifth DNN derivation function (F G 25) Includes convolutional neural networks or fully connected feedforward neural networks.
24. The method (1) according to claim 23, further comprising: According to supervised learning, the fifth DNN deduction function (F) described in (13) is executed. G 25) training.
25. The method (1) according to claim 24, wherein: Execute the fifth DNN deduction function (F) described in (13) G 25) The training includes: According to the fifth DNN deduced function (F G 25)’s weight parameter (θ G ), the target result (y) associated with the data sample (x), and the final result (y final ), calculate (131) loss (L G =L(y,y final θ G )); According to the back-propagation technique and the loss (L G ), update (132) the weight parameter (θ) of the fifth DNN deduction function (F0 21) G ).
26. The method (1) according to claim 25, wherein: Calculating (131) the loss includes: According to the MSE loss function, the target result (y) and the final result (y final ), calculate the loss described in (1311) 27. The method (1) according to claim 25, wherein: Calculating (131) the loss includes: According to the cross entropy loss function, the target result (y) and the final result (y final ), calculate (1312) the loss (L = CrossEntropy (y, y final ).
28. A distributed deduction system (2, 3), comprising: Mobile devices (2); Server equipment (3); The system (2, 3) is used to carry out the method (1) according to any one of the preceding claims.
29. A computer program comprising: Program code, wherein when the program code is executed on a computer, it is used to perform the method (1) according to any one of claims 1 to 27.