Apparatus and method of device scheduling for distributed neural networks

EP4466639A1Pending Publication Date: 2024-11-27HUAWEI TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
EP2022709250
Authority / Receiving Office
EP · EP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2022-02-11
Publication Date
2024-11-27

AI Technical Summary

Technical Problem

In-network learning for distributed neural networks lacks metrics to assess the performance of distant nodes during inference, leading to inefficient allocation of network resources like bandwidth and power, especially in wireless environments prone to noise and interference.

Method used

A system that determines global relevance information by combining local relevance information from distant nodes with their outputs, allowing the decision node to select optimal nodes and allocate resources accordingly, improving the inference process.

Benefits of technology

This approach enables efficient allocation of resources, enhancing the performance and accuracy of neural network inference by selecting high-performing nodes and optimizing resource usage.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 1.1
    Figure 1.1
Patent Text Reader

Abstract

This disclosure relates to a first device and multiple second devices for performing distributive neural network (NN) inference in a network. Each device comprises a NN. The NN of the first device receives activation vectors of the NNs comprised in the second devices as input, and infers a final result based thereon. In addition to the activation vector, each second device determines and provides local relevance information to the first device, so that the first device can determine global relevance information based on the local relevance information and the activation vectors of the second devices. Based on the determined global relevance information, the first device selects one or more second devices for the inference phase and may allocate resources according to the global relevance information. Participating second devices can thus be optimized and resources efficiently utilized. The performance and accuracy of the NN inference can be improved.
Need to check novelty before this filing date? Find Prior Art

Description

[0001] APPARATUS AND METHOD OF DEVICE SCHEDULING FOR DISTRIBUTED NEURAL NETWORKS

[0002] TECHNICAL FIELD

[0003] The present disclosure relates to devices and methods for distributed neural networks in the field of computer technology. The disclosure proposes devices and methods for implementing a distributed neural network in the field of machine learning (ML) in a communication network.

[0004] BACKGROUND

[0005] The unprecedented success of ML techniques in areas such as computer vision, image processing, robotics and natural language processing has lead to an increasing interest in their application to wireless communication. A scheme called “in-network learning” has been proposed, which refers to distributed neural network training and inference based on multiple data streams and through multiple processing nodes in communication networks. In particular, during the inference phase, multiple distant nodes obtain (e.g., observe or measure) input data. Each distant node processes the data using its own neural network, and provides its processing result to a decision node. The decision node then concatenates the processing results as an input, and based thereon it infers a final estimation using its own neural network. For this purpose, the multiple distant nodes and the decision node can be trained collaboratively. Said architecture and scheme are particularly useful for ML application scenarios in wired or wireless communication networks.

[0006] SUMMARY

[0007] The present disclosure and its embodiments are based on the following further considerations.

[0008] The inference phase of the in-network learning is performed remotely and in a distributed manner at multiple distant nodes and based on input information gathered from one or more sources. However, it is not clear for the decision node, whether a remote node obtains (e.g., observes or measures) input data of good quality or not. Moreover, even if a remote node obtains input information of relatively good quality, it may still be affected by noise and interference during the transmission of the outcome from the distance nodes to the decision node, especially, as a typical wireless environment may suffer from time-varying noise and interference. It is often not necessary that all of the distant nodes are involved in the inference phase for a particular task. For example, when one or more distant nodes receive input information of relatively low quality, or have a relatively poor communication channel with the decision node, then said one or more distant nodes may waste network resources like bandwidth and power.

[0009] As stated above, one disadvantage of the aforementioned conventional in-network learning is that there are no metrics to assess the observation and performance of each distant node during the inference phase. As a consequence, network resources like bandwidth and power cannot be allocated efficiently.

[0010] In view of the above, this disclosure aims to improve the in-network learning based on distributed neural networks. An objective is to enable the decision node to determine a subset of optimal distant nodes, and to allocate network resources accordingly.

[0011] This and other objectives are achieved by the solutions of this disclosure as described in the independent claims. Advantageous implementations are further defined in the dependent claims.

[0012] A first aspect of the present disclosure provides a first device for a communication network. The communication network comprises at least two second devices. Each of the first device and the at least two second devices comprises at least one neural network (NN).

[0013] The first device and the at least two second devices may be used to perform in-network learning. That is, the first device may be adapted to function as a decision node of the in-network learning, and the at least two second devices may be adapted to function as distant nodes of the in- network learning.

[0014] The first device is configured to obtain outputs of the NNs of the at least two second devices. In addition, the first device is configured to obtain local relevance information of the at least two second devices. Each local relevance information corresponds to first inputs of the at least one NN of one of the at least two second devices. Each local relevance information may be represented by one or more bits. The one or more bits may be combined with the output of the NN, which is comprised in each second device, for transmission. In this case, the first device may be configured to disassemble the relevant information and the output of the NN of each second device from a complete message that it receives from each second device.

[0015] Then, the first device is configured to determine, for each obtained output, global relevance information based on the obtained outputs of the NNs and based on the local relevance information of the at least two second devices.

[0016] Then, the first device is configured to select one or more second devices, from the at least two second devices, based on the determined global relevance information.

[0017] Optionally, the first device may be configured to send scheduling information or a configuration message to the at least two second devices, wherein this scheduling information or configuration message may indicate whether each second device is selected or not.

[0018] Further, the first device is configured to provide one or more outputs corresponding to the selected one or more second devices as a second input to the at least one NN of the first device, and to run the at least one NN of the first device based on the second input.

[0019] Optionally, any NN in the present disclosure may a feed-forward NN, such as a convolutional neural network (CNN), or a recurrent neural network (RNN) like a long short-term memory (LSTM).

[0020] Optionally, for determining the global relevance information, the first device may be configured to infer a reference result using its NN based on all the obtained outputs from the at least two second devices, and to determine the global relevance information further based on the reference result.

[0021] Optionally, for running the at least one NN of the first device based on the second input, the first device may be configured to not take into account the local relevance information. As a result, the first device may obtain an optimized result corresponding to the second input. By obtaining local relevance information of the at least two second devices, the first device can become aware of the relevance of each output obtained from respectively each second device, and the inference task. Therefore, the first device may compare all the relevance information received from all the second devices, and may access each second device’s output related to other second device(s). In this way, one or more second devices of good performance may be selected by the first device, so that the final inference result obtained by the first device can be improved.

[0022] In an implementation form of the first aspect, each local relevance information comprises a parameter based on the first input to each NN of the one of the at least two second devices, wherein the parameter is usable for obtaining the output of each NN of the second device.

[0023] In another implementation form of the first aspect, each local relevance information may comprise quality information of the first input to each NN of the one of the at least two second devices.

[0024] In another implementation form of the first aspect, each local relevance information may comprise confidence information of the output of each NN of the one of the at least two second devices.

[0025] In another implementation form of the first aspect, for determining the global relevance information for each obtained output, the first device may be configured to: determine, for each obtained output, mutual information between each obtained output and the other one or more outputs; and determine, for each obtained output, a linear combination of each local relevance information and the determined mutual information as one of the global relevance information.

[0026] In another implementation form of the first aspect, each second device and the first device may be connected through a channel, and the first device may be further configured to: determine, for each second device, the quality of the channel; and select the one or more second devices from the at least two second devices based further on the quality of channels of the at least two second devices. In another implementation form of the first aspect, the first device may be further configured to allocate resources to the selected one or more second devices based on the global relevance information and the quality of channels of the selected one or more second devices, wherein the resources are used by the selected one or more second devices for further neural network inferencing.

[0027] In another implementation form of the first aspect, the resources may comprise power resources and / or bandwidth resources.

[0028] In this way, resources including power and / or bandwidth may be efficiently allocated among the selected one or more second devices. This may improve the performance and accuracy of the inference performed on the first device.

[0029] Optionally, even in the case that all of the at least two second devices are selected, the first device may still be configured to allocate resources, proportionally according to the global relevance information, among all the at least two second devices.

[0030] In another implementation form of the first aspect, the first device may be configured to communicate to the selected one or more second devices for further NN inference.

[0031] A second aspect of the present disclosure provides a second device for a communication network. The communication network comprises a first device. Each of the first device and the second device comprises at least one NN. The second device is configured to obtain an output based on an input to the at least one NN of the second device. Then, the second device is configured to provide the obtained output to the first device. Further, the second device is configured to determine local relevance information corresponding to the input and provide the determined local relevance information to the first device.

[0032] The communication network may comprise one or more further second devices, wherein each further second device may be configured to function in the same way as the second device of the second aspect.

[0033] Optionally, the output of each second device may be referred to as a latent vector. The latent vector may be a vector comprising latent variables of the input obtained by each second device. The latent variable may be referred to as a variable that cannot be directly observed or obtained from raw data comprised in the input but can be inferred through the NN of each second device.

[0034] In an implementation form of the second aspect, the local relevance information comprises a parameter based on the input, wherein the parameter is usable for obtaining the output based on the input.

[0035] In another implementation form of the second aspect, for determining the local relevance information, the second device may be further configured to determine the quality of the input as the local relevance information.

[0036] In another implementation form of the second aspect, for determining the local relevance information, the second device may be further configured to determine a confidence of the output as the local relevance information.

[0037] In another implementation form of the second aspect, the second device may be further configured to employ a further NN to determine the local relevance information.

[0038] A third aspect of the present disclosure provides a communication network. The communication network comprises a first device according to the first aspect or any implementation form thereof and at least two second devices, each according to any one of the second aspect or any implementation form thereof.

[0039] A fourth aspect of the present disclosure provides a method performed by a first device for a communication network. The communication network comprises at least two second devices. Each of the first device and the at least two second devices comprises at least one NN. The method comprises the following steps: obtaining outputs of the NNs of the at least two second devices; obtaining local relevance information of the at least two second devices, wherein each local relevance information corresponds to first inputs of the at least one NN of one of the at least two second devices; determining, for each obtained output, global relevance information based on the obtained outputs of the NNs and the local relevance information; selecting one or more second devices, from the at least two second devices, based on the determined global relevance information; providing one or more outputs corresponding to the selected one or more second devices as a second input to the at least one NN of the first device; and running the at least one NN of the first device based on the second input.

[0040] In an implementation form of the fourth aspect, each local relevance information comprises a parameter based on the first input to each NN of the one of the at least two second devices, wherein the parameter is usable for obtaining the output of each NN of the second device.

[0041] In another implementation form of the fourth aspect, each local relevance information may comprise quality information of the first input to each NN of the one of the at least two second devices.

[0042] In another implementation form of the fourth aspect, each local relevance information may comprise confidence information of the output of each NN of the one of the at least two second devices.

[0043] In another implementation form of the fourth aspect, the step of determining the global relevance information for each obtained output may comprise the following steps: determining, for each obtained output, mutual information between each obtained output and the other one or more outputs; and determining, for each obtained output, a linear combination of each local relevance information and the determined mutual information as one of the global relevance information.

[0044] In another implementation form of the fourth aspect, each second device and the first device may be connected through a channel, and the method may further comprise: determining, for each second device, the quality of the channel; and selecting the one or more second devices from the at least two second devices based further on the quality of channels of the at least two second devices.

[0045] In another implementation form of the fourth aspect, the method may further configured to allocate resources to the selected one or more second devices based on the global relevance information and the quality of channels of the selected one or more second devices, wherein the resources are used by the selected one or more second devices for further neural network inferencing.

[0046] In another implementation form of the fourth aspect, the resources may comprise power resources and / or bandwidth resources.

[0047] In another implementation form of the fourth aspect, the method may further comprise communicating by the first device to the selected one or more second devices for further NN inference.

[0048] The method of the fourth aspect and its implementation forms may achieve the same advantages and effects as described above for the first device of the first aspect and its implementation forms.

[0049] A fifth aspect of the present disclosure provides a method performed by a second device for a communication network. The communication network comprises a first device. Each of the first device and the second device comprises at least one NN. The method comprises the following steps: obtaining an output based on an input to the at least one NN of the second device; providing the obtained output to the first device; determining local relevance information corresponding to the input; and providing the determined local relevance information to the first device.

[0050] In an implementation form of the fifth aspect, the local relevance information comprises a parameter based on the input, wherein the parameter is usable for obtaining the output based on the input.

[0051] In another implementation form of the fifth aspect, the step of determining the local relevance information may comprise determining the quality of the input as the local relevance information. In another implementation form of the fifth aspect, the step of determining the local relevance information may comprise determining a confidence of the output as the local relevance information.

[0052] In another implementation form of the fifth aspect, the method may further comprise employing a further NN to determine the local relevance information.

[0053] The method of the fifth aspect and its implementation forms may achieve the same advantages and effects as described above for the second device of the second aspect and its implementation forms.

[0054] A sixth aspect of the present disclosure provides a method for a communication network. The communication network comprises a first device and at least two second devices. Each of the first device and the at least two second devices comprises at least one NN. The method comprises the following steps: obtaining, by each second device, an output based on an input to the at least one NN of the second device; providing, by each second device, the obtained output to the first device; obtaining, by the first device, the outputs of the NNs of the at least two second devices; determining, by each second device, local relevance information corresponding to each input; providing, by each second device, the determined local relevance information to the first device obtaining, by the first device, the local relevance information of the at least two second devices; determining, by the first device for each obtained output, global relevance information based on the obtained outputs of the NNs and the local relevance information; selecting, by the first device, one or more second devices from the at least two second devices, based on the global relevance information; providing, by the first device, one or more outputs corresponding to the selected one or more second devices as a second input to the at least one NN of the first device; and running, by the first device, the at least one NN of the first device based on the second input.

[0055] A seventh aspect of the present disclosure provides a computer program product. The computer program product comprises a program code for performing the method according to the fourth aspect or any implementation form thereof, when executed on a computer.

[0056] An eighth aspect of the present disclosure provides a computer program product. The computer program product comprises a program code for performing the method according to the fifth aspect or any implementation form thereof, when executed on a computer.

[0057] A ninth aspect of the present disclosure provides a non-transitory storage medium storing executable program code which, when executed by a processor, causes the method according to the fourth aspect or any implementation form thereof, or according to the fifth aspect or any implementation form thereof to be performed.

[0058] A tenth aspect of the present disclosure provides a chipset comprising a memory and a processor, which are configured to store and execute program code to perform the method according to the fourth aspect or any implementation form thereof, or according to the fifth aspect or any implementation form thereof.

[0059] It has to be noted that all devices, elements, units, and means described in the present application could be implemented in the software or hardware elements or any kind of combination thereof. All steps which are performed by the various entities described in the present application as well as the functionalities described to be performed by the various entities are intended to mean that the respective entity is adapted to or configured to perform the respective steps and functionalities.

[0060] Even if, in the following description of specific embodiments, a specific functionality or step to be performed by external entities is not reflected in the description of a specific detailed element of that entity, which performs that specific step or functionality, it should be clear for a skilled person that these methods and functionalities can be implemented in respective software or hardware elements, or any kind of combination thereof. BRIEF DESCRIPTION OF DRAWINGS

[0061] The above-described aspects and implementation forms will be explained in the following description of specific embodiments in relation to the enclosed drawings, in which

[0062] FIG. 1 shows a first device and at least two second devices of the present disclosure;

[0063] FIG. 2 shows a first device for determining glocal relevance information;

[0064] FIG. 3 shows a first device for selecting one or more second devices;

[0065] FIG. 4 shows an example of an application scenario of the present disclosure;

[0066] FIG. 5 shows another example of an application scenario of the present disclosure;

[0067] FIG. 6 shows a diagram of a method of the present disclosure;

[0068] FIG. 7 shows a diagram of another method of the present disclosure;

[0069] FIG. 8 shows a diagram of another method of the present disclosure;

[0070] FIG. 9 shows an exemplary achitecure of in-network learning.

[0071] DETAILED DESCRIPTION OF EMBODIMENTS

[0072] In the FIGs. 1 to 9, corresponding elements are labeled with the same reference signs, and share the same features, and function likewise.

[0073] FIG. 9 shows an exemplary architecture of in-network learning, which may be seen as a basis for embodiments of the present disclosure. The in-network learning may be referred to as distributed learning and an inference architecture of neural networks. As depicted in FIG.9, in both of the training phase and the inference phase, two or more distant nodes may be configured to obtain inputs and generate outputs (or activation vectors) using neural networks, respectively, in a distributed manner. Then, the generated outputs may be provided to a decision node. The decision node may be configured to concatenate the outputs and infer a final result using its own neural network based on the concatenated outputs as an input. The configurations (such as architecture, the number of layers, the number of neurons, etc.) of the neural networks of the distant nodes and the decision node, respectively, may be different. Notably, these neural networks may be co-trained in the training phase and may cooperate in the inference phase. It is further noted that the in-network learning disclosed herein with respect to FIG. 9 is useful for understanding the present disclosure. Any other features known of conventional in-network learning shall also apply to the solutions of this present disclosure. FIG. 1 shows a first device and at least two second devices according to the present disclosure. On the left-hand side of FIG. 1, the at least two second devices 120 and 140 are depicted. On the right-hand side of FIG.1, the first device 110 is depicted. In the present disclosure, where features are introduced with respect to one of the second devices, it shall be understood that the same features can be introduced with respect to the other second device(s) as well.

[0074] The first device 110 and the at least two second devices 120, 140 may form a network, and may be adapted to perform the in-network learning for a particular task, such as (but not limited) to localization and image processing. Each second device 120, 140 may be adapted to function as a distant node of the in-network learning, while the first device 110 may be adapted to function as a decision node of the in-network learning. Each of the first device 110 and the at least two second devices 120, 140 comprises at least one neural network (NN) 112, 122, 142. These respective NNs 112, 122, 142 may each be adapted to analyze input and generate output for the in-network learning. Notably, all or a part of the outputs of the NNs 122, 142 of the at least two second devices may be provided as an input to the NN 112 of the first device. Optionally, each second device 120, 140 may be adapted to obtain each input 121, 141 from a same input source or different input sources. Said inputs 121, 141 may respectively comprise raw data for inference of the NNs 122, 142 comprised in the at least two second devices. In the present disclosure, the inputs 121, 141 of the at least two second devices may be referred to as first inputs. Each second device 120, 140 is configured to generate an output 123, 143 based on the obtained first input 121, 141. Said output 123, 143 of each second device may also be referred to as an activation vector. The generated activation vector(s) may not carry sufficient information to conclude a final result for the particular task. However, the activation vector(s) may carry intermediate information useful for further inference on the first device. Therefore, the generated activation vector(s) can also be called a “latent vector”. The latent vector may be referred to as a vector comprising one or more latent variables of the input obtained by each second device. The latent variable may be referred to as a variable that cannot be directly observed or obtained from the raw data but can be inferred through a mathematical model, e.g., the NN 122, 142 of each second device.

[0075] Optionally, in the present disclosure, the first inputs 121, 141 of the second devices may be based on different observations of or measurements performed by the second devices 120, 140 for the particular task. For example, when the task is a localization of a terminal in a wireless communication network, then each second device 120, 140 may be a network device having a radio connection with the terminal, and may be configured to receive or measure signaling (e.g., channel state information) from the terminal. The signaling obtained by different second devices may be different, and can be used for inferring the location of the terminal. For another example, when the task is image processing, each second device 120, 140 may be a computing device, and may be configured to obtain a target image. Due to network quality and interference, the quality of the target image obtained by different second devices may vary. Additionally or alternatively, different second devices may be configured to obtain different parts of the target image. Therefore, in the present disclosure, the at least two second devices 120, 140 may be configured to obtain each first input 121, 141 from a same source or from different sources. The obtained first inputs 121, 141 may refer to a same object but with different attributes or quality, or may refer to different aspects or parts of the same object.

[0076] Each second device 120, 140 is configured to provide the output of its NN, i.e., the latent vector 123, 143 produced by its NN, to the first device 110. In addition, each second device 120, 140 is further configured to determine local relevance information (RI) 125, 145 corresponding to the respective first input 121, 141. Optionally, each local RI 125, 145 may comprise a parameter determined based on the first input 121, 141 of each second device. The parameter may be usable for obtaining the output 123, 143 of the NN of each device. For example, when the task is the localization as mentioned above, the local RI may comprise signal-to-noise ratio (SNR) of the signaling obtained by each second device being a network device.

[0077] Optionally, each local RI 125, 145 may comprise quality information of the first input 121, 141 of each second device. For example, when the task is the image processing as mentioned above, each local RI 125, 145 may comprise one or more parameters relating to image quality, such as (but not limited to) pixels, sharpness, noise, dynamic range, contrast, color accuracy, vignetting, and peak signal-to-noise ratio (PSNR).

[0078] Optionally, each local RI 125, 145 may comprise confidence information of the output 123, 143 of the NN of each second device. For example, the confidence information may be numerically represented by a digit from 0 to 100%. Optionally, the digit may be represented by one or more bits. Optionally, each second device 120, 140 may comprise a second NN 124, 144. The second NN 124, 144 may be configured to generate the local RI 125, 145. In this case, the NN 122, 142 configured to generate a latent vector may be referred to as a first NN 122, 142.

[0079] Optionally, each second NN 124, 144 may be configured to generate a local decision based on the input 121, 141 of the second device. The generated local decision may be comprised in the local RI 125, 145. The local decision herein may be different from the latent vector, for instance, in that the local decision may comprise a clear indication of an inferred result for the inference task, while the latent vector may not comprise a clear inferred result but may comprise intermediate inferred parameters (e.g., latent variables) that are usable for further inferencing on the first device. In this case, the local decision may be similar to the result 111 obtained by the NN 112 of the first device, which is also referred to as an inferred result. The result 111 obtained by the NN 112 of the first device may be referred to as a global decision. In this case, all the second NNs 124, 144 of all the second devices may be trained independently based on training samples of each own during the training phase. For example, for image classification, each second NN 124, 144 may be trained by using a pre-defined training dataset like MNIST dataset during the training phase independently (e.g., not in a distributed manner). During the inference phase, each second NN 124, 124 may be configured to generate an estimation of the first input 121, 141 independently, while each first NN 122, 142 may be configured to generate a latent vector such as a feature map based on the first input 121, 141.

[0080] After determining the local RI 125, 145, each second device 120, 140 is configured to provide the determined local RI 125, 145 to the first device 110.

[0081] Optionally, each local RI 125, 145 may be represented by one or more bits. The latent vectors 123, 143 may be represented by a plurality of bits. Each local RI 125, 145 may be further combined with the corresponding latent vector 123, 143 for transmission.

[0082] Optionally, the at least two second devices 120, 140 and the first device 110 may be connected through a wired or wireless connection, such as (but not limited to) WiFi and cellular network connections. In some cases, the transmission of the latent vector 123, 143 and the local RI 125, 145 may be interfered by noises during transmission through the wired or wireless channel. Therefore, each second device 120, 140 may be configured to perform channel measurement in collaboration with the first device 110 in order to provide information of channel quality, such as (but not limited) to signal strength, SNR, latency, jitter, and packet loss rate. For example, in a cellular network such as a 4G and 5G telecommunication network, the channel quality may comprise one or more of: received signal strength indicator (RS SI), reference signal received power (RSRQ), and reference signal received quality (RSRQ).

[0083] It is noted that each of the first device 110 and the at least two second devices 120, 140 of FIG. 1 may form a basis for an embodiment of the present disclosure. Further, another embodiment of the present disclosure may include a system 100 comprising the first device 110 and the at least two second devices 120, 140.

[0084] FIG. 2 shows a first device for determining glocal relevance information. Each second device 220, 240, 260 on the left-hand side of FIG. 2 is built based on the second device 120, 140 disclosed in FIG. 1 and shall share the same features and function likewise. It is noted that the number of second devices shown in FIG. 2 is for illustration purposes only and shall not be interpreted as a limitation. The first device 210 on the right-hand side of FIG. 2 is built based on the first device 110 disclosed in FIG. 1 and shall share the same features and function likewise.

[0085] The first device 210 is configured to obtain outputs of the NNs comprised in the second devices and local RI from the at least two second devices 220, 240, 260, respectively. Then, the first device 210 is configured to determine, for each obtained output (denoted by U i or Uz) of the at least two second devices, global relevance information (denoted by Global RI i or Global RIi) based on the obtained outputs (U_i, . . ., U n) and the local RI (Local RI I, . . ., Local Ri n) of the at least two second devices. It is noted that i and n are positive integers, n is equal to the total number of the second devices, and i is from 1 to n. Optionally, the first node 210 may comprise a module 214 adapted to determine the global relevance information.

[0086] Optionally, the first device 210 may be configured to determine, for a specific obtained output, mutual information between the specific obtained output and the other one or more outputs excluding the specific obtained output. Then, each global RI may be determined based on a linear combination of each local relevance information and the determined mutual information. An example for determining the global RI by the first device 210 may be as follows: wherein Global RItrefers to global RI determined by the first device 210 for second device i; local RIi refers to local RI provided by second device z; U, refers to output of second device z; Y refers to output 211 inferred through NN 212 comprised in the first device 210 based on all the latent vectors obtained from all the second devices 220, 240, 260 without considering any of the local RI; the function of I () refers to Shannon mutual information; are scaling factors associated with second device z. It is noted that the notion of [U^ Ui--^ Ui+1, Un] are intended to represent a set of Ui-Unexcluding U,. For example, in case of n=3, for second device 1 (e.g., the second device 220 of FIG. 2), the corresponding glocal RIi determined by the first device 210 may be:

[0087] Global RI±= a ■ local RI±+ ■ l(Ui;Y | U2, U3) (2).

[0088] It is noted that the notion of I(X; W | Z) may represent Shannon mutual information between X and W on the condition of Z, which may be referred to as conditional mutual information. Moreover, when Z = [Zi, Z2, ..., Zn], then the comma therein may denote a joint distribution, which is commonly known in probability theory. Optionally, aLand ?; are scaling factors configurable by the first device 210. The sum of aLand ?; may be equal to one. Alternatively, a non-linear combination of each local relevance information and the determined mutual information may be used to determine each global RI.

[0089] It is noted that the NN 212 of FIG. 2 may or may not be the same as the NN 112 of FIG. 1, as long as the at least one NN comprised in the first device is capable of inferring based on the received latent vector(s) from corresponding second device(s). That is, the at least one NN comprised in the first device may comprise NN 112 or NN 212 (when NN 112 and NN 212 are the same), or may comprise NN 112 and NN 212 (when NN 112 and NN 212 are different). Each NN 222, 242, 262 of FIG. 2 may be the same as each NN 122, 142 of FIG. 1 and shall share the same features and function likewise.

[0090] It is noted that each of the first device 210 and the at least two second devices 220, 240, 260 of FIG. 2 may form a basis for an embodiment of the present disclosure. Further, another embodiment of the present disclosure may include a system 200 comprising the first device 210 and the at least two second devices 220, 240, 260. FIG. 3 shows a first device for selecting one or more second devices based on global relevance information. It is noted that the information flow depicted in FIG. 3 may be seen as a continuation of the information flow depicted in FIG. 3. Hence, reference signs of FIG.3 are kept the same as in FIG. 2 where applicable.

[0091] After determining the global RI, the first device 210 on the right-hand side of FIG. 3 may be configured to select one or more second devices, from the at least two second devices, based on the determined global RI. For example, the first and the third second devices 220 and 260 on the left-hand side FIG. 3 are illustrated to be selected. Then, the first device 210 is configured to provide (e.g., filter from all the received latent vectors) one or more latent vectors 304, 305 of the selected one or more second devices to NN 312 comprised in the first device 210 as an input. In the present disclosure, the one or more latent vectors 304, 305 of the selected one or more second devices may be collectively referred to as a second input to the NN 321 of the first device.

[0092] Then, the first device 210 is configured to run the NN 312 and may obtain a final result 306 based on the second input. It is noted that the NN 312 of FIG. 3 may or may not be the same as the NN 212 of FIG. 2, as long as the at least one NN comprised in the first device is capable of inferring based on the received latent vector(s) from the corresponding second device(s). That is, the at least one NN comprised in the first device may comprise any one of NN 112, NN 212, and NN 312 (when NN 112, NN 212, and NN 312 are the same), or may comprise two or more of NN 112, NN 212, and NN 312 (when NN 112, NN 212, and NN 312 are different). The final result 306 in FIG. 3 may be different from the reference result 211 in FIG. 2 and can be seen as a final optimized inference result.

[0093] Optionally, the at least two second devices 220, 240, 260 and the first device 210 may be connected through a wired or wireless connection. Each second device 220, 240, 260 may be configured to communicate with the first device 210 through a corresponding channel. The channel may be a wired or wireless channel. The first device 210 may be configured to perform channel measurement as mentioned in FIG. 1 in collaboration with all the second devices 220, 240, 260 and to obtain the quality of each channel. Then, the first device 210 may be configured to select the one or more second devices from the at least two second devices based further on the quality of channels of the at least two second devices. For example, the selection of the one or more second devices may be based on scores of the at least two second devices as: wherein scorei denotes the score of second device i, Hi denotes the channel quality of the channel between second device i and the first device, are scaling factors similar to aLand Pi as mentioned above. Alternatively, a non-linear combination may be used in equation (3).

[0094] In this way, the impact of time- varying interference of the channel to the final inference result on the first device 210 may be mitigated and the accuracy of the final inference result may be improved.

[0095] Optionally, the first device 210 may be configured to select the best K second devices according to the global RI or the scores if channels are measured, wherein K is a positive integer less than or equal to n. Optionally, the first device 210 may be configured to send a configuration message 301, 302, 303 to each second device 220, 240, 260. The configuration message 301, 302, 303 may comprise a 1-bit value indicating whether the corresponding second device is selected or not. For example, the 1-bit value comprised in the configuration message 301 and 303 may be “1” indicating “selected”, while the 1-bit value in the configuration message 302 may be “0” indicating “not selected”. Optionally, the first device 210 may be further configured to allocate resources such as network bandwidth and / or power according to the global RI or the scores proportionally to the selected one or more second devices. The allocated resources may be indicated through the configuration messages 301, 303 to the selected one or more second devices. For example, in FIG. 3, two second devices are selected, i.e., second devices 220 and 240, which are denoted as second devices 1 and 3. The power (Power / ) allocated to second devices 1 and 3 may be:

[0096] Power_l= (total power budget) x (score_l) / (score_l+ score_3); Power_3= (total power budget) x (score_3) / (score_l+ score_3).

[0097] Optionally, the bandwidth may be allocated in a similar manner.

[0098] Optionally, the selection and the optional allocation may be performed at the beginning of the inference phase, e.g., in the first few inference rounds or in a first round of the inference phase. When several rounds are performed, the global RI and scores of each second device of the several rounds may be averaged. Then, the selection and the optional allocation may be performed based on the averaged global RI and the averaged scores. Once the one or more second devices are selected and the resources are allocated, for future inference rounds afterward, no local RI shall be needed to be generated and transmitted by any second device, unless there is a significant change in the environment or configuration.

[0099] Optionally, when the selection and the optional allocation need to be changed or updated due to environment or configuration changes, the first device 210 may be configured to send an updated configuration message to the at least two second devices regarding the modified configuration and / or resource configuration. In this case, global RI and / or scores may need to be re-calculated according to FIGs. 1-3.

[0100] In this way, resources such as power and bandwidth that are crucial for the performance of NN inference may be efficiently allocated among the selected one or more second devices. As a result, the accuracy of the final estimation on the first device may be increased.

[0101] FIG. 4 shows an example of an application scenario of the present disclosure. In this scenario, the inference task is to perform position estimation of a terminal based on channel state information received by nearby base stations (BS), which may be also referred to as “fingerprint-based localization”. That is, the at least two devices and the first device may be adapted to perform localization of a terminal based on signal properties. For example, three base stations BS1-3 are exemplarily depicted in FIG. 4. BS1 receives input signal XI from the terminal, BS3 receives input signal X2 from the terminal, and BS5 receives input signal X3 from the terminal. BS1-3 are configured to generate latent vectors through each NN (denoted as deep neural network (DNN) in FIG.4), and provide the latent vectors to a decision node where a final estimation of terminal location Y is inferred through its NN. In this scenario, the local RI may comprise one or more of the following information: quality of the channel between the terminal and the corresponding BS. Optionally, the channel quality may be SNR or a squared Frobenius norm of channel matrix; and an individual and independent estimation of the terminal location based local input signal received by each BS. To this end, in addition to the NN adapted to generate the latent vector, each BS may comprise a further (trained) NN adapted for this task. Then, the decision node may function as the first device disclosed in FIGs. 1-3. In particular, the decision node may be configured to determine global RI according to equation (1) or (2) and the optional scores of BS1-3 according to equation (3). Then, the decision node may select the best K (e.g., K=2) BS from BS1-3 according to the global RI or scores, and allocate power and / or bandwidth accordingly.

[0102] It is noted that the DNN and symmetric multiprocessing (SMP) depicted in FIG. 4 are for illustration purposes only and shall not be interpreted as a necessary limitation. That is, a neural network other than DNN may also be used, and the SMP module is entirely optional in this application scenario.

[0103] FIG. 5 shows another example of an application scenario of the present disclosure. In this scenario, the inference task is image classification. In this scenario, each of a number of second devices observes a same image Y (e.g., from a dataset like MNIST dataset or captured by each comprised camera) and obtain a noisy version of Y as XI, X2 and so on. Each of the second devices processes its observation via each NN, e.g. NN1 and NN2, respectively, and sends its latent vector to a first device, whose goal is to classify the image via NN3. For instance, in FIG. 5, the version of the digit observed at second device 1 is corrupted by random occlusion with a square (part of the digit is occluded and is therefore missing) as in XI, while the version of the digit observed at second device 2 is corrupted by Gaussian noise (normalized and truncated). In this scenario, the local RI sent out by each second device may comprise one or more of an individual and independent local decision obtained by each second device based on its observation (or input). Optionally, each second device may comprise a further (trained) NN adapted to generate the local decision; an indicator of the quality of the observed digit, e.g., a value of PSNR, or a one-bit indication of whether the latter is larger or smaller than a prescribed threshold.

[0104] Then, the first device is configured to perform steps as mentioned in FIG. 1-3.

[0105] In the present disclosure, the first device (the decision node of the in-network learning) is configured to take all local RI and all latent vectors sent by all participating second devices (distant nodes of the in-network learning) and based on thereon, mainly perform the following operations: performing its own (global) assessment of how relevant is every observation by each distant device for the inference task; scheduling all or only a subset of the participating distant devices; allocating resources (e.g., power and / or bandwidth) to the scheduled distant devices according to how relevant is each device’s observation and the quality of each device’s channel to the decision node.

[0106] In this way, a distant node may be evaluated based on its capacity for the inference task. Bottleneck limiting the performance of the NN inference of the in-network learning may be identified and removed by not selecting any distant node of poor performance. Moreover, resources can be allocated according to the performance evaluation and more efficiently utilized for the NN inference of the in-network learning. Hence, the performance and the accuracy of the NN inference may be improved.

[0107] FIG. 6 shows a diagram of a method 600 of the present disclosure. The method 600 is performed by a first device for a communication network. The communication network comprises at least two second devices. Each of the first device and the at least two second devices comprises at least one NN. The method 600 comprises the following steps: step 601 : obtaining outputs of the NNs of the at least two second devices; step 602: obtaining local RI of the at least two second devices, wherein each local RI corresponds to first inputs of the at least one NN of one of the at least two second devices; step 603: determining, for each obtained output, global RI based on the obtained outputs of the NNs and the local RI; step 604: selecting one or more second devices, from the at least two second devices, based on the determined global RI; step 605 : providing one or more outputs corresponding to the selected one or more second devices as a second input to the at least one NN of the first device; and step 606: running the at least one NN of the first device based on the second input.

[0108] The steps of the method 600 may share the same functions and details from the perspective of FIG. 1-5 described above. Therefore, the corresponding method implementations are not described again at this point. FIG. 7 shows a diagram of a method 700 of the present disclosure. The method 700 is performed by a second device for a communication network comprising a first device. Each second device comprises at least one NN. The method 700 comprises the following steps: step 701 : obtaining an output based on an input to the at least one NN of the second device; step 702: providing the obtained output to the first device; step 703: determining local RI corresponding to the input; and step 704: providing the determined local RI to the first device

[0109] The steps of the method 700 may share the same functions and details from the perspective of FIG. 1-5 described above. Therefore, the corresponding method implementations are not described again at this point.

[0110] FIG. 8 (FIG. 8A and FIG. 8B) shows a diagram of a method 800 of the present disclosure. The method 800 is for a communication network. The communication network comprises a first device and at least two second devices. Each device comprises at least one NN. The method 800 comprises the following steps: step 801 : obtaining, by each second device, an output based on an input to the at least one NN of the second device; step 802: providing, by each second device, the obtained output to the first device; step 803: obtaining, by the first device, the outputs of the NNs of the at least two second devices; step 804: determining, by each second device, local RI corresponding to each input; step 805: providing, by each second device, the determined local RI to the first device; step 806: obtaining, by the first device, the local RI of the at least two second devices; step 807: determining, by the first device for each obtained output, global RI based on the obtained outputs of the NNs and the local RI; step 808: selecting, by the first device, one or more second devices from the at least two second devices, based on the global RI; step 809: providing, by the first device, one or more outputs corresponding to the selected one or more second devices as a second input to the at least one NN of the first device; and step 810: running, by the first device, the at least one NN of the first device based on the second input.

[0111] The steps of the method 800 may share the same functions and details from the perspective of FIG. 1-5 described above. Therefore, the corresponding method implementations are not described again at this point.

[0112] It is noted that the first device 110, 210 and the second devices 120, 140, 220, 240, 260 in the present disclosure may comprise processing circuitry (not shown) configured to perform, conduct or initiate the various operations of the first device 110, 210 and the second devices 120, 140, 220, 240, 260 described herein, respectively. The processing circuitry may comprise hardware and software. The hardware may comprise analog circuitry or digital circuitry, or both analog and digital circuitry. The digital circuitry may comprise components such as applicationspecific integrated circuits (ASICs), field-programmable arrays (FPGAs), digital signal processors (DSPs), or multi-purpose processors. In one embodiment, the processing circuitry comprises one or more processors and a non-transitory memory connected to the one or more processors. The non-transitory memory may carry executable program code which, when executed by the one or more processors, causes the first device 110, 210 and the second devices 120, 140, 220, 240, 260 to perform, conduct or initiate the operations or methods described herein, respectively.

[0113] In the present disclosure, each of the first device 110, 210 and the second devices 120, 140, 220, 240, 260 may be an electronic device capable of computing, such as a computer, a server blade, a computer cluster, a virtual machine based on computer architectures, a tablet, a graphics processing unit, and the like.

[0114] Moreover, a type of neural network in the present disclosure is not limited. For example, the neural network in the present disclosure may be a feedforward neural network, a convolutional neural network, a recurrent neural network, a deep neural network, or the like. Moreover, neurons of the neural network may be fully connected, or partially connected. In principle, embodiments of the present disclosure may be applied to any type of a neural network that is configured to infer an output based on an input. Moreover, the present disclosure focuses on the application of neural networks during the inference phase. Hence, the neural networks in the present disclosure may be pre-trained or trained. It is neither necessary nor relevant for the present disclosure to discuss how the neural networks in the present disclosure are trained. Any conventional neural network training techniques or conventional training methods of the in-network learning may be used to train the neural networks of the present disclosure.

[0115] The present disclosure has been described in conjunction with various embodiments as examples as well as implementations. However, other variations can be understood and effected by those persons skilled in the art and practicing the claimed matter, from the studies of the drawings, this disclosure and the independent claims. In the claims as well as in the description the word “comprising” does not exclude other elements or steps and the indefinite article “a” or “an” does not exclude a plurality. A single element or other unit may fulfill the functions of several entities or items recited in the claims. The mere fact that certain measures are recited in the mutual different dependent claims does not indicate that a combination of these measures cannot be used in an advantageous implementation.

Claims

CLAIMS1. A first device (110) for a communication network, wherein the communication network comprises at least two second devices (120, 140), wherein each of the first device (110) and the at least two second devices (120, 140) comprises at least one neural network, NN (112, 122,142), and wherein the first device (110) is configured to: obtain outputs (123, 143) of the NNs of the at least two second devices; obtain local relevance information (125, 145) of the at least two second devices, wherein each local relevance information (125, 145) corresponds to first inputs (121, 141) of the at least one NN of one of the at least two second devices; determine, for each obtained output (123, 143), global relevance information based on the obtained outputs (123, 143) of the NNs and the local relevance information (125, 145); select one or more second devices (220, 260), from the at least two second devices (120, 140), based on the determined global relevance information; provide one or more outputs (304, 305) corresponding to the selected one or more second devices as a second input to the at least one NN of the first device; and run the at least one NN of the first device based on the second input.

2. The first device (110) according to claim 1, wherein each local relevance information (125, 145) comprises a parameter based on the first input (121, 141) to each NN of the one of the at least two second devices, wherein the parameter is usable for obtaining the output (123,143) of each NN of the second device.

3. The first device (110) according to claim 1 or 2, wherein each local relevance information (125, 145) comprises quality information of the first input (121, 141) to each NN of the one of the at least two second devices.

4. The first device (110) according to any one of claims 1 to 3, wherein each local relevance information (125, 145) comprises confidence information of the output (123, 143) of each NN of the one of the at least two second devices.

5. The first device (110) according to any one of claims 1 to 4, wherein for determining the global relevance information for each obtained output (123, 143), the first device (110) is configured to: determine, for each obtained output (123, 143), mutual information between each obtained output and the other one or more outputs; and determine, for each obtained output (123, 143), a linear combination of each local relevance information (125, 145) and the determined mutual information as one of the global relevance information.

6. The first device (110) according to any one of claims 1 to 5, wherein each second device and (120, 140) the first device (110) are connected through a channel, and the first device (110) is further configured to: determine, for each second device (120, 140), the quality of the channel; and select the one or more second devices from the at least two second devices (120, 140) based further on the quality of channels of the at least two second devices.

7. The first device (110) according to claim 6, further configured to allocate resources to the selected one or more second devices based on the global relevance information and the quality of channels of the selected one or more second devices, wherein the resources are used by the selected one or more second devices for further neural network inference.

8. The first device (110) according to claim 7, wherein the resources comprise power resources and / or bandwidth resources.

9. The first device (110) according to any one of claims 1 to 8, further configured to communicate to the selected one or more second devices for further NN inference.

10. A second device (120;140) for a communication network, wherein the communication network comprises a first device (110), wherein each of the first device (110) and the second device (120; 140) comprises at least one neural network, NN (122; 142), and wherein the second device (120; 140) is configured to: obtain an output (123; 143) based on an input (121; 141) to the at least one NN of the second device; provide the obtained output (123; 143) to the first device (110);determine local relevance information (125; 145) corresponding to the input (121; 141); and provide the determined local relevance information (125; 145) to the first device (110).

11. The second device (120; 140) according to claim 10, wherein the local relevance information (125; 145) comprises a parameter based on the input, wherein the parameter is usable for obtaining the output (123; 143) based on the input.

12. The second device (120; 140) according to claim 10 or 11, wherein for determining the local relevance information (125; 145), the second device (120; 140) is further configured to determine the quality of the input (121; 141)as the local relevance information (125; 145).

13. The second device (120; 140) according to any one of claims 10 to 12, wherein for determining the local relevance information (125; 145), the second device (120; 140) is further configured to determine a confidence of the output (123; 143) as the local relevance information (125; 145).

14. The second device (120;140) according to any one of claims 10 to 13, wherein for determining the local relevance information (125; 145), the second device (120; 140) is further configured to employ a further NN to determine the local relevance information (125; 145).

15. A communication network comprising a first device (110; 210) according to any one of claims 1 to 9 and at least two second devices (120, 140) according to any one of claims 10 to 14.

16. A method (600) performed by a first device for a communication network, wherein the communication network comprises at least two second devices, wherein each of the first device and the at least two second devices comprises at least one neural network, NN, and wherein the method comprises: obtaining (601) outputs of the NNs of the at least two second devices; obtaining (602) local relevance information of the at least two second devices, wherein each local relevance information corresponds to first inputs of the at least one NN of one of the at least two second devices;determining (603), for each obtained output, global relevance information based on the obtained outputs of the NNs and the local relevance information; selecting (604) one or more second devices, from the at least two second devices, based on the determined global relevance information; providing (605) one or more outputs corresponding to the selected one or more second devices as a second input to the at least one NN of the first device; and running (606) the at least one NN of the first device based on the second input.

17. A method (700) performed by a second device for a communication network, wherein the communication network comprises a first device, wherein each of the first device and the second device comprises at least one neural network, NN, and wherein the method comprises: obtaining (701) an output based on an input to the at least one NN of the second device; providing (702) the obtained output to the first device; determining (703) local relevance information corresponding to the input; and providing (704) the determined local relevance information to the first device.

18. A method (800) for a communication network, wherein the communication network comprises a first device and at least two second devices, wherein each of the first device and the at least two second devices comprises at least one neural network, NN, and wherein the method comprises: obtaining (801), by each second device, an output based on an input to the at least one NN of the second device; providing (802), by each second device, the obtained output to the first device; obtaining (803), by the first device, the outputs of the NNs of the at least two second devices; determining (804), by each second device, local relevance information corresponding to each input; providing (805), by each second device, the determined local relevance information to the first device; obtaining (806), by the first device, the local relevance information of the at least two second devices; determining (807), by the first device for each obtained output, global relevance information based on the obtained outputs of the NNs and the local relevance information;selecting (808), by the first device, one or more second devices from the at least two second devices, based on the global relevance information; providing (809), by the first device, one or more outputs corresponding to the selected one or more second devices as a second input to the at least one NN of the first device; and running (810), by the first device, the at least one NN of the first device based on the second input.

19. A computer program product comprising instructions which, when the program is executed by a first computer, cause the first computer to perform the method according to claim 16.

20. A computer program product comprising instructions which, when the program is executed by a second computer, cause the second computer to perform the method according to claim 17.