Method for distributing the parameters of a neural network, inference method and associated devices

Abstract

This method of distributing parameters of a neural network to at least one device is implemented by an orchestrator. It includes steps of: - partitioning said network into a set of subnetworks, said partitioning comprising: (i) at least one step (E20) called vertical partitioning of the neural network between two layers of the network to obtain at least one so-called vertical subnetwork, a vertical subnetwork being made up of one or more consecutive layers of said network; and (ii) at least one step (E40) called horizontal partitioning of a set of at least one layer of the network to obtain at least one so-called horizontal subnetwork comprising only a part of the neurons of the set, the result of an inference of a horizontal subnetwork being solely a function of the parameters of the neurons of this subnetwork;- sending (E60, E70) the parameters of at least one said subnetwork so that they may be distributed to at least one execution device (W) configured to perform an inference of the subnetwork, the result of the inference of the subnetwork contributing to the result of an inference of said neural network. Fig. 4;

Description

Title of the invention: Method for distributing the parameters of a neural network, inference method and associated devices Prior art

[0001] The present invention is in the field of execution of neural networks, in the inference phase. It is recalled that in automated learning, the inference phase refers to the execution of a model already trained on a training data set then tested on a validation data set. The inference phase (or inference) refers to the deployment of the model and its application in a real production situation.

[0002] The invention is more particularly situated in the context of the execution of neural networks by devices with high resource constraints, particularly in memory.

[0003] Nowadays, especially in the loT (internet of things) context, connected objects generate data, some of which can be processed by neural networks as part of a service. For example, a camera captures images and then transmits them, via a communication network, to an image recognition service provider that uses a neural network to detect and recognize people.

[0004] Neural network inference generally requires more resources than those available to the connected objects that consume the results produced by these networks. Also, in the current state of the art, it is usual to deploy neural networks on cloud servers (in English "cloud computing"); the connected objects send their data to be processed to the neural network and retrieve the result of the inference via the network.

[0005] This mechanism has the disadvantage of causing a massive data transfer between connected objects (or more generally the devices that request these neural networks) and the servers that execute these neural networks. This massive transfer introduces a latency that may not be acceptable for certain applications.

[0006] Another disadvantage is that of preserving the confidentiality of the exchanged data. For example, if a connected object wishes to request a neural network to process images, certain applications require that the images not be disclosed.

[0007] On the other hand, in the event of a network outage, such an operating mode does not guarantee the proper execution of a service.

[0008] Current solutions are therefore not satisfactory. Subject matter and summary of the invention

[0009] According to a first aspect, the invention relates to a method for distributing parameters of a neural network to at least one device, the method being implemented by a distribution orchestrator and comprising steps of: - partitioning of said network into a set of sub-networks, said partitioning comprising: (i) at least one step called vertical slicing of the neural network between two layers to obtain at least one so-called vertical sub-network, a vertical sub-network consisting of one or more consecutive layers of said network; and (ii) at least one step called horizontal cutting of a set of at least one layer of the network to obtain at least one so-called horizontal sub-network comprising only part of the neurons of the set, the result of an inference of a horizontal sub-network being solely a function of the parameters of the neurons of this sub-network; - sending the parameters of at least one said subnetwork so that they are distributed to at least one execution device configured to perform an inference of the subnetwork, the result of the inference of the subnetwork contributing to the result of an inference of said neural network.

[0010] Thus, and in general, the invention proposes to partition a neural network into several sub-networks and to distribute the parameters of at least one sub-network to an execution device configured to carry out an inference of this sub-network.

[0011] According to the invention, the partitioning comprises at least one vertical division and at least one horizontal division.

[0012] Generally speaking, a neural network processes input data to determine output data (for example, a classification of an image, a regression score). This processing can be broken down into several intermediate processes, each requiring fewer resources (memory and processor) than the complete processing. Each intermediate process is carried out by a sub-part of the neural network. Thus, a neural network can be divided into sub-networks.

[0013] The terms “partitioning” or “slicing” a neural network here designate the decomposition of the neural network into several sub-networks of smaller size than the initial neural network, each of these sub-networks being able to carry out part of the processing that the neural network must carry out.

[0014] By "vertical slicing" we refer here to a decomposition of a network composed of successive layers of neurons. The neural network is thus decomposed into several sub-networks each composed of a subset of successive layers.

[0015] The expression "vertical division" is a label. It is used figuratively because by convention a neural network is represented with its input layer(s) on the left, its output layer on the right, and each layer is represented vertically. The notion of "vertical division" refers to a decomposition of a network into sub-networks composed of one or more entire layers.

[0016] It will be considered below that the layers of a data network are ordered, the first layer being the input layer, the last layer being the output layer.

[0017] As an example, [Fig.l] represents a VGG16 neural network composed of convolution layers, pooling layers and dense layers.

[0018] We recall that: - convolution layers perform convolutional filtering to detect the presence of a set of features in input data; - pooling layers are layers, generally placed between two convolution layers, which allow to reduce the number of parameters in the network and to avoid overfitting; and that - dense layers are layers where each neuron is connected to all the neurons in the previous layer.

[0019] In [Fig.l], seven vertical sub-networks SRV1 to SRV7 obtained by vertical division of network VGG16 are represented by braces.

[0020] When slicing vertically, the order of the subnetworks in the neural network is preserved for processing the input data: the first subnetwork calculates first intermediate data from the input data, then the second subnetwork calculates second intermediate data from the first intermediate data, and so on until the last subnetwork calculates the output data of the neural network. Each subnetwork produces results, or outputs, the result produced by the last subnetwork corresponding to the result that the original neural network would produce, without partitioning. Excluding the last subnetwork, the outputs of an intermediate subnetwork constitute the inputs of the next subnetwork.

[0021] By "horizontal slicing" we refer here to the partitioning of one or more consecutive layers of neurons into several sub-networks, each comprising a subset of neurons. This slicing is not carried out between layers but within these layers, the constraint being that the result of an inference of a horizontal sub-network must be solely a function of the parameters of the neurons of this sub-network.

[0022] [Fig.2] illustrates an example of horizontal division of a layer into three horizontal sub-networks SRH1 to SRH3.

[0023] For example, the horizontal division of a dense layer composed of 1000 neurons by the invention can produce 10 dense sub-layers of 100 neurons each, each of these sub-layers determining, from the same input data, a part of the output data of the neuron layer.

[0024] Those skilled in the art understand that, when an original network is horizontally divided into horizontal sub-networks, the outputs of these sub-networks must be merged to produce the output that the original network would produce without division. This merging operation induces a cost referred to below as a synchronization cost.

[0025] [Fig.3] illustrates an example of hybrid partitioning within the meaning of the invention into vertical SRVi and horizontal SRHj sub-networks. As shown in this figure: - SRVi vertical sub-networks are made up of one or more successive and complete layers; - the result of an inference of a horizontal subnetwork is only a function of the parameters of the neurons of this subnetwork. This property comes from the fact that the neurons of a horizontal subnetwork resulting from the horizontal division of a layer or a set of layers are not connected to any neuron of another horizontal subnetwork resulting from the same division.

[0026] By proposing a hybrid partitioning of the original neural network, the invention makes it possible to process the inference of large neural networks by distributing it on devices having fewer resources than those necessary to carry out the inference of the original network in its entirety, without partitioning.

[0027] In the following, we will call execution devices (in English workers) the devices that carry out the inferences of the subnets. The execution devices can be physical machines or virtual machines.

[0028] In one embodiment, a distribution orchestrator performs hybrid partitioning (or equivalently "hybrid partitioning") of the original neural network and itself distributes the parameters of the subnetworks to the execution devices.

[0029] In another embodiment, the distribution orchestrator that performs the partition of the original network into subnetworks may delegate the partition of at least one of these subnetworks to a level 2 distribution orchestrator. This level 2 distribution orchestrator may perform a hybrid partitioning of the subnetwork, into sub-subnetworks and provide the parameters of at least one sub-subnetwork to a device configured to obtain the result of an inference of this sub-subnetwork.

[0030] The number of orchestration levels can be any.

[0031] The invention also relates to a distribution orchestrator configured to distribute parameters of a neural network to at least one device, said distribution orchestrator comprising: - a module for partitioning the network into a set of subnetworks, said module being configured to: (i) performing at least one so-called vertical division of the neural network between two layers to obtain at least one so-called vertical sub-network, a vertical sub-network consisting of one or more consecutive layers of said network; and (ii) performing at least one step called horizontal slicing of a set of at least one layer of the network to obtain at least one so-called horizontal sub-network comprising only part of the neurons of the set, the result of an inference of a horizontal sub-network being solely a function of the parameters of the neurons of this sub-network; - a module for sending the parameters of at least one said subnetwork so that they are distributed to an execution device configured to carry out an inference of the subnetwork, the result of the inference of the subnetwork contributing to the result of an inference of said neural network.

[0032] According to another aspect, the invention relates to a method of inference of a neural network partitioned into a plurality of ordered sub-networks, the plurality of sub-networks comprising: (i) at least one so-called vertical sub-network consisting of one or more consecutive layers of said network; and (ii) at least one so-called horizontal sub-network comprising only part of the neurons of a vertical sub-network, the result of an inference of a horizontal sub-network being solely a function of the parameters of the neurons of this sub-network; the parameters of each of said subnetworks being provided to a device configured to obtain an inference of said subnetwork, said sub-networks being ordered, the order of a sub-network corresponding to the order of the layer from which it comes in the neural network; the method comprising a step of receiving initial data; and for each of the sub-networks of order n taken in said order, steps of: - sending input data to said subnetwork, said input data being: (i) at least part of the initial data for said at least one sub-network of order 1 or; (ii) for a sub-network of order n greater than 1, data obtained from output data obtained by the inference of at least one sub-network of order n-1; - the inference of said neural network being obtained from the output data of said at least one maximum order sub-network.

[0033] The invention also relates to an inference orchestrator configured to carry out an inference of a neural network partitioned into a plurality of sub-networks. ordered, the plurality of sub-networks comprising:

[0034] (i) at least one so-called vertical sub-network consisting of one or more layers consecutive of said network; and

[0035] (ii) at least one so-called horizontal sub-network comprising only part of the neurons of a vertical sub-network, the result of an inference of a horizontal sub-network being solely a function of the parameters of the neurons of this sub-network;

[0036] the parameters of each of said subnetworks being provided to a device configured to obtain an inference of said subnetwork,

[0037] said sub-networks being ordered, the order of a sub-network corresponding to the order of the layer from which it comes in the neural network;

[0038] the inference orchestrator comprising:

[0039] - a data reception module for initial data;

[0040] - a scheduling module configured to send to each of the subnets of order n taken in said order, of the input data of this subnetwork, said input data being:

[0041] (i) at least a portion of the initial data for said at least one subnetwork of order 1 or ;

[0042] (ii) for a sub-network of order n greater than 1, data obtained from output data obtained by the inference of at least one sub-network of order n-1;

[0043] - a module for restoring the inference of said neural network, this being obtained from the output data of said at least one maximum order sub-network.

[0044] When we say that an orchestrator is configured to send or provide parameters of at least one subnetwork for distribution to at least one device configured to obtain an inference of a subnetwork, this may notably mean that: - the orchestrator is configured to directly provide these parameters to an execution device configured to perform the subnet inference and return the result of this inference to the orchestrator; or that - the orchestrator is configured to provide these parameters to another higher-level orchestrator and obtain the result of this inference from this other orchestrator.

[0045] In one embodiment, the distribution and inference orchestrators are implemented on the same machine, physical or virtual.

[0046] The invention also relates to a system comprising: - a distribution orchestrator and / or an inference distributor as mentioned above; and - at least one execution device configured to perform subnet inference.

[0047] At least some execution devices may be located at the periphery of a communication network (or “edge” type devices in English) for example in home gateways.

[0048] The inventors detected that a simple vertical partitioning of a deep neural network did not in practice allow the inference of this network to be distributed on one of the edge type devices. The hybrid partitioning of the invention solves this problem.

[0049] Very advantageously, the inference of the original neural network is obtained from the inferences of the sub-networks, without any performance degradation, because the architecture of the model is not modified, just distributed.

[0050] Very advantageously, the method does not involve retraining one or more neural networks, which involves having data nearby, consuming energy and can negatively impact the performance of the original neural network.

[0051] In one embodiment, the orchestrator can entrust the execution of the same task to several devices and compare the results produced by these devices in order to detect the abnormal behavior of a device, caused by a breakdown or by a malicious act for example. The invention thus makes it possible to carry out several intermediate calculations, this redundancy making it possible to detect calculation errors.

[0052] The policy for dividing and allocating subnets can be defined according to different criteria, for example according to performance criteria and / or according to confidentiality criteria.

[0053] The performance of the partition can notably be impacted by: - a communication cost of the data exchanged between the orchestrators, possibly the (sub-)orchestrator(s) and the execution devices; and by - the synchronization cost induced by the merging of the results produced by the horizontal subnetworks.

[0054] In a particular embodiment of the invention, the vertical slicing is carried out after a grouping layer (pooling layers). Such vertical slicing makes it possible to minimize the volume to be transmitted between the vertical sub-networks, and therefore the communication cost.

[0055] In one embodiment, the distribution method according to the invention comprises a step for determining whether at least one execution device has sufficient available resources to perform the inference of a vertical sub-network.

[0056] In one embodiment, if this is not the case and if the vertical sub-network comprises more than one layer, the vertical sub-network is in turn cut vertically. This step can be repeated until vertical sub-networks consisting of a single layer are obtained.

[0057] In one embodiment, only vertical sub-networks consisting of a single layer can be cut horizontally.

[0058] For example, dense layers can be cut horizontally.

[0059] In one embodiment, the distribution method comprises: - a step for determining whether at least one execution device has sufficient resources to perform an inference of a said sub-network by comparing the number of parameters of this sub-network with a maximum number of parameters that can be processed by this execution device; and if this is not the case; - at least one step of vertical or horizontal cutting of said subnetwork.

[0060] Partitioning thus ensures that the neural subnetworks are processed by execution devices with sufficient resources.

[0061] In practice, this maximum number will be chosen so as not to suffocate the execution device. It will represent, for example, a determined percentage of the available resources of the execution device, for example 30%.

[0062] Indeed, from an experimental point of view, tests have shown that inference times could be reduced on models that could be executed on execution devices whose resources were not saturated by the processing of the inference beyond a certain threshold.

[0063] The slicing policy can be defined according to different criteria, for example according to performance criteria and / or according to confidentiality criteria.

[0064] In one embodiment, the subnetwork division and allocation policy takes into account confidentiality criteria.

[0065] For example, the distribution method comprises a step of determining a sensitive nature of the data processed or generated by a subnetwork and (i) at least the partitioning of the network into subnetworks or (ii) a selection of the devices to which the parameters of said subnetworks are provided is carried out according to this determination.

[0066] For example, if a subnet receives or produces sensitive data: i / this subnetwork can in turn be divided, for example horizontally, to dilute sensitive data in several subnetworks and / or ii / this subnet can be distributed to a secure execution device.

[0067] A secure execution device is for example a device running in a secure area, for example an area isolated from the Internet network, a bastion...

[0068] It is recalled that in information systems security, a bastion is an element of a computer network with reinforced security, separated from other activities of the computer network and accessible from the outside, for example the Internet. A bastion can for example be a demilitarized zone of an intranet, partially filtered by a firewall.

[0069] In one embodiment, the orchestrator (distribution and / or inference) is placed in a bastion.

[0070] In a particular embodiment, the orchestrator (distribution and / or inference) is executed by a home gateway.

[0071] In a particular embodiment, the orchestrator (distribution and / or inference) is installed in a micro data center and the subnetworks are distributed to home gateways located near this data center.

[0072] Advantageously from a security point of view, the execution devices are not aware that the task they are executing (inference of a sub-network) is part of a more general task corresponding to an inference of the original neural network.

[0073] Advantageously, in one embodiment, the execution devices are not aware of the entire original network which is the result of training requiring know-how, data, energy, etc. and therefore represents a value. The method can therefore protect this value.

[0074] In one embodiment, the execution devices are unaware of other devices. They communicate only with an orchestrator (or sub-orchestrator).

[0075] In a particular embodiment, the inference orchestrator stores the results produced by the inferences of the subnets executed by the execution devices so as to allow recovery upon failure.

[0076] Thus, if the operation of an execution device is interrupted during the execution of a task T, the inference orchestrator can resubmit this task to the same execution device or to another execution device without having to request a new execution of the tasks necessary for the production of the input data of this task T.

[0077] In a particular embodiment, the different steps of the distribution method or the inference method are determined by computer program instructions or are implemented by a silicon chip which comprises transistors adapted to constitute logic gates of non-programmable hard-wired logic.

[0078] Consequently, the invention also relates to a computer program on an information medium, this program being capable of being implemented in a controller computer, this program comprising instructions adapted to the implementation of the steps of a distribution method and / or an inference method as described above.

[0079] This program may use any programming language, and be in the form of source code, object code, or code intermediate between source code and object code, such as in a partially compiled form, or in any other desirable form.

[0080] The invention also relates to a computer-readable information medium, and comprising instructions of a computer program as mentioned above. The information medium may be any entity or device capable of storing the program. For example, the medium may comprise a storage means, such as a ROM, a non-volatile memory of the flash type or even a magnetic recording means, for example a hard disk. Furthermore, the information medium may be a transmissible medium such as an electrical or optical signal, which may be conveyed via an electrical or optical cable, by radio or by other means. The program according to the invention may in particular be downloaded from a network such as the Internet. Alternatively, the information medium may be an integrated circuit in which the program is incorporated, the circuit being adapted to execute or to be used in the execution of the method in question. Brief description of the drawings

[0081] Other characteristics and advantages of the present invention will emerge from the description given below, with reference to the appended drawings which illustrate exemplary embodiments thereof which are not in any limiting nature. In the figures:

[0082] [Fig-1] The [Fig.l] already described represents the vertical division of a network of VGG16 type neurons;

[0083] [Fig.2] [Fig.2] illustrates an example of horizontal cutting;

[0084] [Fig.3] [Fig.3] illustrates an example of hybrid cutting;

[0085] [Fig.4] [Fig.4] represents in the form of a flowchart the main stages of a distribution method according to a particular embodiment of the invention;

[0086] [Fig.5] [Fig.5] represents in the form of a flowchart the main stages of a inference method according to a particular embodiment of the invention;

[0087] [Fig.6] [Fig.6] illustrates the inference of a partitioned neural network by the invention;

[0088] [Fig.7] [Fig.7] illustrates an example of application of the invention;

[0089] [Fig.8] [Fig.8] represents the functional architecture of a dis orchestrator tribute in accordance with a particular embodiment of the invention;

[0090] [Fig.9] [Fig.9] represents the hardware architecture of an inference orchestrator in accordance with a particular embodiment of the invention; Description of the embodiments

[0091] We will now describe an embodiment of the invention to illustrate how a neural network RN can be partitioned according to a hybrid partitioning to minimize a communication cost and the inference response latency.

[0092] In this embodiment, an ORC orchestrator plays both the role distribution orchestrator and inference orchestrator. This orchestrator is shown in Figures 8 and 9.

[0093] [Fig.4] shows the main steps of a distribution method according to a particular embodiment of the invention. This method is implemented by the ORC orchestrator.

[0094] During a step E10, the orchestrator ORC obtains a neural network RN. This can be obtained by any means, in particular via a communication network.

[0095] This RN neural network will be partitioned and distributed during steps E20 to E40. To distinguish it from the subnetworks resulting from this hybrid partition, this RN network may also be called the original network.

[0096] This original RN network can be the complete network as resulting from the training and validation phases.

[0097] This original network RN can also be a sub-network resulting from the hybrid partition of the complete network (called level 1 partition), or a sub-sub-network resulting from the hybrid partition of a sub-network resulting from the hybrid partition of the complete network (level 2 partition) and so on.

[0098] In the embodiment described here, during a step E20, the orchestrator ORC performs a vertical division of the complete network RN to obtain vertical sub-networks SRVi.

[0099] During the level 1 hybrid partition, this vertical cutting can consist of cutting the original RN network after the grouping layers.

[0100] During a hybrid partition of level n greater than 1, this vertical division can consist of vertically dividing the original network RN by 2. In other words, if the neural network RN comprises 2N layers (respectively 2N+1 layers), the vertical division of this network can produce two sub-networks of N layers (respectively a sub-network of N layers and a sub-network of N+1 layers).

[0101] A single-layer RN neural network cannot be sliced ​​vertically.

[0102] In the embodiment described here, it is assumed that the execution devices can all process the same maximum number MAX of parameters.

[0103] In the embodiment described here, during a step E30, the orchestrator ORC comprises a step E30 which determines for each vertical sub-network SRVi, whether the number of parameters P(SRVi) of this sub-network is greater than the maximum number MAX of parameters which can be processed by each of the execution devices. If this is the case, no execution device has the capacity to carry out an inference of this sub-network.

[0104] In the embodiment described herein, if the number NL of layers of this subnetwork is strictly greater than 1, the vertical sub-network SRVi is again cut vertically by returning to step E20.

[0105] On the contrary, if the number NL of layers of this sub-network is equal to 1, the vertical sub-network SRVi is cut horizontally during a step E40.

[0106] In the embodiment described here, during this step E40, the orchestrator ORC horizontally divides the sub-network into a partition of horizontal sub-sub-networks in which each horizontal sub-sub-network comprises the maximum number MAX of parameters that can be processed by an execution device.

[0107] Alternatively, during test E30, the orchestration device compares the number of parameters P(SRVi) of a subnetwork with the maximum number MAX of parameters that can actually be processed by each execution device at the present time.

[0108] In the embodiment described here, after step E40, all vertical and horizontal sub-networks can be processed by an execution device.

[0109] Any other hybrid partitioning strategy is part of the invention. A hybrid partitioning can for example start with horizontal divisions, alternate horizontal and vertical divisions...

[0110] In the embodiment described here, the allocation of subnets to the execution devices is carried out according to a confidentiality criterion.

[0111] Thus, in the embodiment described here, during a step E50, the ORC orchestrator determines the sensitive nature of the data processed or generated by a subnetwork.

[0112] If the data processed or generated by a subnetwork are considered sensitive, the parameters of these subnetworks are provided to DES execution devices considered to offer significant security (step E60). If the data processed or generated by a subnetwork are considered non-sensitive, the parameters of these subnetworks can be provided to DEC execution devices considered to offer less significant security (step E70). Of course, more than two levels of sensitivity or security can be envisaged.

[0113] In the embodiment described here, the subnetworks are distributed to the different execution devices through a communication network NET. Each communicating execution device receives the parameters of the subnetworks and installs them.

[0114] [Fig.5] illustrates a method for performing an inference of a neural network RN whose partitioning is illustrated in [Fig.6]. This method is implemented by the orchestrator ORC.

[0115] In the embodiment described here, the RN network comprises 21 ordered layers L1 to L21, the first layer receiving input data D^, the 21st and last layer producing output data DOut-

[0116] As shown in this figure, it is assumed that this neural network RN has been partitioned and that the subnetworks have been distributed over four execution devices DI to D4.

[0117] In this hybrid division: - a first vertical cutting was carried out between layers L9 and L10. The parameters of layers L1 to L9 were supplied to the DI device; - a second vertical cutting was carried out between layers L14 and L15. The parameters of layers L10 to L14 were supplied to device D2; - a third vertical cutting was carried out between layers L18 and L19. The parameters of layers L15 to L18 were supplied to the device D3; - a horizontal division was carried out within the L19 layer to obtain four horizontal sub-networks L19P1 to L19P4. The parameters of an L19Pi sub-network were provided to the Di device; - a fourth vertical cutting was carried out after layer L19. The parameters of layers L20 and L21 were supplied to device D3.

[0118] The subnetworks Li(Pj) are ordered according to the order of the layer from which they come in the neural network. Here the order of a subnetwork Li(Pj) is i.

[0119] During a step F10, the orchestrator receives initial input data DIN.

[0120] During a step F20, the orchestrator determines whether the inference is complete.

[0121] If this is not the case, during a step F30, the orchestrator determines whether the next subnetwork is a vertical subnetwork. If this is the case: - it sends the DIN data to this subnet (step F32); -it receives the DOUT data produced by the inference of this subnetwork on the DIN data (step F34) and stores them (step F36).

[0122] At least part of the output data DOUT constitutes input data for the next sub-network (step F38) and the method returns to step F20.

[0123] If during a step F30, the orchestrator determines that the next sub-network comes from a horizontal division, for each horizontal sub-network coming from this division, the orchestrator: - sends part of the DIN data to this subnet (step F42); - receives the DOUT data produced by the inference of this subnetwork on this data (step F44) and stores it (step F46).

[0124] The orchestrator merges (step F47) the DOUT data received from this subnetwork to produce merged DFOUT data and stores them (step F48). This waiting step can be called synchronization.

[0125] The merged output data DFOUT constitutes input data DIN for the next sub-network (step F49) and the method returns to step F20.

[0126] When all subnets have been requested, the orchestrator determines, during of step F20, that the inference is complete. It produces (step F50) the result of this inference which corresponds to the DIN data of the last occurrence of step F38 or F49. Application example

[0127] With reference to [Fig.7], we now present an example of implementation of the invention by a manufacturer who manufactures aeronautical parts in aluminum by an extrusion process.

[0128] To be able to detect defects in these parts, the manufacturer uses an artificial intelligence solution through image recognition, using a VGG16 type neural network.

[0129] In this example the manufacturer wishes to: i / involve in image recognition, underused devices of its infrastructure which are located in services (accounting COMPTA, secretarial SECR, graphics GRAPH, etc.) less secure than a production service PROD; ii / limit the risks of disclosure of manufacturing secrets by controlling the location of sensitive data (raw images in particular); iii / protect its know-how and investments by limiting the risk of a third party gaining knowledge of its entire artificial intelligence model; iv / limit the risks of activity espionage by a third party

[0130] For this, two types of devices are distinguished: i / those whose security policy is compatible and controlled by the production environment (represented in thick lines); ii / those considered “unsafe” by the production environment (represented in thin lines).

[0131] In this example, an ORC orchestrator divides the original VGG16 neural network into sub-networks based on the technical characteristics of the devices it manages.

[0132] In a particular embodiment, the ORC orchestrator can manage sub-orchestrators and delegate all or part of the hybrid slicing to them.

[0133] Each orchestrator or sub-orchestrator manages the execution of tasks by subnetworks resulting from the hybrid division for which it is responsible. We note: - DPi: secure devices from the PROD production service; - DGi: less secure devices from the GRAPH graphics service; - DMi: less secure devices from the COMPTA accounting service; - DSi: less secure devices from the SECR secretarial service.

[0134] In the embodiment described herein, it is considered that the ORC orchestrator manages all of the devices.

[0135] The ORC orchestrator is responsible for scheduling the tasks to be executed by the different subnets by the devices (or “workers” in English).

[0136] The devices execute the tasks assigned to them by the ORC orchestrator (or where appropriate by a sub-orchestrator) via a communication network.

[0137] In the example of [Fig.7], an MR routing module enables communication between the ORC orchestrator and the devices.

[0138] Thanks to this architecture: - only the ORC orchestrator and the MR routing module know the entire original VGG16 neural network model; - only the ORC orchestrator and the MR routing module know the data in its entirety; - the devices only know the subnet(s) entrusted to them and the part of the data entering and leaving these subnets.

[0139] In this example, the subnetworks obtained by dividing the original VGG16 network which process or produce data considered to be the most sensitive can be distributed on the most secure devices, the less secure devices being used to process only intermediate and partial data which are difficult to interpret since they have undergone unknown transformations.

[0140] The execution solution thus allows the manufacturer to take advantage of its infrastructure by protecting its secrets, including on devices with less strong security policies.

[0141] In a particular embodiment, the routing module used is MQTT.

[0142] We recall that the MQTT protocol is a communication protocol adapted for cross different networks and built around addressing logic.

[0143] In the embodiment described herein, the orchestrator can address the devices either point-to-point or in a broadcast mode.

[0144] In one embodiment, the orchestrator can entrust the execution of the same task to several devices and compare the results produced by these devices in order to detect the abnormal behavior of a device, caused by a breakdown or by a malicious act for example.

[0145] This embodiment makes the process more reliable.

[0146] Using MQTT to distribute the execution of a neural network is original.

[0147] In the embodiment described here, an optimization policy can be implemented to distribute the subnetworks across the different devices so as to limit a cost function taking into account: i / a communication cost taking into account the volume of data sent by the ORC orchestrator to each of the devices, the volume of data received by the ORC orchestrator from each of the devices, a distance (physical distance, number of hops in the network, latency, etc.) between the ORC orchestrator and each of the devices, ii / a synchronization cost to merge the results produced by the different sub-networks resulting from the same horizontal division.

[0148] Advantageously, the optimization policy can limit the loading / unloading of sub-models into memory, by assigning to each device a task (in other words a sub-network) which does not change over time. This policy has a double advantage since it allows both to save significant time in loading sub-models, but also because it allows to specifically map the knowledge of each device. This policy is of interest for the processing of batch tasks.

[0149] A distribution ORCD orchestrator according to an embodiment of the invention will now be described in relation to [Fig.8]. This orchestrator is a computer device, such as a computer.

[0150] It includes:

[0151] - a processing unit or processor 801, or CPU, intended to load ins instructions in memory, to execute them, to perform operations;

[0152] - a set of memories, including a volatile memory 802, or RAM used for execute code instructions, store variables, etc., and an EEPROM type storage memory 803. In particular, the storage memory 803 is arranged to store a parameter distribution software module which comprises code instructions for implementing the steps of the distribution method as described above.

[0153] The distribution orchestrator also includes: - an MPD module for partitioning a neural network into a set of sub-networks, said module being configured to: (i) performing at least one so-called vertical division of the neural network between two layers to obtain at least one so-called vertical sub-network, a vertical sub-network consisting of one or more consecutive layers of said network; and (ii) performing at least one step called horizontal slicing of a set of at least one layer of the network to obtain at least one so-called horizontal sub-network comprising only part of the neurons of the set, the result of an inference of a horizontal sub-network being solely a function of the parameters of the neurons of this sub-network; - a MED module for sending the parameters of at least one said subnetwork so that they are distributed to at least one execution device (W) configured to carry out an inference of the subnetwork, the result of the inference of the subnetwork contributing to the result of an inference of said neural network.

[0154] An ORCI inference orchestrator according to an embodiment of the invention will now be described in relation to [Fig.9]. This orchestrator is a computer device, such as a computer.

[0155] It includes:

[0156] - a processing unit or processor 901, or CPU, intended to load ins instructions in memory, to execute them, to perform operations;

[0157] - a set of memories, including a volatile memory 902, or RAM used for execute code instructions, store variables, etc., and an EEPROM type storage memory 903. In particular, the storage memory 903 is arranged to store an inference software module which comprises code instructions for implementing the steps of the inference method as described above.

[0158] The inference orchestrator also includes: - an MRI module for receiving initial data;

[0159] - a MOI scheduling module configured to send to each of the sub- n-order networks from a hybrid partition, input data of this sub-network, said input data being: (i) at least part of the initial data for said at least one sub-network of order 1 or; (ii) for a sub-network of order n greater than 1, data obtained from output data DOUT, DFOUT obtained by the inference of at least one sub-network of order n-1; - an MCI module for restoring the inference of said neural network, this being obtained from the output data of said at least one maximum order sub-network.

[0160] According to the invention, the sub-networks are ordered, the order of a sub-network corresponding to the order of the layer from which they come in the neural network.

Claims

Claims

1. Method for distributing parameters of a neural network to at least one device, the method being implemented by an orchestrator (ORC, SORC) and comprising steps of: - partitioning said network into a set of sub-networks, - sending (E60, E70) the parameters of at least one said sub-network so that they are distributed, through a communication network (NET), to at least one execution device (W), the result of the inference of the sub-network contributing to the result of an inference of said neural network, the selection of the execution devices to which the parameters of said sub-networks are supplied being carried out according to a determination of a sensitive nature of the data processed or generated by a sub-network.

2. Distribution method according to claim 1, wherein said partitioning comprises: (i) at least one step (E20) called vertical division of the neural network between two layers of the network to obtain at least one so-called vertical sub-network, a vertical sub-network being made up of one or more consecutive layers of said network; and (ii) at least one step (E40) called horizontal division of a set of at least one layer of the network to obtain at least one so-called horizontal sub-network comprising only a part of the neurons of the set, the result of an inference of a horizontal sub-network being solely a function of the parameters of the neurons of this sub-network.

3. Distribution method according to claim 2, characterized in that at least one vertical cutting is carried out after a grouping layer of the neural network.

4. Distribution method according to claim 2 or 3 comprising: - a step (E30) for determining whether at least one execution device has sufficient resources to carry out an inference of a said sub-network by comparing the number of parameters of this sub-network with a maximum number of parameters that can be processed by this execution device; and if this is not the case; - at least one step (E20, E40) of vertical or horizontal division of said sub-network.

5. A dispensing method according to any one of claims 2 to 4, characterized in that said at least one horizontal cutting is applied to an assembly consisting of a single layer of said network.

6. A dispensing method according to claim 5, characterized in that said at least one horizontal cutting is applied to a dense layer.

7. A distribution method according to claim 4 to 6, wherein said at least one horizontal split is applied to a layer if and only if the method does not identify any execution device having sufficient resources to perform an inference of this layer.

8. Method for inferring a neural network partitioned into a plurality of ordered sub-networks, the parameters of each of said sub-networks being supplied to a device (SORC, W) configured to obtain an inference of said sub-network, the selection of the execution device being carried out according to a determination of a sensitive nature of the data processed or generated by a sub-network; said sub-networks being ordered, the order of a sub-network corresponding to the order of the layer from which it originates in the neural network; the method comprising a step of receiving (F 10) initial data; and for each of the sub-networks of order n taken in said order, steps of: - sending (F32, F42), through a communication network (NET), input data to said sub-network, said input data being: (i) at least part of the initial data for said at least one sub-network of order 1 or;(ii) for a sub-network of order n greater than 1, data obtained from output data (DOUT, DFOUT) obtained by the inference of at least one sub-network of order n-1; - the inference of said neural network being obtained from the output data of said at least one sub-network of maximum order.;

9. An inference method according to claim 8, wherein the plurality of sub-networks comprises: (i) at least one so-called vertical sub-network consisting of one or more consecutive layers of said network; and (ii) at least one so-called horizontal sub-network comprising only part of the neurons of a vertical sub-network, the result of an inference of a horizontal sub-network being solely a function of the parameters of the neurons of this sub-network.

10. Inference method according to claim 8, comprising a step of storing (F36, F46) the output data produced (DOUT) by the inference of a said sub-network.

11. Distribution orchestrator (ORCD) configured to distribute, through a communication network (NET), parameters of a neural network to at least one device, said orchestrator comprising: - a module (MPD) for partitioning said network into a set of sub-networks, - a module (MED) for sending the parameters of at least one said sub-network so that they are distributed to at least one execution device (W) configured to carry out an inference of the sub-network, the result of the inference of the sub-network contributing to the result of an inference of said neural network, the selection of the execution devices to which the parameters of said sub-networks are supplied being carried out according to a determination of a sensitive nature of the data processed or generated by a sub-network.

12. Inference orchestrator (ORC) configured to perform an inference of a neural network partitioned into a plurality of ordered sub-networks, the parameters of each of said sub-networks being provided to a device (SORC, W) configured to obtain an inference of said sub-network, the selection of the execution device being carried out according to a determination of a sensitive nature of the data processed or generated by a sub-network; said sub-networks being ordered, the order of a sub-network corresponding to the order of the layer from which it originates in the neural network; the inference orchestrator comprising: - a module (MRI) for receiving initial data;- a scheduling module (MOI) configured to send, through a communication network (NET) to each of the sub-networks of order n taken in said order, input data of this sub-network, said input data being: (i) at least part of the initial data for said at least one sub-network of order 1 or; (ii) for a sub-network of order n greater than 1, data obtained from output data (DOUT, DFOUT) obtained by the inference of at least one sub-network of order n-1; - a module (MCI) for restoring the inference of said network of; neurons, the latter being obtained from the output data of said at least one maximum order subnetwork.

13. System comprising: - a partition orchestrator (ORC) according to claim 11 and / or an inference orchestrator according to claim 12; and - at least one execution device (W) configured to perform the inference of at least one sub-network.

14. The system of claim 13 wherein said execution device (W) is integrated into a home gateway.

15. Computer program (PG) comprising instructions for executing the steps of the distribution method according to any one of claims 1 to 7 and / or instructions for executing the steps of the inference method according to any one of claims 8 or 10 when said program is executed by a computer.

16. A computer-readable recording medium on which a computer program (PG) according to claim 15 is recorded.

Images