Techniques for configuring and operating neural networks

Abstract

This disclosure relates to the configuration and operation of neural networks comprising multiple consecutive layers. The consecutive layers thus include an input layer, an output layer, and at least one hidden layer located between the input and output layers. A method for configuring a neural network includes dividing the neural network into at least a first level and a second level, each of the first and second levels including one layer, or multiple layers consecutive to each other, wherein the first level includes at least an input layer, and the second level includes at least one additional layer. The method also includes distributing at least two levels to at least two separate computing platforms, and defining at least one communication interface for each computing platform. The communication interface in one of the computing platforms allows the first or last layer of a corresponding associated level to communicate with the last layer of a preceding level or the first layer of a following level on the other computing platform.

Description

Technical Field

[0001] This disclosure generally relates to the field of data processing using neural networks. It particularly relates to methods for configuring and operating neural networks. This disclosure also relates to a computer program product having program code for performing the corresponding methods, an apparatus for configuring a neural network, a computing platform on which a portion of the neural network is configured, and a system comprising at least two computing platforms. Background Technology

[0002] Neural networks are particularly useful for classifying data or predicting future events. This involves drawing conclusions based on previous events. Specific applications include, for example, pattern recognition (speech recognition, facial recognition, etc.), process optimization (in industrial manufacturing methods), and quality assurance.

[0003] Because the aforementioned application areas require high computational power, the use of neural networks has long been confined to the research field. However, due to rapid technological advancements, a growing number of potential users are showing increasing interest in neural networks. Nevertheless, there are computing platforms, such as mobile computing platforms like mobile phones and vehicle control units, whose computational power remains relatively low, thus failing to fully utilize the advantages of neural networks.

[0004] Neural networks typically consist of various layers, in which computational operations are performed. These layers are organized hierarchically as an input layer, an arbitrary number of hidden layers, and an output layer.

[0005] The input layer is the first layer of the network and is used to process input data, such as training data or measurement data, which will be analyzed by the trained network. The output layer is the last layer of the neural network and is used to output the result after the neural network has processed the input data. Hidden layers located between the input and output layers are used for further processing of data after the input layer.

[0006] The layers of a neural network are further divided into so-called nodes. Nodes in one layer are connected to nodes in both the previous and next layers. It will be understood that the input layer has no connections to the previous layer, and the output layer has no connections to the next layer.

[0007] Each connection between nodes is weighted, which determines how the strength of the node's outcome is considered in the next layer. The connection weights are generated or adjusted during the training of the neural network.

[0008] In order to process input data in a targeted manner, neural networks must be trained. Taking facial recognition as an example, this means that an effective neural network must have already learned to recognize facial images based on a large number of images. The larger the amount of training data, the faster the neural network can process it, and the more accurate the predictions it can produce. Therefore, training data typically includes extremely large amounts of data.

[0009] The need for large amounts of data to train neural networks can, in turn, cause problems because the required training data is often not freely available (e.g., due to licensing terms), and acquiring it is often complex and expensive. Collecting new training data is also typically time-consuming (and expensive), and can lead to legal issues due to data protection guidelines. Furthermore, collecting new training data can result in low acceptance from potential users (e.g., in cases involving personal data such as images used for facial recognition).

[0010] In current technologies, the aforementioned problems are addressed in various ways. For example, pre-trained datasets can be purchased; however, this is typically expensive. Neural networks can also be trained on specialized high-performance computing platforms (e.g., mainframes) and only after training are they transferred to a less powerful computing platform (e.g., a mobile phone) for use. However, this approach makes subsequent training or expansion of the transferred neural network extremely complex. Furthermore, the less powerful computing platform must still possess sufficiently high capabilities to ensure the trained neural network running on it functions correctly. Summary of the Invention

[0011] Therefore, the object of the present invention is to provide a technique for configuring and operating neural networks that solves some or more of the problems mentioned above, or solves other problems.

[0012] According to a first aspect, a method is provided for configuring a neural network comprising multiple consecutive layers. The consecutive layers include an input layer, an output layer, and at least one hidden layer located between the input and output layers. The method includes dividing the neural network into at least a first level and a second level, each level comprising one of a plurality of layers, or a plurality of layers consecutive to each other, wherein the first level includes at least an input layer, and the second level includes at least one of the additional layers. The method further includes distributing at least two levels across at least two separate computing platforms, and defining at least one communication interface for each computing platform. The communication interface of one computing platform allows the first or last layer of a correspondingly associated level to communicate with the last layer of a preceding level or the first layer of a following level on another computing platform.

[0013] The terms "previous" and "next" should be understood in the context of the direction of information flow within the neural network during its operation. It will be understood that during the training of the neural network, information can propagate in the opposite direction of this flow.

[0014] According to one implementation, a computing platform includes an environment in which program code, i.e., software, can be executed. The computing platform can be a hardware-based and software-based platform, or a combination thereof. Hardware-based platforms include, for example, personal computers (PCs), mobile devices (e.g., tablets or mobile phones), vehicle control units, and game consoles. Software-based platforms include, for example, operating systems, frameworks, browsers, cloud computing platforms, or virtual machines.

[0015] The number of hidden layers and their distribution across the various levels are design decisions that may vary depending on the application. For example, each level may consist of only one layer, whereas in this example, at least three levels must exist to allow for the corresponding distribution of the input layer, the output layer, and one of the hidden layers located between them.

[0016] Communication interfaces include, for example, conventional network interfaces, and can be both hardware-based and software-based. For communication between computing platforms, communication interfaces can implement various communication protocols, such as the standard network protocol Transmission Control Protocol (TCP). Communication interfaces can be designed for wired or wireless communication. Combinations are also possible when the neural network is distributed across three or more computing platforms.

[0017] In a variation according to the first aspect, the first layer of a hierarchy associated with one computing platform in the computing platform corresponds to the last layer of the preceding layer on another computing platform in the computing platform. Additionally or alternatively, the last layer of a hierarchy associated with one computing platform in the computing platform corresponds to the first layer of the following layer on another computing platform in the computing platform.

[0018] If the last and first layers of two consecutive hierarchies correspond to each other on a single computing platform, the nodes of the layers can be partitioned such that the first part of each node is located in the last layer of the preceding hierarchies, and the corresponding second part of each node is located in the first layer of the following hierarchies. Data transfer between corresponding layers of two consecutive hierarchies can therefore be performed node by node (e.g., from the corresponding first part of a particular node on one computing platform to the corresponding second part of that node on the next computing platform in the flow direction of the neural network).

[0019] In one implementation, defining at least one communication interface includes configuring at least one communication interface for serializing data from the last layer of a hierarchy on a computing platform into at least one data packet, which will be transmitted according to a communication protocol; and / or for deserializing serialized data for the first layer of a hierarchy on a computing platform contained in at least one data packet received according to the communication protocol.

[0020] Depending on the form of the neural network, for example, the output data of the (child) nodes of the last layer of the considered hierarchy, or the sum of the weighted input data of the (child) nodes of the last layer of the considered hierarchy, can be serialized. Specifically, in the case of the node partitioning, the output data of a corresponding first portion from a specific node on one computing platform can be serialized for transmission along the flow direction of the neural network to a corresponding second portion of a node on the next computing platform, where it is then deserialized.

[0021] The at least one communication interface can also be configured to send data packets to a communication interface associated with a subsequent layer on another computing platform within the computing platform. Additionally or alternatively, the at least one communication interface can be configured to receive data packets from a communication interface associated with a preceding layer on another computing platform within the computing platform. Particularly during training (e.g., review training), the communication interface can also allow data transmission in the opposite direction to the flow of the neural network during its normal operation.

[0022] When using connection-oriented protocols (e.g., TCP), connections between two computing platforms can be initiated through a communication interface. Furthermore, to ensure consistency in the context of distributed computing in neural networks, software-related mechanisms, hardware-related mechanisms, or a combination of both can be used for error detection during data packet transmission (e.g., known error correction methods for data transmission).

[0023] In the variation according to the first aspect, the computing platform is configured according to a client-server model. Therefore, one computing platform acts as a client, and the other computing platform acts as a server.

[0024] According to a variant, the computing platform acting as a client can request computational operations related to the neural network as a service from the computing platform acting as a server. These computational operations may, in particular, involve data preprocessed by the computing platform acting as the client. This preprocessing can be performed in the input layer and one or more optional hidden layers on the computing platform acting as the client. The preprocessing in the computing platform acting as the client anonymizes the input data, thus serving a data protection function. The results computed by the computing platform acting as the server can be returned to the computing platform acting as the client (e.g., for further processing or direct output).

[0025] The computing platform used as a client is, for example, an end-user device such as a mobile phone, laptop computer, or vehicle control unit. Depending on the computing power required by the neural network, the computing platform used as a server may be, for example, a high-performance computer, server computer, or computing center operated by a service provider.

[0026] In another variation, the computing platform used as a server is configured to serve multiple computing platforms that act as clients (e.g., in parallel or serial fashion), each client providing at least one identical tier. Therefore, the number of computing platforms used as clients can be scaled as needed, such that, for example, all vehicles in a fleet can be used as clients. Another example is a mobile phone client, where the mobile terminal device can be used as a client.

[0027] A computing platform acting as a client may receive input data for a neural network, either from an external data source (e.g., a user, a vehicle camera, or other vehicle sensors) or based on internal computation. This input data will be processed by the neural network (at least after its initial training). In one embodiment, the method includes processing the input data in the computing platform acting as a client to generate first output data, and inputting the first output data into a computing platform acting as a server to generate second output data. The method also includes returning the second output data from the computing platform acting as a server to the computing platform acting as a client, and providing the second output data or processed third output data derived therefrom through the computing platform acting as a client. This providing step may include output (e.g., to a user) or internal storage.

[0028] In a variant, the computing platform used as a client includes a first layer having at least an input layer, wherein first output data is generated by the first layer. The computing platform used as a server includes a second layer having at least an output layer, wherein second output data is generated by the output layer. For example, the computation result of the neural network is returned from the computing platform used as a server to the computing platform used as a client.

[0029] In another variation, the computing platform used as a client includes a first layer having an input layer, and the computing platform used as a server includes a second layer having at least one hidden layer of one or more hidden layers, wherein second output data is generated by the last hidden layer of the second layer. The computing platform used as a client also includes a third layer having at least an output layer, wherein third output data is generated by the output layer. In this variation, intermediate results are sent, for example, from the computing platform used as a server to the computing platform used as a client, thereby saving bandwidth in data transmission if the storage required for the intermediate results is less than the storage required for the final result of the neural network.

[0030] In this variant, the computing platform used as a server has more computing power than the computing platform used as a client. For example, in situations where a large number of computing platforms are used as clients and correspondingly require the computing power of computing platforms used as servers, massive amounts of data in the big data sense can be acquired distributedly (on the client side) and processed centrally (on the server side). Distributed acquisition includes specific preprocessing, such as in the input layer of a neural network.

[0031] In a variant, at least one of the individual computing platforms, particularly the computing platform used as a server, is based on a cloud computing platform. Additionally or alternatively, at least one of the individual computing platforms, particularly the computing platform used as a client, is based on a mobile computing platform. As stated in the described examples, in one variant, the mobile computing platform includes a vehicle-based platform (e.g., a motor vehicle control unit) or a portable computing platform (e.g., a mobile phone).

[0032] According to some implementations, a method is provided that includes performing a random-based initialization of the neural network before it is partitioned, and training the neural network after it is distributed to a computing platform.

[0033] According to a further embodiment, a method is provided, which includes performing a first training on the neural network before partitioning it, and performing a second training on the neural network after distributing it to a computing platform.

[0034] In this variant, the initial training of the neural network is based on the principle of transfer learning, either using another neural network (e.g., whose learning can be transferred) or using training data specific to a related task. An example of using training data specific to a related task is object recognition from image data from different application domains. Other known methods for training neural networks, such as supervised learning or reinforcement learning, can also be used. Further training can be performed using previously validated training data.

[0035] In a further example, the neural network is initially trained using a small amount of data, and its functionality is gradually optimized through repeated training after the neural network is distributed across separate computing platforms.

[0036] In the variant, training after distribution to the computing platform includes inputting training data into the computing platform with a first layer to generate output data, and inputting the output data into the computing platform with a second layer. The second training also includes training the second layer based on the output data.

[0037] In one variation, the layer with the input layer is distributed to a computing platform used as a client, and the layer with another layer is distributed to a computing platform used as a server. In a further variation, the output data of the first layer is used as an anonymized version of training data or other data. For example, the input data is anonymized through a first processing step on the client, such that when the processed data is sent to the server, only the anonymized data is sent. This variation is used to comply with data protection guidelines.

[0038] In a further variation, the training data is generated using the neural network that has been initially trained. This allows for inexpensive generation, and optionally, allows for expansion of the training dataset and storage of anonymized training data.

[0039] According to another implementation, a method is provided in which a neural network is configured for the purpose of skipping or repeatedly executing at least one layer configured on a particular computing platform. In one example, the layers are analogous to known connections between layers of a neural network, and may have forward connections, recursive connections, backward connections, or so-called skip connections.

[0040] According to a second aspect, a method for operating a computing platform is provided, on which a portion of a neural network comprising multiple consecutive layers is configured, wherein the consecutive layers include an input layer, an output layer, and at least one hidden layer located between the input and output layers. Thus, the neural network is divided into at least two levels, each level comprising one of a plurality of layers, or a plurality of layers consecutive to each other, wherein a first level includes at least an input layer, and a second level includes at least one of the additional layers. The at least two levels are thereby distributed across at least two separate computing platforms, and at least one communication interface is defined for each computing platform. The method further includes the step of being performed by one of the computing platforms. The method includes communicating via the communication interface between the first or last layer of a level associated with that computing platform and the last layer of a preceding level or the first layer of a following level on another computing platform.

[0041] In a variation of the second aspect, communication performed by one of the computing platforms via a communication interface includes serializing data from the last layer of a hierarchy associated with the computing platform into at least one data packet, which will be transmitted according to a communication protocol. Additionally or alternatively, the communication may include deserializing serialized data for the first layer of the hierarchy on the computing platform contained in the at least one data packet received according to the communication protocol.

[0042] In a further variation of the second aspect, operation of a computing platform based on a client-server model is provided.

[0043] According to a third aspect, a computer program product is provided having program code for performing the method according to one of the foregoing aspects when executed on a computing platform or a computer separate therefrom. The computer program product can be stored on a computer-readable storage medium.

[0044] According to a fourth aspect, an apparatus for configuring a neural network is provided, the neural network comprising a plurality of consecutive layers. Thus, the consecutive layers include an input layer, an output layer, and at least one hidden layer located between the input layer and the output layer, wherein the apparatus is designed to perform the following steps: Dividing the neural network into at least a first level and a second level, each level comprising one of a plurality of layers, or a plurality of layers consecutive to each other, wherein the first level comprises at least an input layer, and the second level comprises at least one of the other layers. Distributing the at least two levels on at least two separate computing platforms, and defining at least one communication interface for each computing platform, wherein the communication interface allows the first or last layer of a correspondingly associated level to communicate with the last layer of a preceding level or with the first layer of a following level on another computing platform.

[0045] In a variation according to the fourth aspect, the device is designed to perform the method according to the first aspect.

[0046] According to a fifth aspect, a computing platform is provided on which a portion of a neural network comprising multiple consecutive layers is configured, wherein the consecutive layers include an input layer, an output layer, and at least one hidden layer located between the input layer and the output layer. This divides the neural network into at least a first level and a second level, each level comprising one of a plurality of layers, or a plurality of layers consecutive to each other, wherein the first level includes at least an input layer, and the second level includes at least one additional layer. The computing platform also includes at least one of the multiple levels and at least one communication interface, the communication interface allowing the first or last layer of that level to communicate with the last layer of a preceding level or with the first layer of a following level on another computing platform within the computing platform.

[0047] In a variation based on the fifth aspect, the computing platform is designed as either a client or a server, according to the client-server model.

[0048] A computing platform can be designed to serialize data from the last layer of a hierarchy associated with the computing platform into at least one data packet, which will be transmitted according to a communication protocol. Additionally or alternatively, the computing platform can be designed to deserialize serialized data for the first layer of a hierarchy associated with the computing platform contained in at least one data packet received according to a communication protocol.

[0049] According to the sixth aspect, a system is provided that includes at least two computing platforms according to the fifth aspect, wherein, according to a client-server model, the first computing platform is configured as a client and the second computing platform is configured as a server.

[0050] In a variant according to the sixth aspect, the system includes multiple computing platforms configured as clients and designed to communicate with a computing platform configured as a server. Attached Figure Description

[0051] Other features and advantages of the technology presented herein will become apparent from the accompanying drawings and the following detailed description based on exemplary embodiments. In the drawings:

[0052] Figure 1 A schematic diagram of a neural network is shown;

[0053] Figure 2A A schematic diagram of a neural network is shown, which is divided into multiple layers and distributed across separate computing platforms with associated communication interfaces.

[0054] Figure 2B A schematic diagram illustrating the serialization and deserialization of communication data in a segmented and distributed neural network is shown.

[0055] Figure 2C A schematic diagram of a divided and distributed neural network is shown, in which the last layer of one level corresponds to the first layer of the next level;

[0056] Figure 3 A schematic diagram of two computing platforms based on a client-server model is shown, which includes a neural network divided into two layers;

[0057] Figure 4 A schematic diagram of two computing platforms based on a client-server model is shown, which includes a neural network divided into three layers;

[0058] Figure 5 A schematic diagram is shown, according to which a computing platform used as a server is configured to serve multiple computing platforms used as clients;

[0059] Figure 6 This is a flowchart of the method used to configure the neural network;

[0060] Figure 7 It is a flowchart of a method for data processing based on the neural network configured according to the first aspect;

[0061] Figure 8 This is a flowchart of a method for training neural networks;

[0062] Figure 9 It is a flowchart of a method for operating a computing platform on which a portion of a neural network is configured;

[0063] Figure 10 This is a flowchart of methods for recursive and reverse connections in neural networks; and

[0064] Figure 11 This is a flowchart of a method for so-called skip connections in neural networks. Detailed Implementation

[0065] In the specific description, the corresponding reference numerals indicate the same or similar components and functions.

[0066] The general structure of Neural Network 10 will first be referenced below. Figure 1 This will be explained. The neural network 10 constructed in this way is also used in embodiments of the present invention.

[0067] Figure 1 The neural network 10 shown includes multiple consecutive layers, wherein the consecutive layers include an input layer 12, an output layer 14, and multiple hidden layers 16 located between the input layer 12 and the output layer 14. Data received by one of the layers 12, 14, 16 is processed by an algorithm in a manner known to that layer in that layer.

[0068] Each of the layers 12, 14, and 16 includes multiple so-called nodes 18 (only in...). Figure 1(In the text, the input layer 12 is labeled), and nodes 18 symbolize neurons. A node 18 in one layer is thus connected via connections 20 to nodes 18 in the (possibly) preceding and (possibly) following layers. The connections 20 between nodes 18 are weighted connections 20. The weighting of a single connection 20 between two nodes 18 arranged in different layers 12, 14, 16 typically occurs during the training phase of the neural network 10. However, within the scope of some embodiments of this teaching, it is also conceivable to initialize (and optionally further train) connections 20 based on the training results of another neural network.

[0069] The following text refers to Figures 2A to 4 Various exemplary embodiments of the neural network 10 are described. The corresponding neural network 10 is thus divided into multiple layers 22, wherein the precise division of the layers is based on design decisions related to a specific planning application.

[0070] Figure 2A A schematic diagram of a neural network 10 is shown. According to an exemplary embodiment, the neural network is divided into n layers 22 and distributed across separate computing platforms 24 with associated communication interfaces 26. Each layer 22 can therefore be considered as a logical container for one or more consecutive layers 12, 14, 16.

[0071] exist Figure 2A In the example scenario shown, each level 22 comprises multiple layers, and each level 22 resides on a separate computing platform 24. Each computing platform 24 is a hardware- or software-based platform (or a combination thereof) that allows the execution of program code, i.e., software. Corresponding examples include personal computers (PCs), mobile devices (such as tablets or mobile phones), vehicle control units, game consoles, embedded systems, and combinations thereof. Software-based platforms 24 include, for example, operating systems, browsers, cloud computing platforms, virtual machines, and combinations thereof.

[0072] At least one communication interface 26 is defined for each computing platform 24, which allows the first layer of at least one layer 22 associated with that computing platform 24 to communicate with the last layer of the preceding layer 22 on another computing platform 24 (except for the input layer, see below). Figure 1 (Ref. 12 in the accompanying drawings). Alternatively or additionally, at least one communication interface 26 is defined for each computing platform 24, which allows the last layer of at least one layer 22 associated with that computing platform 24 to communicate with the first layer of the next layer 22 on another computing platform 24 (except for the output layer, see Figure 12). Figure 1 (See attached figure 14). Computational platform 24 with hidden layers (see attached figure 14). Figure 1 Reference numeral 16 in the figure may also include a communication interface 26 on the input side and a communication interface on the output side.

[0073] The definition of at least one communication interface 26 includes configuring the at least one communication interface 26 for serializing data from the last layer of at least one layer in layer 22 into one or more data packets of a communication protocol. Additionally or alternatively, the definition of at least one communication interface 26 includes configuring the at least one communication interface 26 for deserializing data contained in one or more data packets of a received communication protocol. These aspects will be discussed in more detail below.

[0074] exist Figure 2A In this example, the communication interface 26 between computing platforms 1 and 2 allows communication between layer i of layer 1 and layer i+1 of layer 2. The output data of layer 1, after being processed in the final layer i, is then sent as input data to the first layer i+1 on computing platform 2. Communication according to this example can continue through the neural network 10 as a whole. The data output by the last layer n of the neural network 10 is the output data, which reflects the final result processed by the neural network 10 as a whole.

[0075] Communication interface 26 includes, for example, a network interface, and can be a hardware- or software-based interface. For communication between computing platforms 24, various communication protocols, such as standard network protocols like TCP, can be used via communication interface 26. Communication interface 26 can be designed for wired or wireless communication between computing platforms 24. Combinations are also possible when the neural network 10 is distributed across three or more computing platforms 24.

[0076] Figure 2B An example of a neural network 10 is shown, which, according to an exemplary embodiment, is divided into two layers 22 and distributed across two separate computing platforms 24. The communication interface 26 of computing platform 1, as shown, is configured to serialize data from layer i into data packets of a communication protocol. In this example, the communication interface 26 of computing platform 1 can also send data packets to the communication interface 26 of the subsequent computing platform 2. The communication interface 26 of computing platform 2 is thus configured to deserialize the data contained in the received data packets of the communication protocol.

[0077] When using connection-oriented protocols (such as TCP), additional steps can be initiated. Figure 2B The connection between the two computing platforms 24 is shown. Furthermore, to ensure consistency in the distributed computing context of neural networks, error detection during data packet transmission can be performed using software- or hardware-related mechanisms or a combination of both (e.g., known error correction methods for data transmission).

[0078] Figure 2CAnother exemplary embodiment of the neural network 10 is schematically illustrated, which is divided into two layers 22 and distributed across two separate computing platforms 24. In this example, the last layer (S) of layer 1 on computing platform 1 is shown. i ) corresponds to the first layer of the next level 2 on computing platform 2 (also S) i A variant of ). In this case, when partitioning the neural network, node 18 of layer i (see Figure 1 It is divided into two child nodes, such as Figure 2C As shown. It will be understood that layer i includes multiple such nodes 18, all of which are correspondingly partitioned.

[0079] exist Figure 2C The example shown is node 18 from the previous layer S(S) i-1 , Figure 2C Node 18 (not shown in the diagram) receives input data (E1 to E... n First, use the weights (G1 to G). n They are weighted separately. Then, the so-called transfer function Σ generates the sum of the weighted input data. This is then processed by the so-called activation function. This is used to process the sum of the weighted input data, thereby generating the output data for node 18.

[0080] Node 18 is partitioned and distributed across corresponding layers i on two computing platforms 1 and 2, typically in the transition function Σ and activation function. This is performed between such that the first part of the shown node 18 exists and contains at most and includes layer S of layer 1 on computing platform 1. i The transition function Σ in the middle, and the activation function The second part exists in layer S of layer 2 on computing platform 2. i In the middle. The serialization of data (here: data generated by the transfer function Σ) used for transmission to computing platform 2 via communication interface 26 of computing platform 1, and correspondingly, the data contained in the data packets subsequently received by computing platform 2 (i.e., data used for activation functions via communication interface 26 of computing platform 2). The subsequent deserialization (further processing) is performed node by node in this example.

[0081] Figure 3 and Figure 4A schematic diagram of two computing platforms 24 configured according to a client-server model, according to an exemplary embodiment, is shown. Thus, one of the computing platforms 24 acts as a client 24a, while the other acts as a server 24b. In a variation of the client-server model, the computing platform 24 acting as client 24a can request computational operations related to the neural network as a service from the computing platform 24 acting as server 24b. These computational operations may in particular involve data preprocessed by the computing platform 24 acting as client 24a. This preprocessing can be performed in the input layer 12 and one or more optional hidden layers 16 on the computing platform 24 acting as client 24a. The preprocessing thereby anonymizes the data input to the input layer 12 before the preprocessed data is output to the computing platform 24 acting as server 24b for further processing in subsequent layers. The results obtained by the computing platform 24 acting as server 24b can be returned to the computing platform 24 acting as client 24a (e.g., for further processing or direct output), or can be directly output or stored by the computing platform 24 acting as server 24b.

[0082] In one embodiment according to the client-server model, the computing platform 24 used as server 24b (as a first hardware platform) has more computing power than the computing platform 24 used as client 24a (as a second hardware platform). The computing platform 24 used as client 24a is, for example, an end-user device such as a mobile phone, laptop computer, or motor vehicle. Depending on the computing power required by the neural network 10, the computing platform 24 used as server 24b may be constituted, for example, by a high-performance computer, server computer, or computing center operated by a service provider.

[0083] Figure 3 A schematic diagram of two computing platforms 24 according to a client-server model is shown, which includes a neural network 10 divided into two layers 22. Figure 3 In the example shown, the "client" level includes an input layer 12 and (optionally) multiple hidden layers 16. The "server" level, in the selected example, includes multiple hidden layers 16 and also includes an output layer 14. In the arrangement according to this example, the results obtained by the computing platform 24 acting as server 24b can be returned to the computing platform 24 acting as client 24a for direct output (e.g., output to a user) or further processing (e.g., via an embedded system such as a vehicle control unit or a mobile phone).

[0084] Figure 4A schematic diagram of two computing platforms 24 according to a client-server model is shown, comprising a neural network 10 divided into three layers 22. In the example shown, the "client" platform 24a comprises two layers 22. The first layer 22a, denoted as "client-input," comprises an input layer 12 and (optionally) multiple hidden layers 16. The third layer 22c, denoted as "client-output," comprises multiple hidden layers 16 (optionally) and an output layer 14. Thus, both the input layer 12 and the output layer 14 reside on the computing platform 24 acting as the client 24a. The computing platform 24 acting as the server 24b comprises layers 22 containing only hidden layers 16. In this arrangement according to this example, intermediate results obtained by the computing platform 24 acting as the server 24b can be returned to the computing platform 24 acting as the client 24a for further processing.

[0085] Figure 5 A schematic diagram of an exemplary embodiment is shown, according to which a computing platform 24 serving as server 24b is configured to serve multiple computing platforms 24 serving as clients 24a. All computing platforms 24 serving as clients 24a thus have their own instances of the same at least one level 22, such that each computing platform 24 serving as client 24a, together with the computing platform 24 serving as server 24b, represents the same neural network 10. Figure 5 Various examples of computing platforms 24 that can be used as clients 24a are further illustrated, such as desktop computers, mobile phones, or motor vehicles. In this example, the server computer is used as a server. It will be understood that the various clients 24a can be of the same type (e.g., mobile phones) or different types.

[0086] In one embodiment, at least one of the individual computing platforms 24, particularly the computing platform 24 used as server 24b, is based on a cloud computing platform. Additionally or alternatively, at least one of the individual computing platforms 24, particularly the computing platform 24 used as client 24a, is based on a mobile computing platform. As stated in the mentioned examples, the mobile computing platform 24, in variations, includes a vehicle-based computing platform or a portable computing platform 24.

[0087] In this example, the number of computing platforms 24 used as clients 24a can be scaled as needed. A possible application is a configuration 30 for a neural network 10 for a fleet of motor vehicles, where each motor vehicle is implemented as a client 24a. The neural network 10 can thus be used to evaluate various types of data. For the example of a fleet of motor vehicles, this could particularly relate to the fields of telematics, maintenance, or autonomous driving.

[0088] After describing in general terms an exemplary embodiment of the hierarchical configuration of the neural network 10 in conjunction with various computing platforms 24, the following is a description of the configuration and operation of such a neural network 10.

[0089] Figure 6 A flowchart 30 illustrates an exemplary embodiment of a method for configuring neural network 10. The corresponding steps are graphically shown on the right side of flowchart 30.

[0090] It should be noted that neural network 10 may have been initialized before configuration. Initialization of neural network 10 may include initializing network parameters based on randomization (e.g., according to...). Figure 1 The initial training (which determines the weights of the connections 20) or at least the network parameters can be used to determine these parameters. Initial training can be performed, for example, based on previously validated training data. Therefore, a training database with a smaller data volume compared to a regular training dataset can be used. Alternatively or additionally, the initial training can be based on transfer learning, which uses a separate neural network or training data specific to a related task (whose network parameters can then be at least partially used in neural network 10). An example of using training data specific to a related task is object recognition from image data from different application domains. Thus, neural network 10 can be trained based on an existing comprehensive image database, for example, for driver assistance systems (recognizing objects such as road signs, vehicles, and people), and then transferred to another application domain, such as, for example, cancer cell segmentation, for use here after further specialized training with a small image database. Other known methods for training neural network 10, such as supervised learning or reinforcement learning, can also be used, for example.

[0091] Reference Figure 6 The neural network 10 is divided into at least a first layer 22a and a second layer 22b in step 32 of the method. Figure 5 Each of the layers 22 shown includes multiple consecutive layers, wherein the first layer 22a includes an input layer 12 and the second layer 22b includes one or more other layers (such as one or more hidden layers 16 and / or an output layer 14).

[0092] After dividing the neural network 10, the layers 22 are distributed across individual computing platforms 24 in step 34. This is in... Figure 5 The layer 22 is represented by distributing it to computing platforms 24 such as mobile phones, motor vehicles, servers, or cloud computing platforms 24.

[0093] Step 36 includes defining communication interfaces 26 for the respective computing platforms 24, wherein at least one communication interface 26 is defined for each computing platform 24. These communication interfaces 26 allow data exchange between individual computing platforms 24, as described above in conjunction with Figure 2.

[0094] Figure 7 A flowchart 38 illustrates an exemplary embodiment of data processing by a neural network 10, wherein a client-server model is shown in this example, having a computing platform 24 serving as a client 24a and a computing platform 24 serving as a server 24b (see, for example...). Figures 3 to 5 The right side of flowchart 38 illustrates the corresponding steps graphically.

[0095] Step 40 includes receiving input data via a computing platform 24 acting as client 24a or multiple computing platforms 24 acting as client 24a. Input data can be received, for example, from external sources (e.g., from a user, vehicle camera, or other vehicle sensor systems) or based on internal computing.

[0096] Then, the input data in the first layer 22a of the neural network 10 is processed in step 42, and first output data is generated. The result of the processing of the input data by the first layer 22a of the neural network 10 in step 42 is that the generated first output data is anonymized. This ensures compliance with data protection guidelines, for example in the case of image recognition, because only anonymized data is used as the output of the computing platform 24 of the client 24a.

[0097] Step 44 involves inputting the output data of the first layer 22a into a computing platform 24, which serves as a server 24b. The input data is further processed there, typically in at least one hidden layer 16, to generate second output data.

[0098] Step 46 includes returning the second output data to the computing platform 24, which serves as the client 24a. If the neural network 10 includes, for example, two layers 22, such as... Figure 3 As shown, the last layer of layer 22 on the computing platform 24 used as server 24b is the output layer 14. The output layer 14 generates the final computation result of the neural network 10. On the other hand, if the neural network 10 includes, for example, three layers, such as... Figure 4 As shown, the last layer of layer 22 on the computing platform 24, which serves as server 24b, is a hidden layer 16, which generates intermediate results of neural network 10. According to the embodiment, the final result or intermediate result can be returned to the computing platform 24, which serves as client 24a, in the fourth step accordingly.

[0099] Step 48 includes providing a second output data to be returned or a third output data derived therefrom through processing. If the final result of the neural network 10 is returned in step 46, the final result can be directly output in step 48 via the computing platform 24 acting as client 24a (e.g., output to a user or embedded system). On the other hand, if an intermediate result is returned, the result is further processed by another layer 22 on the computing platform 24 acting as client 24a. Therefore, the layer 22 for further processing of intermediate results is a layer 22 other than the first layer 22a with input layer 12 (see...). Figure 4 This intermediate result is advantageous when it is returned by the computing platform 24, which acts as server 24b, to the computing platform 24, which acts as client 24a, especially when the intermediate result includes a much smaller amount of data than the final result, thus saving bandwidth in data transmission.

[0100] An exemplary embodiment of the method for training the neural network 10 is in Figure 8 The flowchart 50 is shown. The corresponding steps are in... Figure 8 The flowchart is illustrated graphically on the right side of 50.

[0101] In the illustrated method, training data is input into a computing platform 24 having a first layer 22a in step 52 to generate output data. The training data can thus be generated by the computing platform 24 itself, which includes the first layer 22a. For example, the training data may include images from a camera on a mobile phone or motor vehicle. As described above, the output data is therefore an anonymized version of the training data. This anonymized version of the training data can be stored for future use. Therefore, anonymized training data can be obtained simply and inexpensively, or existing training data can be expanded to allow for a larger amount of data to be used for training.

[0102] Step 54 includes inputting the output data into a computing platform 24 having a second level 22b.

[0103] In step 56, training of the second layer 22b is performed. This training can be performed incrementally, optionally using different training data, in order to gradually increase the functionality of the neural network.

[0104] In an exemplary embodiment where multiple layers 22 on different computing platforms 24 follow the first layer 22a, all subsequent layers 22 can also be trained. The output data of the corresponding preceding layer 22 thus serves as the input data for the corresponding following layer.

[0105] exist Figure 9The diagram shows a flowchart 60 of an exemplary embodiment of a method for operating at least one computing platform 24 on which a portion of a neural network 10 is configured.

[0106] Step 62 of the method illustrates the operation 60 of the computing platform 24, which includes a portion of the already configured neural network 10. The computing platform 24 may include, for example, a first layer 22a having an input layer 12, or a subsequent layer 22b.

[0107] In step 64, communication is illustrated between the first or last layer of a hierarchy 22 associated with the computing platform 24 and the last layer of a preceding hierarchy 22 or the first layer of a subsequent hierarchy 22 on another computing platform 24. This communication is conducted via the communication interface 26 of the respective computing platform 24, and, as in... Figure 2B and 2C As described in detail in the relevant description, such communication may include data serialization, additionally or alternatively, including the deserialization of at least one data packet. Figure 9 (Not shown in the image).

[0108] The operation of computing platform 24 or more computing platforms 24 and the communication between the multiple computing platforms 24 can thus be, for example, by a configuration independent of neural network 10 (see...). Figure 6 The service provider or user can perform the operation. Therefore, the configuration and operation of the computing platform 24 can be performed in various locations.

[0109] Figure 10 and Figure 11 Flowcharts 70 and 80 of a method according to an exemplary embodiment are shown, wherein communication between layers 22 of the neural network 10 is not only performed continuously in the forward direction. The corresponding steps are graphically illustrated on the right side of the respective flowcharts 70 and 80.

[0110] Figure 10 A flowchart 70 illustrates a method for recursive and reverse connections 20 in a neural network 10. Step 72 shows the input of the output data of layer p to the next layer p+1. The next layer p+1 processes the data and outputs it again, wherein, in step 74, the output data of layer p+1 is fed back to the previous layer (e.g., layer p) or (e.g., selectively, for example, based on the occurrence of a specific condition) to itself.

[0111] These recursive and reverse connections 20 allow, for example, at least one selected layer 22 (which may have multiple layers) of a neural network 10 to be traversed multiple times, similar to the pattern of recursive and reverse connections 20 in individual layers of a known neural network.

[0112] Figure 11A flowchart 80 illustrates a method for so-called skip connections in neural network 10. In step 82, the output data of layer p is input into the next layer p+2, wherein at least one layer p+1 located in between is skipped (e.g., selectively, depending on the occurrence of a specific condition), as shown in step 82. Therefore, the output data is not input from level p to the immediately following layer p+1.

[0113] For example, skip connections 20 are used when multiple layers 22 exist, in order to skip at least one layer 22 (which may have multiple layers) that does not significantly affect the computation of the neural network 10, in order to, for example, improve computational speed. Similar skip connections 20 are known in a single layer of a conventional neural network.

[0114] As will be clear from the description of the exemplary embodiments, the method presented herein makes it possible to solve multiple problems, which in some cases are interconnected. Therefore, problems related to the large amounts of data required (in some cases) for training neural networks can be addressed. Furthermore, data protection issues (e.g., regarding anonymization) can be met. The method described herein simplifies subsequent training or extended training of neural networks. Computational platforms with relatively low computational power can also benefit from the advantages of neural networks. Moreover, flexible load distribution can be achieved by combining the computation of neural networks.

Claims

1. A method for configuring a neural network comprising a plurality of successive layers, wherein, The succession of layers comprises an input layer, an output layer, and at least one hidden layer located between the input layer and the output layer, wherein the method comprises the steps of: partitioning the neural network into at least a first level and a second level, each of the first level and the second level comprising one of the succession of layers, or a plurality of layers of the succession of layers consecutive to each other, wherein the first level comprises at least the input layer, and the second level comprises at least one further layer of the succession of layers; distributing the at least two levels to at least two separate computing platforms; and defining at least one communication interface for each of the computing platforms, wherein the communication interface of one of the computing platforms allows the first or last layer of the respective associated level to communicate with the last layer of the preceding level or with the first layer of the following level on another one of the computing platforms, wherein the following conditions are met: - the first layer of the level associated with one of the computing platforms corresponds to the last layer of the preceding level on another one of the computing platforms; and - the last layer of the level associated with one of the computing platforms corresponds to the first layer of the following level on another one of the computing platforms, wherein the respective layers of the neural network are further partitioned into nodes, and wherein the nodes of the corresponding layers of two consecutive levels on separate computing platforms are partitioned so that a first part of each node is located on the last layer of the preceding level, and a corresponding second part of each node is located on the first layer of the following level.

2. The method of claim 1, wherein, Defining the at least one communication interface comprises: configuring the at least one communication interface for at least one of: serializing data from the last layer of the level on one of the computing platforms into at least one data packet to be transmitted according to a communication protocol; and deserializing serialized data contained in at least one data packet received according to a communication protocol for the first layer of the level on one of the computing platforms.

3. The method of claim 1, comprising: configuring the computing platforms according to a client-server model, wherein at least one of the computing platforms acts as a client and another one of the computing platforms acts as a server.

4. The method of claim 3, wherein the computing platform acting as the server is configured to serve a plurality of computing platforms acting as clients, each client providing the same at least one level.

5. The method of claim 4, comprising: receiving, by the at least one computing platform acting as the client, input data to be processed by the neural network after at least an initial training; processing the input data in the computing platform acting as the client to generate first output data; inputting the first output data into the computing platform functioning as the server to generate second output data; returning the second output data from the computing platform functioning as the server to the computing platform functioning as the client; and providing, by the computing platform functioning as the client, the second output data or third output data derived from the second output data by processing.

6. The method of claim 5, wherein, the computing platform functioning as the client comprises a first tier having at least the input layer, wherein the first output data is generated by the first tier; and the computing platform functioning as the server comprises a second tier having at least the output layer, wherein the second output data is generated by the output layer.

7. The method of claim 5, wherein, the computing platform functioning as the client comprises a first tier having at least the input layer, wherein the first output data is generated by the first tier; the computing platform functioning as the server comprises a second tier having at least one of the one or more hidden layers, wherein the second output data is generated by a last hidden layer of the second tier; and the computing platform functioning as the client comprises a third tier having at least the output layer, wherein the third output data is generated by the output layer.

8. The method of claim 1, comprising: random-based initialization of the neural network prior to partitioning the neural network; and training of the neural network after distributing the neural network to the computing platforms.

9. The method of claim 1, comprising: first training of the neural network prior to partitioning the neural network; and second training of the neural network after distributing the neural network to the computing platforms.

10. The method of claim 9, wherein, the first training of the neural network is based on transfer learning using another neural network or using training data for a related task.

11. The method of claim 9, wherein, the training after distributing to the computing platforms comprises: inputting training data into a computing platform having a first tier to generate output data; inputting the output data into a computing platform having a second tier; and training the second tier based on the output data.

12. The method of claim 11, wherein, the output data functions as an anonymized version of the training data.

13. The method of claim 11, wherein, the training data is generated using the neural network subjected to the first training.

14. The method of claim 1, wherein, the neural network is configured to enable skipping or iterative execution of at least one tier configured on a particular computing platform.

15. A method for operating a computing platform on which is configured a portion of a neural network comprising a plurality of successive layers, wherein, The consecutive layers comprise an input layer, an output layer, and at least one hidden layer located between the input layer and the output layer, wherein the neural network is divided into at least a first level and a second level, each of the first level and the second level comprising one of the consecutive layers, or a plurality of the consecutive layers consecutive to each other, wherein the first level comprises at least the input layer, and the second level comprises at least one further layer of the consecutive layers, wherein the at least two levels are distributed to at least two separate computing platforms, and wherein at least one communication interface is defined for each of the computing platforms, wherein the following conditions are met: a first layer of a level associated with one of the computing platforms corresponds to a last layer of a preceding level on another one of the computing platforms; and a last layer of a level associated with one of the computing platforms corresponds to a first layer of a following level on another one of the computing platforms, wherein the individual layers of the neural network are further divided into nodes, and wherein the nodes of the corresponding layers of two consecutive levels on separate computing platforms are divided such that a first portion of each node is located at the last layer of the preceding level, and a corresponding second portion of each node is located at the first layer of the following level, wherein the method comprises the following steps, performed by one of the computing platforms: communicating, via the communication interface, the first layer or the last layer of the level associated with this computing platform with the last layer of the preceding level or with the first layer of the following level on another one of the computing platforms.

16. The method according to claim 15, wherein the communicating via the communication interface by one of the computing platforms comprises at least one of: serializing data from the last layer of the level associated with this computing platform into at least one data packet to be transmitted according to a communication protocol; and deserializing serialized data contained in at least one data packet received according to a communication protocol for the first layer of a level on one of the computing platforms.

17. The method according to claim 15, comprising: operating the computing platforms according to a client-server model.

18. An apparatus for configuring a neural network, the neural network comprising a plurality of successive layers, wherein, The consecutive layers comprise an input layer, an output layer, and at least one hidden layer located between the input layer and the output layer, wherein the device is designed to perform the following steps: dividing the neural network into at least a first level and a second level, each of the first level and the second level comprising one of the consecutive layers, or a plurality of the consecutive layers consecutive to each other, wherein the first level comprises at least the input layer, and the second level comprises at least one further layer of the consecutive layers; distributing the at least two levels to at least two separate computing platforms; and - the first layer of a hierarchy associated with one of the computing platforms corresponds to the last layer of a previous hierarchy on another one of the computing platforms; and - the last layer of a hierarchy associated with one of the computing platforms corresponds to the first layer of a next hierarchy on another one of the computing platforms, wherein each layer of the neural network is further divided into nodes, and wherein the nodes of the corresponding layers of two consecutive hierarchies on separate computing platforms are divided such that a first portion of each node is located at the last layer of a previous hierarchy and a corresponding second portion of each node is located at the first layer of a next hierarchy. The consecutive layers include an input layer, an output layer, and at least one hidden layer located between the input layer and the output layer, wherein the neural network is divided into at least a first hierarchy and a second hierarchy, each of the first hierarchy and the second hierarchy comprising one of the consecutive layers or a plurality of the consecutive layers consecutive to each other, wherein the first hierarchy comprises at least the input layer and the second hierarchy comprises at least one further layer, wherein the following conditions are met: the first layer of a hierarchy associated with one of the computing platforms corresponds to the last layer of a previous hierarchy on another one of the computing platforms; and the last layer of a hierarchy associated with one of the computing platforms corresponds to the first layer of a next hierarchy on another one of the computing platforms, wherein each layer of the neural network is further divided into nodes, and wherein the nodes of the corresponding layers of two consecutive hierarchies on separate computing platforms are divided such that a first portion of each node is located at the last layer of a previous hierarchy and a corresponding second portion of each node is located at the first layer of a next hierarchy, wherein the computing platforms comprise: at least one of the hierarchies; 19. A computing platform on which is configured a portion of a neural network comprising a plurality of successive layers, wherein, at least one communication interface allowing the first or last layer of the hierarchy to communicate with the last layer of a previous hierarchy or with the first layer of a next hierarchy on another computing platform. According to a client-server model, a first one of the computing platforms is configured as a client and a second one of the computing platforms is configured as a server. ​ 20. A system comprising at least two computing platforms according to claim 19, wherein, ​

Images