Computer-implemented method for converting the internal structure of an artificial neural network

Abstract

The invention relates to a computer-implemented method for converting the internal structure of an artificial neural network (ANN), wherein the internal structure is described by network parameters (1) formed by weighting factors (3) assigned to at least the individual neurons (2) of the artificial neural network (ANN). The method according to the invention is characterized in that at least a part of the internal structure of the artificial neural network (ANN) is represented by at least one parametric function (5) described by function parameters (4), wherein the number of network parameters (1) of the part of the internal structure represented by the parametric function (5) is greater than the number of function parameters (4).

Description

[0001] The invention relates to a computer-implemented method for converting the internal structure of an artificial neural network according to the type defined in more detail in the preamble of claim.

[0002] Artificial intelligence is becoming increasingly prevalent in everyday life. One subfield of artificial intelligence is machine learning. Artificial neural networks can be used in machine learning. An artificial neural network consists of numerous neurons arranged and interconnected across multiple layers. Each neuron is assigned a weighting factor, also known as a parameter. Through appropriate training, the weighting factors of the neurons are modified, resulting in a specific structure within the network. Data to be processed is fed into the artificial neural network via input nodes and passed through the neurons from layer to layer until it reaches the output nodes.The path that the data to be processed takes through the network depends on the level of the respective weighting factors, i.e., the structure.

[0003] Artificial neural networks can have a large number of such layers and thus a large number of neurons. The weighting factors assigned to each neuron must be stored, which means that a comparatively large amount of storage space is required on a computer-readable storage medium to store the artificial neural network. This is particularly problematic for mobile systems, such as those in vehicles, because the computing units used here, especially in the form of embedded systems, have only limited hardware resources available.

[0004] Therefore, there is a need to reduce the file size of corresponding artificial neural networks.

[0005] The conversion of neuron types to hardware is known from US 2015 / 0269479 A1. This publication describes the application of mathematical operations to the weighting factors assigned to the neurons of an artificial neural network in order to transform the internal structure of the artificial neural network.

[0006] The present invention is based on the objective of providing an improved computer-implemented method for converting the internal structure of an artificial neural network, with the help of which the storage space requirement of the artificial neural network on a computer-readable storage medium can be reduced.

[0007] According to the invention, this problem is solved by a computer-implemented method with the features of claim 1. Advantageous embodiments and further developments are described in the dependent claims.

[0008] A generic computer-implemented method for converting the internal structure of an artificial neural network, wherein the internal structure is described by network parameters formed by weighting factors assigned to at least the individual neurons of the artificial neural network, is further developed according to the invention in that at least a part of the internal structure of the artificial neural network is mapped by at least one parametric function described by function parameters, wherein the number of network parameters of the part of the internal structure mapped by the parametric function is greater than the number of function parameters. This allows the memory requirement of the artificial neural network to be massively reduced.Thus, the weighting factors assigned to the respective neurons of the artificial neural network are no longer stored directly; instead, the corresponding structure is approximated by one or more parametric functions. Since these parametric functions comprise fewer function parameters than the number of neurons in the artificial neural network's internal structure being modeled, the storage space requirement decreases. Fewer function parameters than network parameters need to be stored. The larger the portion of the artificial neural network's internal structure that is modeled by these parametric functions, the greater the reduction in storage space.

[0009] To perform calculations with the artificial neural network, the original network structure can be restored based on the approximation of the respective parametric functions.

[0010] An advantageous further development of the method according to the invention provides that the following process steps are carried out: - Analysis of the internal structure of the artificial neural network; - based on the analysis: selection of a basis function from a set of parametric functions suitable for converting the internal structure; and - Adjusting the function parameters of the basis function until a deviation between the internal structure to be mapped and the internal structure mapped by the basis function reaches a minimum.

[0011] Different artificial neural networks are characterized by different internal structures. These internal structures depend on the number of layers, the number of neurons assigned to each layer, and the respective neuron-specific weighting factors. In addition to weighting factors, individual neurons may be assigned further network parameters, such as bias factors. These can also be approximated by the aforementioned functions. Depending on the type of internal structure, different types of parametric functions may be more or less suitable for representing the network structure. Such relationships can be identified in previous studies, allowing for the quick and reliable selection of a suitable parametric function to represent a specific part of the artificial neural network's internal structure after analyzing the internal structure.The parameters of the basis function are then modified until the resulting representation of the internal structure most closely matches the original internal structure of the artificial neural network. Standard optimization methods can be used for this step. This process can also be referred to as "re-fitting."

[0012] According to a further advantageous embodiment of the method according to the invention, it is further provided that, by means of the at least one parametric function: - the weighting factors of a subset of neurons in a single network layer are mapped; - the weighting factors of all neurons in a single network layer are mapped; - the weighting factors of multiple neurons across multiple network layers are mapped; or - the weighting factors of all neurons of the artificial neural network are mapped.

[0013] Thus, a single parametric function can be used to map different subsets of the internal structure of the artificial neural network. Depending on the situation, the subset of the internal structure to be mapped can be determined manually by a developer or automatically by a suitable computer program. In particular, empirical experience gained from the analysis of the internal structure reveals which part of the internal structure should be mapped by said parametric function.

[0014] Different parts of the internal structure can also be represented by multiple parametric functions. This increases the flexibility in implementing the method according to the invention. If a good fit cannot be achieved in certain network layers using a parametric function, the original weighting factors of the respective neurons can be retained. This approach enables dynamic adaptation of the network structure, whereby the reduction of network parameters simplifies the generation of gradients and learning signals.

[0015] A further advantageous embodiment of the method according to the invention provides that at least one parametric function is based on a Gabor function, a Fourier function, or a wavelet function. Each of these function types is characterized by robust mapping capabilities in a wide variety of situations. This reduces the risk that the network parameters in question will be inadequately represented by the application of these parametric functions.

[0016] Preferably, several parametric functions are used to convert the internal structure of the artificial neural network, with Gabor functions being used for the front network layers and Fourier functions and / or wavelet functions for the back network layers. Empirical evidence shows that using Gabor functions to model the front network layers allows for a particularly accurate approximation of the original internal structure, while Fourier functions and / or wavelet functions enable a more accurate approximation in the back network layers.

[0017] A further advantageous embodiment of the method according to the invention provides for its application to an untrained artificial neural network, an artificial neural network undergoing training, or a trained artificial neural network. Thus, the method according to the invention can be used for artificial neural networks that are in different training stages.

[0018] If the artificial neural network is sufficiently trained, it is characterized by a pronounced internal structure. The respective weighting factors of the neurons thus differ from one another in a characteristic way. Using the method according to the invention, functional parameters are determined from one or more parametric functions that approximate this network structure with the greatest possible agreement. If, on the other hand, the artificial neural network is not yet sufficiently trained or not trained at all, the weighting factors of the neurons will differ less distinctly or only marginally. Suitable parametric functions and functional parameters can also be found to model such an internal structure.

[0019] The function parameters are registered as network parameters of the artificial neural network. This allows the optimization typically performed during training to be applied. In the corresponding learning process, i.e., the training, the function parameters, rather than the weighting factors of the neurons, are optimized for the depicted parts of the internal structure of the artificial neural network. These parameters are directly integrated into the gradient flow, which, since these parametric functions are differentiable, allows for efficient optimization using commonly employed gradient-based optimization methods. Because fewer parameters need to be optimized, execution efficiency increases. In particular, sufficient training can be achieved in a shorter time. The resulting energy consumption also decreases. Although the individual network layers, or...Although filters need to be recreated at each inference step of the artificial neural network based on the function parameters, optimization remains possible and efficient.

[0020] The subsequent "training" of the artificial neural network can also be referred to as fine-tuning. This involves adjusting the function parameters to the specific characteristics of the input data used, in order to maximize the accuracy and performance of the artificial neural network's inference capabilities. Fine-tuning helps to make small adjustments that ensure the artificial neural network continues to operate precisely and efficiently, even after the original weighting factors of the neurons have been replaced by these parametric functions.

[0021] The method according to the invention is suitable for different learning methods such as reinforcement learning or deep learning.

[0022] According to a further advantageous embodiment of the method according to the invention, the method is applied to an artificial neural network in the form of a convolutional neural network. A convolutional neural network is also referred to as a "convolutional neural network" (CNN). Convolutional neural networks are successfully used in the context of machine vision. In particular, artificial neural networks with a particularly complex internal structure are used in machine vision. The storage space requirement is correspondingly large. With the aid of the method according to the invention, a particularly large reduction in storage space can thus be achieved in this context, especially when implemented in a vehicle, since, as already mentioned, the available computing resources are severely limited there.

[0023] A further advantageous embodiment of the method according to the invention provides that at least the reconstruction of the internal structure of the artificial neural network, based on the depicted internal structure, is performed on a computing unit in a vehicle, and the reconstructed artificial neural network processes data in the vehicle. As described above, embedded systems, for example implemented as a system-on-a-chip (SoC), have limited hardware resources. Accordingly, comparatively small storage media are installed, so the available storage space must be used efficiently. The method according to the invention is therefore fully realized when used in a vehicle. This enables the maintainability and executability of complex artificial neural networks on simple hardware components in the vehicle.The number of artificial neural networks implemented on a single processing unit can thus be increased. Fine-tuning is also quick and efficient, even with simple hardware. The need to install particularly complex and therefore expensive hardware resources in the vehicle is thus eliminated or significantly reduced. The quality of the results can also be improved, which can have a positive impact on driving safety. The appropriately converted and reconstructed artificial neural network can then be used for a wide variety of applications in the vehicle. It can process a wide range of input data in the course of providing various vehicle functions.

[0024] The artificial neural network reconstructed in this way is preferably used in the context of machine vision. For example, the converted and reconstructed artificial neural network can be used to recognize and, in particular, classify static and / or dynamic environmental objects in camera images generated by a vehicle's surround-view camera. The artificial neural network can also be used to process information derived from corresponding camera images.

[0025] The method according to the invention particularly preferably provides that a control command for the vehicle, especially for taking over longitudinal and / or lateral vehicle control, is determined depending on an inference result of the reconstructed artificial neural network. This control command can also be issued by the artificial neural network itself. Various vehicle functions can be controlled, such as a driver assistance system or a system for deriving automated or even autonomous control commands for vehicle guidance.

[0026] Further advantageous embodiments of the computer-implemented method according to the invention for converting the internal structure of an artificial neural network also result from the exemplary embodiments which are explained in more detail below with reference to the figures.

[0027] This shows: Fig. 1 a schematic representation of the use of an artificial neural network in a vehicle; Fig. 2 a schematic representation of a Gabor function; Fig. 3 a schematic representation of a Fourier function; Fig. 4. A schematic representation of a conversion of a part of the internal structure of an artificial neural network using the in Fig. 2 Gabor function shown; and Fig. 5 a flowchart of the method according to the invention.

[0028] Artificial neural networks (ANNs) are used for a wide variety of applications in vehicles. Fig. Figure 1 shows an exemplary application of an artificial neural network (ANN) in the context of machine vision. The ANN is fed a camera image (6) from a front camera of the vehicle as input data. Based on its training, the ANN is able to process the camera image (6) and recognize and classify relevant environmental objects within it. This is demonstrated in Fig. 1 in the lower image area is indicated by the bounding boxes 7 in the processed camera image 6. For example, the artificial neural network ANN recognizes an oncoming vehicle, a stop sign, and a traffic light. Specifically, the artificial neural network ANN is a convolutional neural network.

[0029] The artificial neural network ANN comprises a large number of neurons 2, arranged in several network layers L1, L2, L3, L4. The individual neurons 2 are interconnected to relay data. Due to training, a pronounced internal structure has developed within the artificial neural network ANN, which is indicated by thick lines. Fig. Figure 1 shows an exemplary section of the network structure, illustrating the weighting factors 3 assigned to the neurons 2 of the second network layer L2. The weighting factors 3 represent network parameters 1 and exhibit characteristic values ​​thanks to training. The magnitude and distribution of the weighting factors 3 represent the internal structure.

[0030] The individual neurons 2 are connected to the neurons 2 of the preceding and subsequent network layers L1 and L3. In the illustrated embodiment, the second network layer L2 alone comprises 25 network parameters 1. However, the actual network structure is much larger, and the number of neurons 2 and weighting factors 3 is therefore greater. Each weighting factor 3 must be individually stored on a computer-readable storage medium, so the underlying artificial neural network (ANN) requires a comparatively large amount of storage space. This storage requirement can be reduced using a method according to the invention for converting the internal structure of the artificial neural network (ANN).

[0031] The core idea of ​​the method according to the invention is to approximate the individual network parameters 1 of at least a part of the internal structure of the artificial neural network KNN by a parametric function 5 shown in the following figures, using function parameters 4 for this purpose. The number of function parameters 4 is less than the number of weighting factors 3 of the part of the artificial neural network KNN to be represented, so that, since fewer network parameters 1 need to be stored, memory space is saved.

[0032] As a parametric function 5, in particular one in Fig. The Gabor function shown in Figure 2 can be used. A numerical example is shown. The function parameters 4 are defined here by the factors λ, θ, ψ, σ, and γ. The respective Gabor functions are particularly preferred for use in front network layers.

[0033] Alternatively, one in Fig. The Fourier function shown in Figure 3 can be used as a parametric function. This allows the network parameters to be mapped using a corresponding Fourier transform. Fourier functions are particularly advantageous for mapping lower network layers. Wavelet functions can also be used additionally or alternatively.

[0034] Fig. Figure 4 shows the embodiment implemented with respect to the Gabor function. The technical effect of the method according to the invention becomes apparent. While in the second network layer L2 alone there are 25 network parameters 1 based on the weighting factors 3, these have been reduced to only 5 function parameters 4 by using the parametric function 5.

[0035] The process of the inventive method is described in Fig. Figure 5 shows that in step 501, the inventive method starts by providing a pre-trained artificial neural network (ANN).

[0036] Step 502 involves an analysis of the internal structure of the artificial neural network (ANN). This analysis examines the interconnection of the... Fig. The neurons shown in Figure 1 are considered across the respective network layers L1 to L4. This network depends on the magnitude of the respective weighting factors 3 of the neurons 2.

[0037] Based on previously identified influencing factors through corresponding analyses, it can be determined which sub-areas can be particularly well approximated by corresponding parametric functions 5. If such an approximation is not possible, unconverted parts of the internal structure of the artificial neural network (ANN) can be used unchanged in step 504. This means that the weighting factors 3 of the neurons 2 are reused in the existing structure.

[0038] If, however, in step 505 a conversion based on a respective parametric function 5 is possible such that the obtained approximation almost matches the original internal structure, then the respective part of the internal structure of the artificial neural network KNN is replaced accordingly.

[0039] In an optional step 506, fine-tuning of the artificial neural network can be performed.

[0040] In step 507, the artificial neural network KNN, with its reduced storage requirements, is made available for use. QUOTES INCLUDED IN THE DESCRIPTION

[0000] This list of documents cited by the applicant was automatically generated and is included solely for the reader's convenience. The list is not part of the German patent or utility model application. The DPMA accepts no liability for any errors or omissions. Cited patent literature

[0000] US 2015 / 0269479 A1

[0005]

Claims

[1] Computer-implemented method for converting the internal structure of an artificial neural network (ANN), wherein the internal structure is described by network parameters (1), formed by weighting factors (3) assigned to at least the individual neurons (2) of the artificial neural network (ANN), characterized by , that at least a part of the internal structure of the artificial neural network (ANN) is mapped by at least one parametric function (5) described by function parameters (4), wherein the number of network parameters (1) of the part of the internal structure mapped by the parametric function (5) is greater than the number of function parameters (4). [2] Method according to claim 1, characterized by the following procedural steps: - Analysis of the internal structure of the artificial neural network (ANN); - based on the analysis: selection of a basis function from a set of parametric functions suitable for converting the internal structure (5); and - Adjusting the function parameters (4) of the basis function until a deviation between the internal structure to be mapped and the internal structure mapped by the basis function reaches a minimum. [3] Method according to claim 1 or 2, characterized by , that by means of at least one parametric function (5): - the weighting factors (3) of a subset of the neurons (2) of a single network layer (L1, L2, L3, L4) are mapped; - the weighting factors (3) of all neurons (2) of a single network layer (L1, L2, L3, L4) are mapped; - the weighting factors (3) of several neurons (2) of several network layers (L1 - L4) are mapped; or - the weighting factors (3) of all neurons (2) of the artificial neural network (ANN) are mapped. [4] Method according to any one of claims 1 to 3, characterized by , that at least one parametric function (5) is based on a Gabor function, a Fourier function or a wavelet function. [5] Method according to claim 4, characterized by , that several parametric functions (5) are used to convert the internal structure, with Gabor functions being used for front network layers and Fourier functions and / or wavelet functions being used for back network layers. [6] Method according to any one of claims 1 to 5, characterized by an application to an untrained artificial neural network (ANN), an artificial neural network (ANN) in training, or a trained artificial neural network (ANN). [7] Method according to any one of claims 1 to 6, characterized by an application to an artificial neural network (ANN) in the form of a convolutional neural network. [8] Method according to any one of claims 1 to 7, characterized by , that at least the reconstruction of the internal structure of the artificial neural network (ANN) based on the depicted internal structure takes place on a computing unit in a vehicle, and the reconstructed artificial neural network (ANN) processes data in the vehicle. [9] Method according to claim 8, characterized by that the reconstructed artificial neural network (ANN) is used in the context of machine vision. [10] Method according to claim 8 or 9, characterized by , that a control command for the vehicle, in particular to take over the longitudinal and / or lateral control of the vehicle, is determined depending on an inference result of the reconstructed artificial neural network (ANN).

Images