Information storage type neural network architecture

The neural network architecture processes complex-valued data using a regular offset function and Möbius transformation to enhance information utilization, addressing inefficiencies in existing architectures and improving accuracy in object classification and localization tasks.

JP2024161006APending Publication Date: 2024-11-15ROBERT BOSCH GMBH
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
JP2024074813
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Priority Date
2023-05-04
Filing Date
2024-05-02
Publication Date
2024-11-15

AI Technical Summary

Technical Problem

Existing neural network architectures for processing complex-valued measurement data, such as images or point clouds, fail to effectively utilize the complete information content, including the relationship between the real and imaginary parts or magnitude and phase, leading to inefficiencies in tasks like object classification and localization.

Method used

A neural network architecture that processes complex-valued inputs using a regular offset function with trainable parameters, including weights and biases, and applies a non-linear activation function, such as a Möbius transformation, to maintain relationships between the real and imaginary parts, enabling better generalization and smoothing decision boundaries.

Benefits of technology

This approach ensures that the complete information content of complex-valued measurement data is utilized, improving accuracy in object classification and localization tasks by maintaining orthogonal feature relationships and reducing the risk of singularities.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024161006000001_ABST
    Figure 2024161006000001_ABST
Patent Text Reader

Abstract

To provide a neural network architecture (1) for processing measurement data (1b).SOLUTION: A neural network architecture (1) includes a plurality of layers (2, 3 and 4) having a plurality of neurons (2a, 3a and 4a) respectively, and is configured to find its output (8) by processing the respective neurons using a regular offset function (5) for activating (6) of inputs (2b, 3b and 4b) of complex numerals and applying a nonlinear activation function (7) to the activation (6), wherein the activation function is regular as well.SELECTED DRAWING: Figure 1
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The invention relates to a neural network architecture for evaluating measurement data, e.g. images or point clouds, which arise incidentally, in particular in the case of vehicle environment monitoring. [Background technology]

[0002] Background technology For at least partially automated operation of a vehicle or robot in road traffic or on business premises, constant monitoring of the vehicle's or robot's environment is essential. Such environmental monitoring involves collecting records of measurement data, e.g. images or point clouds, and evaluating these records with machine learning models trained on predefined tasks. For this purpose, neural network architectures consisting of multiple layers are often used. Typically, these measurement data are input to an input layer and subsequently pass through one or more intermediate layers before the results of the processing are output from an output layer.

[0003] As long as the measurement data are complex-valued, the conventional approach is to treat the real and imaginary parts of each measurement, or also the magnitude and phase in the polar representation of each measurement, as mutually independent inputs of a neural network architecture. Summary of the Invention [Problem to be solved by the invention]

[0004] Disclosure of the Invention The present invention provides a neural network architecture for processing measurement data. The network architecture includes a number of layers, each layer having a number of neurons. These neurons can be replaced by other processing units with substantially equivalent effects. In the following, for simplicity, only neurons are taken up. [Means for solving the problem]

[0005] In particular, in the neural network architecture, for example, the output of a layer can be fed as input to an adjacent layer, i.e. the input measurement data is first processed in the input layer before the result is further processed in one or more intermediate layers, and the final result of the processing is determined in the output layer, which is output from the neural network architecture.

[0006] Each neuron is configured to process a complex-valued input with a regular offset function for activation. The behavior of the neural network architecture is substantially characterized by trainable parameters of this offset function. These parameters can include, for example, weights by which inputs fed to the neuron are offset for activation. Additionally, these parameters can include, for example, summation biases included in the activation.

[0007] Each neuron is further configured to determine its output by applying a non-linear activation function to the activations so obtained, where the activation function is also regular.

[0008] It was realized that in this way the complete information content of the complex-valued measurement data can be used, which includes not only the real and imaginary parts (or absolute values ​​and phases) of the complex-valued variables, but also the relationship between the real and imaginary parts or between the absolute values ​​and phases. In particular, the training of the neural network architecture can also be carried out using this complete information content. In such a training, the value of a cost function (also called loss function) often occurs concomitantly as feedback on the output of the neural network. From the value of this cost function gradients are determined by so-called backpropagation, along which the trainable parameters of the neural network architecture should be significantly modified in order to achieve an improvement in the value of the cost function with high probability. This backpropagation presupposes that the processing of the measurement data or the training data used during training is carried out in a fully differentiable manner. Here, due to the complex differentiability, the aforementioned relationship between real and imaginary parts can also be fully included in the calculation of the gradient of the parameters to be searched for.

[0009] To clarify this with a simple example, for damped harmonic vibration, the displacement s is the following complex-valued function as a function of time t: s = exp(i ω t-δ t) where ω is the angular frequency and δ is the damping constant. In this complex exponential description, it is directly apparent how the free oscillations with angular frequency ω are modified by the damping term with damping constant δ. In contrast, if the real and imaginary parts of s are described using real-valued trigonometric functions, this relationship becomes less clear.

[0010] Further relevant for the initially mentioned vehicle or robot environmental monitoring are application cases in which the measurement data represent the spatial and / or temporal distribution of at least one electromagnetic field, so that in particular Maxwell's equations, as the basis of electrodynamics, are complex-valued, and this complex-valued dynamics contains much more information than simply juxtaposing real and imaginary parts against each other.

[0011] The electromagnetic field may in particular originate, for example at least in part, from the reflection of an electromagnetic interrogation beam at one or more objects. Measurement data of this kind arises incidentally, for example, when the environment is explored with a radar or lidar beam as an interrogation beam for objects. The amount of information obtainable by full regularization processing can then lead to higher accuracy, for example, in the subsequent identification of the object's location and / or classification of the object's type. Thus, for example, a complex-valued radar signal contains a Doppler spectrum over all physical and / or virtual antenna channels involved in the reception of the signal, as well as the azimuth spectrum of each transmitted radar pulse (chirp). Such a straight spectrum is useful for identifying and classifying objects. Virtual antenna channels exist, for example, when using "multiple-input multiple-output" (MIMO) radars. Holomorphic processing is also advantageous, for example, for the precise determination of the direction of arrival (DOA) of a radar or lidar reflection, which is complicated by constantly changing environmental conditions and the possibility of multipath propagation of each interrogation beam.

[0012] The same applies to other wireless applications where the propagation of electromagnetic fields is preferably represented by complex-valued variables.

[0013] The most obvious advantage of a neural network architecture with a regular activation function, independent of the concrete application, is that it generalizes better to unknown inputs during training than, for example, an architecture whose activation function processes real and imaginary parts separately from each other. This therefore entails that the decision boundary in the input space of the neural network architecture is smoothed, for example, in particular reducing the occurrence of discontinuities that are promoted by, for example, the use of a complex ReLU activation function.

[0014] In a particularly preferred embodiment, the activation function and / or the derivative of this activation function is a conformal mapping. If this conformal mapping is supplied with two complex numbers z1 and z2 that contain an intermediate angle in the complex plane, the result provided by the conformal mapping contains the equivalent intermediate angle in the complex plane. Furthermore, the length ratio between the complex numbers z1 and z2 can be maintained between the results provided by the conformal mapping as well. Thus, although the conformal mapping changes the long-range order in the complex plane, at least some degree of nearby order is maintained. In particular, the representations of mutually orthogonal features remain mutually orthogonal as well after the application of the conformal mapping. Straight orthogonality is a relationship between features that embodies particularly important information. In contrast, if the mapping is not a conformal mapping, the aforementioned representations will produce results that are very close to each other and that can no longer be numerically distinguished from each other.

[0015] In a more particularly preferred embodiment, the activation function has the following form with free coefficients a, b, c, and d: f(z) = (a z + b) / (c z + d) These coefficients can in particular be trained together, for example, when training a neural network architecture. In this way, within the scope of the parameterized approach, it is possible to find the transformation that provides the best results for the overall task to be solved by the neural network architecture. Conventional, everyday activation functions, such as, for example, ReLU, Sigmoid and tanh, do not have this kind of freedom and therefore cannot be adapted to the concrete task to be solved. Training the coefficients together is clearly better motivated here than choosing exactly one concrete activation function from a catalog of available functions.

[0016] Particularly preferably, the coefficients a, b, c, d of the Möbius transformation are real-valued. In this way, the search space is clearly reduced. At the same time, it is guaranteed that the positive imaginary half-plane is mapped onto itself. Furthermore, the probability that the output of the neural network architecture will "collapse" onto the real-valued axis is reduced.

[0017] In a further particularly preferred embodiment, the matrix of coefficients of the Möbius transformation is

number

number

[0018] A neural network architecture may in particular be configured, for example, as a feature extractor. Thus, the outputs of neurons in different layers of the feature extractor indicate the representation of features of different scales and / or complexities in the measured data. One example of a feature extractor is a Convolutional Neural Network (CNN), which generates a feature map by applying a filter kernel to the measured data in a sliding manner. The part of the feature map generated by applying a filter kernel to the measured data is also called the "channel" of the feature map that corresponds to this filter kernel. That is, a feature map is a stack of such channels.

[0019] In a further particularly preferred embodiment, the neural network architecture comprises a task head configured to solve a predefined task from one or more outputs of the feature extractor with respect to the measurement data. The architecture of the task head is here adapted to the predefined task. If the task head is to be switched after training of the neural network architecture, it will be sufficient to train a new task head. In contrast, the feature extractor can, for example, only undergo a shortened further training ("fine-tuning") or be completely fixed in its previously trained state. The task head can, in particular, provide, for example, real-valued results. That is to say, the task head can, for example, initially receive the complex-valued output of the feature extractor and, in so far as such, still have access to the complete information content. This information is then condensed into a solution for the predefined task and simultaneously transferred to a real-valued axis.

[0020] The task head may in particular be configured to determine a classification score for one or more classes of a predefined classification for the measurement data, for example the task head may classify a type of object whose presence is indicated by the measurement data.

[0021] In a further particularly preferred embodiment, a control signal is formed from one or more outputs provided by the neural network architecture, with which a vehicle, a driver assistance system, a robot, a quality control system, an area monitoring system and / or a medical imaging system is controlled. In this way, the reaction performed by the respective technical system in response to the control signal is more likely to be appropriate in the situation embodied by the measurement data.

[0022] The invention also relates to a method for training the neural network architecture described above.

[0023] Within the framework of the method, training records of measurement data are provided, which may for example be annotated (labeled) with a target output for supervised training.

[0024] Here, the term "record" refers to a data set of related data, comparable to the information on an index card in a card index box. A record contains, for example, values ​​of multiple measurement variables that together characterize the operating state of a technical system, or image data and possibly associated metadata. The term "record" is used here instead of "dataset" because the term "dataset" is already exemplified in many ways in the technical terminology of machine learning and refers to a collection of all records, comparable to a card index box containing all index cards.

[0025] The training records are fed to a neural network architecture to be trained and are processed by the neural network architecture into outputs. These outputs are evaluated using a predefined real-valued cost function. Thus, for example, the deviation of the output from a target output can be measured during supervised training. However, for example, the feature extractor can also be self-supervised trained, for example in conjunction with a decoder that tries to reconstruct the original input from the output of the feature extractor.

[0026] The parameters characterizing the behavior of the neural network architecture are optimized with the aim of improving the evaluation by the cost function during further processing of the training records. These parameters include not only the usual tuning screws of neural networks, such as weights and bias values ​​for offsetting the input to the activation of a neuron, but also the free coefficients of the parameterized approach for the canonical activation function. That is, these coefficients can be integrated into a vector, for example, together with the weights and bias values. However, it is also possible to update in the training step, for example, in switching, on the one hand, a vector with weights and bias values, and on the other hand, a vector with coefficients. Each gradient is thus more meaningful in itself for a concrete application than a gradient in a "mixed" space.

[0027] As explained above, in this way, a large flexibility is created with regard to the selection of the activation function. From a large class of functions that allows a parameterized approach, the one that is best for achieving the goal that is ultimately pursued in the neural network architecture is selected exactly. This selection is therefore automatically motivated better than, for example, manually selecting the activation function. As a parameterized approach, for example, the class of Möbius transformations introduced above can be selected.

[0028] In a particularly preferred embodiment, within the scope of the optimization step, it is checked whether the deviation of the determinant det(A) of the matrix A formed from the coefficients of the parameterized approach for the regular activation function from a preset value exceeds a preset absolute value. If this is the case, the elements of the matrix are calculated by multiplying the square root of the determinant det(A) by

number

[0029] The neural network architecture may in particular be, for example, computer-implemented. The invention therefore also relates to a computer program comprising machine-readable instructions for causing one or more computers to implement at least one instance of the neural network architecture described above when the computer program is executed on the one or more computers. In this context, in particular, for example graphic processors, GPUs, vehicle control devices or embedded systems for incorporation into other devices, each of which is also capable of executing machine-readable instructions, may also be considered as computers.

[0030] The invention likewise relates to a machine-readable data carrier and / or a download product comprising one or more computer programs, a download product being a digital product that can be transferred over a data network, i.e. downloaded by a user of the data network, which can be sold for instant download, for example in an online shop.

[0031] Furthermore, one or more computers and / or computing instances may comprise one or more computer programs, machine-readable data carriers, or downloadable products.

[0032] Further improvements of the invention will be described in more detail below in conjunction with the description of preferred embodiments of the invention based on the drawings. [Brief description of the drawings]

[0033] [Figure 1] FIG. 1 illustrates an embodiment of a neural network architecture 1. [Diagram 2] FIG. 2 illustrates an exemplary use of output 1c of neural network architecture 1. [Diagram 3] FIG. 1 illustrates an embodiment of a method 100 for training a neural network architecture 1. [Figure 4a] On similar training data (Fig. 4a), we compare the behavior of network architecture 1 trained with an irregular activation function (Fig. 4b) to the behavior of network architecture 1 trained with a regular activation function 7 (Fig. 4c). [Figure 4b] On similar training data (Fig. 4a), we compare the behavior of network architecture 1 trained with an irregular activation function (Fig. 4b) to the behavior of network architecture 1 trained with a regular activation function 7 (Fig. 4c). [Figure 4c] On similar training data (Fig. 4a), we compare the behavior of network architecture 1 trained with an irregular activation function (Fig. 4b) to the behavior of network architecture 1 trained with a regular activation function 7 (Fig. 4c). DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0034] Working Example 1 is a schematic diagram of an embodiment of a neural network architecture 1. The neural network architecture 1 includes an input layer 2 with neurons 2a that receives measurement data 1b as input. These measurement data are then successively processed in a number of hidden layers 3 with neurons 3a. The input layer 2 and the hidden layers 3 together form a feature extractor 9a.

[0035] The neural network architecture 1 additionally further comprises an output layer 4 which processes the output of the feature extractor 9a into a final result 1c provided by the entire neural network architecture 1 with respect to the task to be solved, i.e. the output layer 4 is simultaneously also used as a task head 9b which continues to process the output of the feature extractor 9a.

[0036] In the center of FIG. 1, it is exemplarily shown how the processing is carried out in a neuron 3a in the hidden layer 3. The input 3b fed to the neuron 3a is processed with a regular offset function, here a weighted sum, for the activation 6. The weights of this weighted sum and an optional additive bias value added to the activation 6 belong to the parameters 1a characterizing the behavior of the neural network architecture 1. The activation 6 is processed by a regular nonlinear activation function 7 for the output 8 of the neuron 3a. In the example shown in FIG. 1, an approach is chosen for this nonlinear activation function 7 that is parameterized with free coefficients 7a. These coefficients 7a thus likewise become parameters 1a characterizing the behavior of the neural network architecture 1.

[0037] 2 shows exemplarily how the output 1c provided by the neural network architecture 1 can be further used. A control unit S receives the output 1c of the neural network architecture 1 and determines a drive control signal 10 by means of which a vehicle 50, a driver assistance system 51, a robot 60, a quality control system 70, an area monitoring system 80 and / or a medical imaging system 90 are driven and controlled.

[0038] FIG. 3 is a schematic flow chart of one embodiment of a method 100 for training the neural network architecture 1 described above.

[0039] In step 110, training records 1b* of measurement data 1b are provided. These training records 1b* are fed in step 120 to a neural network architecture 1, which processes them into outputs 1c. The outputs 1c so obtained are evaluated in step 130 using a predefined real-valued cost function 11. Here, an evaluation 11a occurs.

[0040] In step 140, parameters 1a characterizing the behavior of the neural network architecture 1 are optimized with the aim of improving the evaluation 11a by the cost function 11 during further processing of the training records 1b, including the free coefficients 7a of the parameterized approach for the regular activation function 7. The state in which the optimization of the parameters 1a and 7a is completed is indicated by the symbols 1a* and 7a*. These parameters 1a, 7a also determine the state 1* in which the training of the neural network architecture 1 is completed.

[0041] Within the framework of this optimization step, according to block 141, it is possible in particular to check whether the deviation of the determinant det(A) of the matrix A, formed, for example, from the coefficients of the parameterized approach for the regular activation function, from a preset value exceeds a preset absolute value. If this is the case (truth value 1), according to block 142, the elements of this matrix are calculated by multiplying the square root of the determinant det(A) by 1 / 2.

number

[0042] Starting from similar training data, Figure 4 illustrates how switching from an irregular to a regular activation function affects the training of neural network architecture 1. In the example shown in Figure 4, neural network architecture 1 is a binary classifier that assigns a single complex-numbered input 1b to one of two possible classes C1 and C2.

[0043] Figure 4a shows the records 1b* of the training data. For each record 1b*, the imaginary part Im(1b*) is plotted against the real part Re(1b*). Record 1b* on curve C1 is labeled with class C1 as the target output. Record 1b* on curve C2 is labeled with class C2 as the target output.

[0044] Figure 4b shows how a neural network architecture trained with the record 1b* shown in Figure 4a assigns exemplary measurement data 1b in a plane extended by their real part Re(1b) and imaginary part (1b) to which class as output 1c each of these measurement data 1b is assigned. The activation function is here a singular complex ReLU function. The lack of complex differentiability is manifested here in the form of a sharp bend or discontinuity in the decision boundary between classes C1 and C2.

[0045] Figure 4c shows, in comparison, how switching to a regular Möbius transform f(z)=1-(1 / z) as activation function 7 works. It can be clearly seen that the decision boundary between classes C1 and C2 is now noticeably smoother; sharp bends and discontinuities have completely disappeared. This also makes the neural network architecture 1 more robust in the sense that small changes in the measurement data 1b are less likely to lead to large changes in the output 1c.

Claims

1. 1. A neural network architecture (1) for processing measurement data (1b), the neural network architecture (1) comprising a number of layers (2, 3, 4) each having a number of neurons (2a, 3a, 4a), each neuron configured to process a complex-valued input (2b, 3b, 4b) with a regular offset function (5) for activation (6) and to determine its output (8) by applying a non-linear activation function (7) to the activations (6), the activation functions being also regular.

2. The activation function (7) and / or the derivative of the activation function (7) are expressed as two complex numbers z 1 and z 2 After being applied to the two complex numbers z 1 and z 2 2. The neural network architecture of claim 1, wherein the intermediate angles between are maintained in the complex plane.

3. The activation function (7) has the following form with free coefficients a, b, c, and d: f(z)=(a・z+b) / (c・z+d) 3. The neural network architecture (1) according to claim 1 or 2, comprising a Möbius transformation of

4. 4. The neural network architecture (1) of claim 3, wherein the coefficients a, b, c, d of the Moebius transform are real-valued.

5. The matrix of coefficients of the Moebius transform [0010] 5. The neural network architecture (1) according to claim 3 or 4, wherein the determinant det(A) deviates from a preset value by at most a preset absolute value.

6. 6. The neural network architecture (1) according to claim 1, wherein the neural network architecture (1) is at least partially configured as a feature extractor (9a), the outputs of neurons (2a, 3a, 4a) in different layers (2, 3, 4) of the feature extractor (9a) being indicative of the degree of representation of features of different scales and / or complexities in the measurement data (1b).

7. 7. The neural network architecture (1) of claim 6, further comprising a task head (9b) configured to solve a predefined task from one or more outputs of the feature extractor (9a) on the measurement data (1b).

8. 8. The neural network architecture (1) according to claim 7, wherein the task head (9b) is adapted to determine a classification score for one or more classes of a predefined classification for the measurement data (1b).

9. Use of a neural network architecture (1) according to any one of claims 1 to 8 for processing measurement data (1b) representative of the spatial and / or temporal distribution of at least one electromagnetic field.

10. The use according to claim 9 , wherein the electromagnetic field originates, at least in part, from reflection of an electromagnetic interrogation beam at one or more objects.

11. a drive control signal (10) is formed from one or more outputs (1c) provided by said neural network architecture (1); 11. The use according to claim 9 or 10, wherein the driving control signal (10) is used to drive and control a vehicle (50), a driving assistance system (51), a robot (60), a quality control system (70), an area monitoring system (80) and / or a medical imaging system (90).

12. A method (100) for training a neural network architecture (1) according to any one of claims 1 to 8, comprising the steps of: - a step (110) in which a training record (1b*) of measurement data (1b) is provided; a step (120) in which said training records (1b*) are fed to said neural network architecture (1) and processed by said neural network architecture (1) to an output (1c); a step (130) in which the output (1c) is evaluated using a pre-defined real-valued cost function (11); - a step (140) in which parameters (1a) characterizing the behavior of said neural network architecture (1) are optimized with the aim of improving its evaluation (11a) by said cost function (11) upon further processing of said training records (1b); Including, The method (100), wherein the parameters (1a) also include free coefficients (7a) of a parameterized approach for a regular activation function (7).

13. Within the scope of said optimization step (140), It is checked (141) whether the deviation of the determinant det(A) of the matrix A formed from the coefficients of the parameterized approach of the regular activation function from a preset value exceeds a preset absolute value; If this is the case, the elements of the matrix are the square roots of the determinant det(A). [0025] The method (100) of claim 12, wherein the first division factor is a positive integer multiple of 142.

14. 14. A computer program comprising machine-readable instructions for causing one or more computers to realize at least one instance of the neural network architecture (1) of any one of claims 1 to 8 and / or to perform the method (100) of claim 12 or 13, when the computer program is executed on the one or more computers.

15. A machine-readable data carrier and / or download product comprising a computer program according to claim 14.

16. One or more computers comprising a computer program according to claim 14 and / or a machine-readable data carrier and / or a download product according to claim 15.