Characterization of an artificial neural network

Abstract

Method (200) for determining a characterization (205) of a trained artificial neural network (105) for detecting an object in the vicinity of a motor vehicle (115), wherein the method (200) comprises the following steps: receiving a first artificial neural network (105); determining responses of the first network (105) to predetermined stimuli (210); wherein a stimulus (210) comprises a predetermined object in a predetermined relative position to the motor vehicle (115); training a Kohonen map (220) with the stimuli (210) and their respective associated responses; determining a characterization (205) of the first artificial neural network (105) based on the trained Kohonen map (220); comparing the characterization (205) with another characterization (205) determined in the same way with respect to a second trained artificial neural network (105);Verifying the behavior of the first artificial neural network (105) based on the comparison; and deploying the first artificial neural network (105) after ensuring an expected function on board the motor vehicle (115), whereby the motor vehicle (115) is controlled based on the first artificial neural network (105).

Description

[0001] The present invention relates to the characterization of an artificial neural network. In particular, the invention relates to the determination of a characteristic value that characterizes the artificial neural network.

[0002] An artificial neural network is trained using a large amount of training data to perform a predetermined task. The quality with which the network can perform the task depends directly on the quality and quantity of the training data. However, it is usually not possible to determine from an artificial neural network what training data it was trained on. The network can be described by specifying a topology of neurons in an input layer, intermediate layers, and an output layer, as well as the weights of the connections between the neurons. However, this information is not usually available in an existing network, so the network must be characterized based on its behavior.

[0003] Relevanter Stand der Technik ist offenbart in Féraud, Raphael; Clérot, Fabrice: A methodology to explain neural network classification. In: Neural networks, 15, 2002, 2, 237-246, in Wang, Min, et al.: Deep supervised quantization by self-organizing map. In: Proceedings of the 25th ACM international conference on Multimedia, 2017, 1707-1715, in De, Tanusree, et al.: Explainable AI: a hybrid approach too generate human-interpretable explanation for deep learning prediction. In: Procedia Computer Science, 168, 2020, 40-48, und in Hamel, Lutz: Visualization of support vector machines with unsupervised learning. In: 2006 IEEE Symposium on Computational Intelligence and Bioinformatics and Computational Biology, 2006, 1-8.

[0004] An exhaustive analysis of the behavior of the neural network is usually not practical. The present invention therefore aims to provide an improved technique for characterizing a trained neural network. The invention achieves this objective by means of the subject matter of the independent claims. Dependent claims describe preferred embodiments.

[0005] According to a first aspect of the present invention, a method for determining a characterization of a trained artificial neural network for detecting an object in the vicinity of a motor vehicle comprises the following steps: receiving a first artificial neural network; determining responses of the first network to predetermined stimuli, wherein a stimulus comprises a predetermined object in a predetermined relative position to the motor vehicle; training a Kohonen map with the stimuli and their respective associated responses; determining a characterization of the first artificial neural network based on the trained Kohonen map; comparing the characterization with a further characterization determined in the same way with respect to a second trained artificial neural network; and verifying the behavior of the first artificial neural network based on the comparison.and deployment of the first artificial neural network after an expected function has been ensured on board the motor vehicle, with the motor vehicle being controlled on the basis of the first artificial neural network.

[0006] The Kohonen map, also called a Kohonen network or self-organizing map, is a type of artificial neural network. Designed for unsupervised learning, the Kohonen map relies on the linear or planar arrangement of neurons for processing multidimensional stimuli. Neurons that process similar stimuli are grouped in relatively close topological proximity. Therefore, the Kohonen map is particularly well-suited for the automatic grouping of data.

[0007] The Kohonen map can be used to characterize the behavior of a trained artificial neural network for predetermined stimuli. It may be sufficient to use only a few different stimuli. In particular, a core functionality of the trained network can be better represented in this way. Training the Kohonen map can be unsupervised, making it easy to automate the process.

[0008] The characterization can be determined, in particular, based on the topography of the trained Kohonen map. Specifically, the trained Kohonen map can comprise a number of neurons, each neuron being assigned a combination of weights, and the characterization is determined based on these weights. In a simple embodiment, the characterization can be specified by the arrangement of the neurons' weights. A Kohonen map typically includes input neurons for stimuli and map neurons that are connected to all input neurons but not to each other. Starting from a predetermined initial state, which, for example, assigns random weights to the map neurons, a stimulus is applied to the input neurons. The map neuron whose weights best match the stimulus is adjusted to further align its weights with the stimulus.Map neurons that are in relative topographic proximity are also adjusted, but by an amount that is inversely proportional to their distance.

[0009] A preferred characterization method is to determine the neurons based on a grouping of neurons assigned similar weights and located topographically close to one another. This approach leverages the property of a Kohonen map to arrange map neurons that respond to similar stimuli in close topological proximity. The distribution and weighting of map neurons on the trained Kohonen map typically encompasses regions of neurons with similar weights. These regions can then be grouped together. If the Kohonen map is two-dimensional, an image compression method such as JPEG can be used.

[0010] A vector can be generated from the combination of a stimulus to the artificial neural network and its response to that stimulus. This vector can then be used to train the Kohonen map. All vectors that can be generated from the predetermined stimuli together form an epoch, which is typically used in random or pseudorandom order to train the Kohonen map. In multiple training iterations, the vectors of the epoch are preferentially used in a different order to train the Kohonen map. After each iteration of the vectors in an epoch, it can be checked whether the topography or the weights of the map neurons have changed by more than a predetermined amount. If so, training can continue; otherwise, the training can be considered complete.

[0011] The map neurons of the Kohonen map require initialization for learning, and different initializations can lead to different trained Kohonen networks. It is therefore preferable to include the initialization of the map neurons in the characterization. The untrained Kohonen map comprises a number of neurons, each assigned a combination of weights, and the characterization is determined based on these weights. The weights correspond to the initial state, which can be determined, for example, randomly, pseudorandomly, or systematically according to a predetermined heuristic. Considering the initial state ensures that the Kohonen map can be trained reproducibly based on the stimuli. In particular, it ensures that two Kohonen maps trained on the same stimuli provide comparable characterizations.

[0012] The artificial neural network is designed to detect an object. The object can be moving or stationary and is typically located in the vicinity of a motor vehicle. The object can be scanned using one or more sensors, such as a camera, radar sensor, and LiDAR sensor. The artificial neural network can then detect the object, distinguishing between a predetermined number of different objects. Training the artificial neural network is usually done using a large number of training datasets, each containing a scan of an object and a classification of the object. These training datasets can be reused many times to train the artificial neural network.

[0013] The stimuli described herein may or may not be included in the training data. Depending on the number of predetermined stimuli, training the Kohonen map can be significantly faster than training the artificial neural network. The Kohonen map can therefore be advantageously used to characterize the artificial neural network with respect to the predetermined stimuli.

[0014] Since artificial neural networks are usually not comparable with respect to their internal structure, determining the Kohonen maps allows for a comparison of the networks' behavior. In particular, it can be determined whether two artificial neural networks exhibit corresponding behavior in response to predetermined stimuli.

[0015] Based on this comparison, a measure of the similarity of the trained artificial neural networks can be determined. By training the Kohonen maps based on the responses of the artificial neural networks to predetermined stimuli, the characterizations derived from the Kohonen maps allow for the determination of a similarity measure. The similarity can encompass the topology and / or the weights of the map neurons and can be determined, for example, using an algorithm that assesses the similarity between two two-dimensional structures, particularly images.

[0016] In a training course, a measure of the similarity of training data used to train artificial neural networks can be determined based on a comparison of Kohonen maps. This requires that the two artificial neural networks have comparable topologies. In particular, this allows for the detection of unauthorized use of training data for training an artificial neural network. Unauthorized copying or use of the training data can thus be identified and prosecuted accordingly.

[0017] In a further extension, characterizations are determined for artificial neural networks trained using successively structured collections of training data. This allows researchers to track how the behavior of an artificial neural network changes as the training data is further developed. A sequence of appropriately provided Kohonen maps can be used to interpolate the behavior of an artificial neural network created between two known configurations of training data. For example, if it is necessary to determine whether a network was trained using training data from a database, a Kohonen map can be determined for that network. This map can then be compared to a Kohonen map derived from at least two other Kohonen maps whose associated artificial neural networks were trained at different times using the training data.

[0018] The technique of characterizing a trained artificial neural network using a Kohonen map can also be used to ensure the integrity of an artificial neural network. For example, an artificial neural network deployed on board a vehicle can be updated and its behavior checked according to the invention. The Kohonen map-based characterization can thus serve as a kind of fingerprint of the artificial neural network at the application level.

[0019] An artificial neural network is received; the network can be characterized using a technique presented herein; and the resulting characterization can be compared to a received characterization. This verification is similar to a known checksum or hash value verification, except that instead of checking the data structure of the artificial neural network for exact match with a reference, the network's functionality is verified. Furthermore, it can be determined that the identified characterization differs from the received characterization by more than a predetermined amount. Appropriate action can then be taken to address this discrepancy. For example, replacing an existing artificial neural network with the received neural network can be rejected.Furthermore, a warning can be issued or a new artificial neural network can be requested. This can prevent the accidental or malicious execution of an unsuitable or dysfunctional artificial neural network.

[0020] The artificial neural network and its characterization can be received, for example, on board a vehicle. The data can be provided by an external entity, which can optionally also update other data structures or computer programs on board the vehicle. Training the Kohonen map using the received artificial neural network can be performed on board the vehicle after a stationary state of the vehicle has been determined. This ensures the vehicle's mobility, particularly if training the Kohonen map requires significant processing resources. It also prevents any interference with other processes running on the vehicle. The vehicle is configured to be controlled based on the received artificial neural network.In particular, the vehicle can be controlled based on an object in its environment detected by the artificial neural network, especially to avoid a collision. Training the Kohonen map based on the received network can take place in a virtualized or secure environment (sandbox). The received network is only used after an expected function has been verified on board the vehicle.

[0021] According to a second aspect of the present invention, a device for determining a characterization of a trained artificial neural network for detecting an object in the vicinity of a motor vehicle comprises a receiving device for input to a first artificial neural network; a device for providing predetermined stimuli; a device for training a Kohonen map with the stimuli and the respective associated responses of the first artificial neural network to the stimuli; and a device for determining the characterization based on the trained Kohonen map.and a device for comparing the characterization with another characterization determined in the same way with respect to a second trained artificial neural network, for checking the behavior of the first artificial neural network based on the comparison, for deploying the first artificial neural network after an expected function has been ensured on board the motor vehicle, and for controlling the motor vehicle based on the first artificial neural network.

[0022] The device can be particularly advantageously used on board a vehicle. According to a further aspect of the present invention, a vehicle comprises a device as described herein. The vehicle is configured to be controlled on the basis of the trained artificial neural network.

[0023] The processing equipment may be configured to execute all or part of a method described herein. For this purpose, the processing equipment may include a programmable microcomputer or microcontroller, and the method may be in the form of a computer program product with program code. The computer program product may also be stored on a computer-readable data carrier. Features or advantages of the method may be transferred to the equipment or vice versa.

[0024] The invention will now be described in more detail with reference to the attached figures, in which: Fig. 1 a system for providing an artificial neural network; Fig. 2. a device for characterizing an artificial neural network; and Fig. Figure 3 shows an exemplary representation of a progression of similarities between two artificial neural networks.

[0025] Fig. Figure 1 shows a system 100 for providing an artificial neural network 105, which is hereinafter also abbreviated as ANN. The system 100 comprises a provider 110, a client 115, and a communication link 120 established between them, preferably wireless. The provider 110 can, for example, be configured as a central unit, a server, or a service in a cloud. The client 115 is shown here by way of example as a vehicle, in particular a motor vehicle. The client 115 may require the ANN 105 for its operation or to fulfill a core functionality. In this case, a sensor 125 for scanning its environment is mounted on board the vehicle 115. The sensor 125 can, for example, be an ultrasonic sensor, a camera, a LiDAR sensor, or a radar sensor.A sample result from sensor 125 can be provided to the KNN 105, and depending on a response from the KNN 105, the vehicle 115 can be controlled, for example, by means of an actuator 130. The actuator 130 can control longitudinal movement of the vehicle 115 and may include, for example, a drive or braking system. The actuator 130 can also be configured for lateral control of the vehicle 115 and may, in particular, include a steering device. A communication device 135 forms an endpoint of the communication link 120. A KNN 105 received via the communication device 135 can be checked by means of a device 140 before it is used on board the vehicle 115.

[0026] The supplier 110 provides a communication device 150 as the second endpoint of the communication link 120. A processing device 155 is configured to train a KNN 105 with training data from a training database 160. Different KNNs 105 can be provided for different versions or subsets of training data in the training database 160, which can be assigned, for example, to different vehicles 115, different levels of maturity, or different scopes. A trained KNN 105 can be provided via the communication device 150 at the initiative of the supplier 110 or the client 115. Preferably, a device 140 for verifying a KNN 105 is also provided here, the functionality of which can correspond to that of the device 140 on the vehicle 115 side.

[0027] The devices 140 are each configured to provide a functional characterization of a KNN 105. The characterization is determined on the basis of a Kohonen map and forms a representation of a functionality of the KNN 105 with respect to predetermined stimuli.

[0028] Fig. Figure 2 shows the function of a device 140 for characterizing a KNN 105. Fig. 2 simultaneously serves to explain a corresponding procedure 200 for determining the characterization.

[0029] A KNN 105 is trained using training data from the training database 160. Supervised learning can be used in this process. Specifically, the KNN 105 can be trained to provide a predetermined response to a predetermined stimulus 210. The stimulus 210 can, for example, originate from a sensor 125, and the response can include the classification of an object scanned by the sensor 125. In other words, the KNN 105 can be trained to recognize objects.

[0030] To determine a characterization 205 of the KNN 105, predetermined stimuli 210 can be defined, which can correspond to exemplary sensor signals during the usual operation of the KNN 105. For example, if the KNN 105 is configured to detect one or more objects in the vicinity of the vehicle 115, a stimulus 210 can comprise a predetermined object in a predetermined relative position to the vehicle 115. For testing purposes, the surrounding area can be hidden. Responses of the KNN 105 are then determined for the predetermined stimuli 210. A stimulus 210 and an associated response of the KNN 105 can be combined in a vector 215, so that a vector 215 is defined for each stimulus 210.

[0031] A Kohonen card 220 is loaded with an initialization 225, which includes weights for a multitude of card neurons, each connected to several input neurons. The initialization 225 can be determined randomly or pseudorandomly. The initialized Kohonen card 220 can then be fed vectors 215 in any order, so that the Kohonen card 220 learns the vectors 215. All vectors 215 can be grouped into an epoch, and preferably, the vectors 215 of an epoch are applied to the Kohonen card 220 in a random or pseudorandom order. By changing the order of the vectors 215, a multitude of epochs can be created, each of which can be used to train the Kohonen card 220.If the Kohonen card 220 changes by less than a predetermined amount between two successive epochs, the state of the Kohonen card 220 can be recognized as stable and the learning process can be terminated.

[0032] Based on the Kohonen map 220, and in particular a topology and / or weights of the map neurons, a characterization 205 can now be determined. The characterization 205 can group areas of map neurons that are topographically close to one another and / or have similar weights. The Euclidean distance between two map neurons with respect to their arrangement can be used to determine their distance. Furthermore, information regarding the initialization 225 and / or the stimuli 210 used can be incorporated into the characterization 205. The characterization 205 can then represent a reproducible characterization of the function of the KNN 105 on the predetermined stimuli 210.

[0033] The KNN 105 and the characterization 205, with or without the information on the initialization 225 or the stimuli 210, can be integrated or transmitted separately from a provider 110 to a client 115. The client 115 can determine a characterization 205 with respect to the received KNN 105 in the manner described and compare it with a previously received characterization 205. The comparison can represent a measure of similarity between the KNN 105 that were used to determine the respective characterization 205.

[0034] Fig. Figure 3 shows an exemplary representation of a trend 305 of similarities between two KNN 105. In the horizontal direction, a time or a version number is plotted, and in the vertical direction, a similarity between two predetermined KNN 105. 0 denotes the absence of similarities and 1 practical identity.

[0035] The depicted curve 305 shows an example of the chronological development of comparisons between KNN 105 from different development lines. Individual measurements are marked by circles. In section 310, the determined similarities increase significantly. It can be concluded that training data used to train the KNN 105 of one development line has been incorporated into the development line of the other KNN 105 in section 310. The following section can examine in more detail whether training data from a training database 160 was indeed used and how to respond to this. Reference sign 100 System 105 artificial neural network (ANN) 110 suppliers 115 clients 120 communication links 125 Sensor 130 actuator 135 Communication device 140 Device for checking a KNN 150 communication equipment 155 Processing unit 160 training database 200 procedures 205 Characterization 210 Stimulus 215 vector 220 Kohonen card 225 Initialization 305 Course Section 310

Claims

Method (200) for determining a characterization (205) of a trained artificial neural network (105) for detecting an object in the vicinity of a motor vehicle (115), wherein the method (200) comprises the following steps: receiving a first artificial neural network (105); determining responses of the first network (105) to predetermined stimuli (210); wherein a stimulus (210) comprises a predetermined object in a predetermined relative position to the motor vehicle (115); training a Kohonen map (220) with the stimuli (210) and their respective associated responses; determining a characterization (205) of the first artificial neural network (105) based on the trained Kohonen map (220); comparing the characterization (205) with another characterization (205) determined in the same way with respect to a second trained artificial neural network (105);Verifying the behavior of the first artificial neural network (105) based on the comparison; and deploying the first artificial neural network (105) after ensuring an expected function on board the motor vehicle (115), whereby the motor vehicle (115) is controlled based on the first artificial neural network (105). Method (200) according to claim 1, wherein the characterization (205) is determined on the basis of a topography of the trained Kohonen map (220). Method (200) according to claim 1 or 2, wherein the trained Kohonen map (220) comprises a number of neurons, each neuron being assigned a combination of weights and the characterization (205) being determined on the basis of the weights. Method (200) according to claim 3, wherein the characterization (205) is determined on the basis of a summary of neurons to which similar weights are assigned and which are topographically adjacent. Method (200) according to one of the preceding claims, wherein the Kohonen card (220) is trained until it has reached a stable state. Method (200) according to one of the preceding claims, wherein the untrained Kohonen map (220) comprises a number of neurons, each neuron being assigned a combination of weights, and the characterization (205) is determined on the basis of the weights. Method according to one of the preceding claims, wherein a measure of the similarity of the trained artificial neural networks (105) is determined on the basis of the comparison. Method (200) according to one of the preceding claims, wherein a measure of the similarity of training data with which the artificial neural networks (105) are trained is determined on the basis of the comparison. Method (200) according to one of the preceding claims, wherein characterizations (205) regarding artificial neural networks (105) are determined which are trained by means of successive collections (160) of training data. Method (200) according to one of the preceding claims, wherein one of the artificial neural networks (105) and the characterization of the other artificial neural network (105) are received from an external location (110). Method (200) according to claim 10, wherein it is determined that the determined characterization (205) differs from the received characterization (205) by more than a predetermined measure. Method (200) according to claim 10 or 11, wherein the training of the Kohonen card (220) is carried out on board a motor vehicle (115) after a parked state of the motor vehicle (115) has been determined. Device (140) for determining a characterization (205) of a trained artificial neural network (105) for detecting an object in the vicinity of a motor vehicle (115), the device (140) comprising: a receiving device (135) for inputting a first artificial neural network (105); a device (135, 140) for providing predetermined stimuli (210); a device (140) for training a Kohonen map (220) with the stimuli (210) and the corresponding responses of the first artificial neural network (105) to the stimuli (210); a device (140) for determining the characterization (205) based on the trained Kohonen map (220);and a device (140) for comparing the characterization (205) with another characterization (205) determined in the same way with respect to a second trained artificial neural network (105), for checking the behavior of the first artificial neural network (105) on the basis of the comparison, for deploying the first artificial neural network (105) after an expected function has been ensured on board the motor vehicle (115), and for controlling the motor vehicle (115) on the basis of the first artificial neural network (105).

Images