Method and apparatus for implementing functional testing

Abstract

Method and device for carrying out a functional test. Method and device for determining an operating variable, in particular for testing a technical system for at least partially autonomous driving, actuator control and / or image processing, in particular for testing a technical system for image processing of such a system, characterized in that, for at least one intermediate layer (104) of an artificial neural network (100), information about the activation state of neurons (104-1,..., 104-k) of the at least one intermediate layer (104) is determined from input data, wherein at least one weight is provided for at least one of the neurons (104-1,..., 104-k) in addition to the weights of the artificial neural network (100), wherein a weighted activation state of the neurons (104-1,..., 104-k) of the at least one intermediate layer (104) is determined from the weight and from the information about the activation state, and wherein the operating variable is determined from the weighted activation state.

Description

Technical Field

[0001] This invention relates to a method and apparatus for performing functional testing, particularly functional testing of artificial neural networks in a system. Background Technology

[0002] Operational safety is particularly important in vehicles, especially in driver assistance systems or systems used for at least partial autonomous driving. The same applies to the use of artificial neural networks in medical technology devices, in generating labels for input data, or in systems for pattern or facial recognition.

[0003] To evaluate the quality of artificial neural networks, laborious and costly testing methods are used. To measure this quality, it is particularly important to determine how well the tests are implemented within the testing method and when the testing method can be terminated.

[0004] Therefore, it is desirable to demonstrate a reliable method for evaluating the quality of artificial neural networks. Summary of the Invention

[0005] This is achieved by the method and apparatus for performing functional tests according to the present invention.

[0006] A method for determining operating parameters (particularly for testing technical systems involving at least partial autonomous driving, control actuators, and / or image processing, and especially for testing technical systems involving image processing of such systems) specifies that, for at least one intermediate layer of an artificial neural network, information about the activation states of neurons in that at least one intermediate layer is determined based on input data, wherein at least one weight is provided for at least one of these neurons in addition to the weights of the artificial neural network, wherein a weighted activation state of the neurons in that at least one intermediate layer is determined based on this weight and based on the information about the activation states, and wherein the operating parameters are determined based on the weighted activation states. Preferably, there is access to information about the activation states of all neurons in all intermediate layers of the artificial neural network. Information about the activation states of an intermediate layer is, for example, an activation map, that is, the output of that intermediate layer.

[0007] To perform functional testing, the method specifies that, for at least one intermediate layer of an artificial neural network, information about a first activation state of neurons in the at least one intermediate layer is determined based on first input data, while information about a second activation state of neurons in the at least one intermediate layer is determined based on second input data, and wherein the at least one weight is determined based on the information about the first activation state and the information about the second activation state. In this example, the operating parameter is coverage. The coverage can be distinguished between significant and insignificant contributions to coverage based on whether the neurons in the intermediate layer are activated or deactivated.

[0008] In one aspect, the method specifies that: information about a first activation state is defined by neurons in the at least one intermediate layer that are activated and / or deactivated based on first input data; and / or information about a second activation state is defined by neurons in the at least one intermediate layer that are activated and / or deactivated based on second input data. This means that the coverage is determined based on an activation map from multiple tests containing information about activated and / or deactivated neurons.

[0009] The method preferably specifies that: a first vector having information about a first activation state is determined, wherein a second vector having information about a second activation state is determined, and wherein the operating parameter and / or the at least one weight is determined based on the first vector and the second vector, in particular as a scalar.

[0010] The method preferably specifies that the distance between the first vector and the second vector is determined. This is a parameter that can be determined particularly easily.

[0011] The method preferably specifies that: first input data defines reference data points, wherein a reference distribution of information about a first activation state is determined based on a large amount of the first input data; second input data defines data points to be evaluated, wherein a distribution of information about a second activation state is determined based on a large amount of the second input data; and wherein the operating parameter and / or the at least one weight is determined based on the distance between the distribution and the reference distribution. This distance is particularly well-suited as a parameter for evaluation.

[0012] The method preferably specifies that the distance between the distribution and the reference distribution is limited by the probability that the distribution is within the reference distribution.

[0013] The method preferably specifies that at least one prototype point is determined based on the reference distribution, wherein at least one data point is determined based on the distribution, and wherein the distance between the distribution and the reference distribution is limited by the distance between the data point and the prototype point.

[0014] In one aspect, the method specifies that: information about a first path is determined based on information about a first activation state from multiple intermediate layers, through which first input data is mapped to a target; information about a second path is determined based on information about a second activation state from multiple intermediate layers, through which second input data is mapped to the target; and the running parameter and / or the at least one weight is determined based on the information about the first path and based on the information about the second path. By incorporating these paths, it is possible to consider which activations are important for a particular target.

[0015] The method preferably specifies that a large amount of input data is provided, particularly from data records, wherein the operating parameter is determined based on a plurality of weighted activation states, which are determined based on the large amount of input data. This allows the coverage to be determined for the large amount of input data. If the coverage indicates, for example, that each of these neurons is activated at least once, the functional test is successful. This is used to check whether the quality objective has been achieved. If the quality objective has not been achieved, the method can continue using other input data.

[0016] The method preferably specifies that, for the output of a unit in an artificial neural network, a vector is determined that defines information about the activation state or weighted activation state of the neuron in that unit, wherein the activation state indicates that the neuron is activated if the output parameter of the neuron in that unit exceeds a threshold or is within a predetermined range, wherein a large number of vectors are determined for a large number of units in the artificial neural network, wherein multiple vectors among the large number of vectors define the state of elements of a state vector, wherein if information about the activation state of one of the multiple vectors indicates that at least one neuron is activated, then an element of the state vector indicates that element is activated, and wherein the operating parameter is defined by the frequency with which the element of the state vector is activated for at least the first input data and the second input data. In the intermediate layer of the artificial neural network, multiple neurons constitute a unit. The output of a unit is represented by a vector. The outputs of multiple units are represented by multiple vectors and mapped to an element of the state vector. Multiple elements of the state vector can be determined based on the different unit outputs caused by different multiple vectors. Thus, activation is considered according to the architecture of the artificial neural network.

[0017] This method specifies that, in order to determine the frequency of activation of an element in the state vector, the number of times the element is activated is summed or the value of a function depending on the number of times the activation occurs is determined. Standardization may also be specified. In an object recognition method, an object can be identified in a first digital image based on first input data. A bounding box defines a region of pixels in the first digital image, which are assigned to the object in the object recognition method. In the first digital image, the object itself is defined by the determined pixel values ​​of the pixels representing the object. The first input data is defined by the pixel values ​​of the pixels in the first digital image. In an artificial neural network, the first input data activates specific neurons. A feature map defines a group of neurons that are activated based on the pixels in the first digital image representing the object. To verify the feature map, the first input data is determined, for example, based on a first digital image with an unmasked object. Then, the object is masked, for example, by determining a second digital image based on the first digital image, in which pixels in the lower portion of the bounding box are masked while other pixels remain unchanged. In this context, occlusion means that the images have variations in their color, saturation, and / or grayscale levels. Next, second input data is determined based on a second digital image containing the occluded object. Then, a first average value is determined based on information about the activation states of individual neurons in a first group of neurons activated based on the first input data. Next, a second average value is determined based on information about the activation states of individual neurons in a second group of neurons activated based on the second input data. The second input data includes pixels of the occluded object. The grouping of neurons activated for the first input data differs from the grouping of neurons activated for the second input data. The average values ​​of these groups also differ. Based on the differences between these average values, the previously identified feature map can be verified if the second average value for the second digital image containing the occluded object is greater than the first average value for the first digital image containing the unoccluded object.

[0018] The method specifies that the operating parameter is determined based on at least one output channel of at least one intermediate layer of an artificial neural network. The first digital image includes, for example, pixels that present an object, particularly a human, without obscuring it. The second digital image is, for example, identical to the first digital image, wherein at least a portion of the pixels in the second digital image are obscured, meaning the object was presented without obscuring it in the first digital image. For example, the lower portion of a human being presented in a bounding box in the first digital image is obscured. The operating parameter is determined based on the output channel that specifically identifies the obscured portion.

[0019] Preferably, the operating parameter is determined based on a first data record including labeled data points, wherein input data is extracted from a second data record including unlabeled data points, wherein the contribution to improving the operating parameter is determined for these input data, particularly based on the weighted activation state of these input data, wherein data points selected from the second data record are selected based on their contribution to improving the operating parameter, wherein labels are provided for these selected data points, and wherein the operating parameter is determined based on a plurality of data points selected from the second data record. In this way, data points particularly suitable for improving the operating parameter are determined, labeled, and used to further improve the operating parameter.

[0020] Preferably, the distance between a data point and a reference data point is determined, and this distance is used to determine whether the data point is abnormal. For example, if the data point is found to be normal, it is used as input data. Other responses to abnormalities can also be specified.

[0021] Similarly, a device configured to implement the method is also specified. Attached Figure Description

[0022] Other advantageous embodiments are derived from the following description and accompanying drawings. In the drawings:

[0023] Figure 1 A schematic diagram of a test system for artificial neural networks is shown.

[0024] Figure 2 The steps in the first method are shown;

[0025] Figure 3 A schematic diagram of the first digital image is shown;

[0026] Figure 4 A schematic diagram of the second digital image is shown;

[0027] Figure 5 The steps in the second method are shown;

[0028] Figure 6 The steps in the fourth method are shown. Detailed Implementation

[0029] exist Figure 1The diagram illustrates a system having an artificial neural network 100. The artificial neural network 100 includes an input layer 102, at least one intermediate layer 104, and an output layer 106. The at least one intermediate layer 104 includes a plurality of neurons 104-1,...,104-k. Neurons 104-1,...,104-k can form groups of neurons 104-1,...,104-i, which define at least one unit 108 of the intermediate layer 104. The output of the output layer 106 of the artificial neural network 100 is determined according to the weights assigned to neurons 104-1,...,104-k by the artificial neural network 100.

[0030] The system includes an apparatus 110 configured to implement the methods described below.

[0031] The device 110 is configured to access output parameters of at least one intermediate layer 104 and / or at least one unit 108. These output parameters are determined according to weights used to define the output of the output layer 106. In particular, the output parameters of at least one intermediate layer 104 can be used as an activation map. In particular, the output parameters of at least one unit can be used as a feature map.

[0032] The device 110 is configured to determine operating parameters, particularly for performing functional tests, that is, for testing the operating parameters of a technical system using the artificial neural network 100. This technical system is, for example, configured for at least partially autonomous driving, for controlling actuators, and / or for image processing, particularly image processing for such systems. The artificial neural network 100 can be used, for example, in vehicles, medical devices, or static or mobile machines.

[0033] The method for determining operating parameters, described in detail below, specifies that for at least one intermediate layer 104 of the artificial neural network 100, information about the activation states of neurons 104-1,...,104-k of the at least one intermediate layer 104 is determined based on input data. This activation state is determined according to the weights of the artificial neural network 100. In addition to the weights of the artificial neural network 100, at least one weight is provided for at least one neuron among the neurons 104-1,...,104-k, and a weighted activation state of neurons 104-1,...,104-k of the at least one intermediate layer 104 is determined based on this weight and based on the information about the activation state. The operating parameters are determined based on this weighted activation state. This means that for certain input data at the input layer 102, the output of the output layer 106 depends on the activation state and is independent of the weighted activation state.

[0034] This weight can be determined in different ways. Different approaches are shown below.

[0035] In the following text, based on Figure 2 The first method is described below. In this first method, operational parameters are described, particularly for testing artificial neural networks used in systems with at least partial autonomous driving. This testing can also be used for systems used to manipulate actuators and / or for image processing, especially image processing systems for such systems. The testing determines the metric coverage. Based on this coverage, the operational safety of the system is significantly improved.

[0036] In this example, digital images from data records are used as input data for the artificial neural network. These digital images can be captured by a camera that takes digital images of the vehicle's surrounding environment. Figure 3 An exemplary first digital image 300 is shown schematically.

[0037] This method uses a large amount of input data to determine the coverage area. The following description uses the first and second input data as examples. Other input data are processed accordingly.

[0038] In step 202, first input data is determined based on the first digital image 300. The first input data can be characterized as follows.

[0039] In the object recognition method, based on first input data, an object 302 can be identified in a first digital image 300. A bounding box 304 defines a region of pixels in the first digital image 300 that are assigned to the object 302 in the object recognition method. In the first digital image 300, the object 302 itself is defined by the determined pixel values ​​of the pixels representing the object 302.

[0040] The first input data is defined by the pixel values ​​of the pixels in the first digital image 300.

[0041] In step 204, the first input data is provided to the input layer 102. The first input data is provided based on the pixels of the first digital image 300.

[0042] In the artificial neural network 100, the first input data activates a specific neuron.

[0043] In step 206, the neurons activated according to the first input data are determined. This means that for at least one intermediate layer 104 of the artificial neural network 100, information about the first activation state of the neurons in the at least one intermediate layer 104 is determined based on the first input data. The information about the first activation state is defined by neurons in the at least one intermediate layer 104 that are activated and / or deactivated according to the first input data. These neurons can be known from an activation map for the corresponding intermediate layer 104.

[0044] In this example, a first vector v1 with information about the first activation state is determined.

[0045] The feature map defines groups of neurons that are activated based on pixels of the presented object 302 in the first digital image 300. In this example, neurons are identified to verify the feature map, and these neurons are activated based on a fully presented, i.e., unmasked, object 302.

[0046] Next, in step 208, object 302 is masked. Object 302 is masked, for example, by determining a second digital image 400 based on the first digital image 300, in which pixels in the lower portion 406 of the bounding box 304 are masked while other pixels remain unchanged.

[0047] The second digital image 400 includes occluded objects 402. In this context, occlusion means that the images have variations in their color, saturation, and / or grayscale levels.

[0048] Next, in step 210, second input data is determined based on a second digital image 400 showing the occluded object 402. The second input data is provided based on the pixels of the second digital image 400.

[0049] In this example, the second input data is defined by the pixel values ​​of the pixels of the second digital image 400.

[0050] In step 212, the second input data is given to the input layer 102.

[0051] In the artificial neural network 100, the second input data activates a specific neuron.

[0052] In step 214, the neurons activated based on the second input data are determined.

[0053] In this example, neurons are identified to verify the feature map. These neurons are activated based on an incompletely presented, i.e., occluded object 402. This means that information regarding the second activation state of neurons in at least one intermediate layer 104 is determined based on the second input data. This information about the second activation state is defined by neurons in at least one intermediate layer 104 that are activated and / or deactivated based on the second input data. These neurons can be learned from the activation map of the corresponding intermediate layer 104.

[0054] In this example, a second vector v2 is determined, which contains information about the second activation state.

[0055] Next, in step 216, a first average value is determined based on information about the activation states of each neuron in the first group of neurons activated based on the first input data. In this example, the first group of neurons includes neurons activated based on pixels of the first digital image 300 located within the bounding box 304. This first average value is determined, for example, based on the weighted activation states of neurons 104-1,...,104-k in at least one intermediate layer 104. The weighted activation states of neurons 104-1,...,104-k in at least one intermediate layer 104 are determined, for example, by a weighted sum of the values ​​of the elements of the first vector assigned to the first group of neurons.

[0056] In this example, in addition to the weights of the artificial neural network 100, at least one weight is provided for at least one neuron among neurons 104-1,...,104-k, and the weighted activation state is determined based on this weight and based on the activation states of neurons 104-1,...,104-k with respect to at least one intermediate layer 104. It can be specified that the first average value is normalized using the sum of the values ​​of all elements of the first vector. For the activation map of the first digital image, for example, by...

[0057] To determine this first average value,

[0058] in

[0059] M1 first average value,

[0060] v1(p) represents the value of element p in the first vector.

[0061] g1(p) represents the weights of the elements p in the first vector.

[0062] l, ..., i are the indices of the neurons in the activation map that are assigned to the pixels of bounding box 304.

[0063] 1, ..., k are the indices of all neurons in the activated map.

[0064] Elements of vector p.

[0065] In step 218, a second average value is determined based on information about the activation states of each neuron in the second group of neurons activated based on the second input data. The second input data includes pixels of the occluded object. In this example, the second group of neurons includes the same neurons as the first group of neurons. The second average value is determined, for example, by adding the values ​​of the elements of the second vector assigned to the second group of neurons. It can be specified that the second average value is normalized using the sum of the values ​​of all elements of the second vector. For the activation map of the second digital image, for example, by...

[0066] To determine the second average value,

[0067] in

[0068] M2 second average,

[0069] v2(p) represents the value of element p in the second vector.

[0070] g2(p) represents the weights of the elements p in the second vector.

[0071] l, ..., i are the indices of the neurons in the activation map that are assigned to the pixels of bounding box 304.

[0072] 1, ..., k are the indices of all neurons in the activated map.

[0073] Preferably, there is access to information about the activation state of all neurons in all intermediate layers 104 of the artificial neural network 100.

[0074] In step 220, a first average value M1 is compared with a second average value M2. It can be specified that standardized average values ​​are used for the comparison. Herein, a parameter determined based on information about the activation state from the first input data is compared with a second parameter determined based on information about the activation state from the second input data.

[0075] In the following step 222, the operating parameter is determined based on the result of the comparison. This means that the operating parameter is determined based on the weighted activation state. In this example, the operating parameter is determined based on the first vector v1, the weight g1(p) of the element p of the first vector v1, and the second vector v2 and the weight g2(p) of the element p of the second vector v2.

[0076] The grouping of neurons activated by the first input data differs from the grouping of neurons activated by the second input data. The averages or standardized averages of these groups, determined based on these vectors, also differ.

[0077] The weights can be the same for the grouping of these neurons or for all neurons in the intermediate layer 104. These weights are pre-given, for example, based on knowledge of the structure of the artificial neural network 100. For example, a first weight is pre-given for the neurons in the first hidden layer of the artificial neural network 100. For example, a second weight is pre-given for the neurons in the second hidden layer of the artificial neural network 100. For example, this weight is a fraction of the first weight.

[0078] This weight can be determined based on information about the first activation state and / or information about the second activation state.

[0079] In the first aspect, the feature map of the intermediate layer 104 is validated using this running parameter. In this regard, the feature map is validated based on the difference between the following average values, which are determined for the neurons of the intermediate layer 104 according to these vectors. In this example, it is checked whether the second average value for the second digital image 400 with the occluded object 402 is greater than the first average value for the first digital image 300 with the unoccluded object 302. If so, it can be specified that a test result indicating that the feature map is validated is output. Otherwise, it can be specified that a test result indicating that the feature map is not validated is output.

[0080] In a second aspect, the operating parameter is determined based on at least one output channel of at least one intermediate layer 104 of the artificial neural network. The output channel, for example, comprises one or more groups of neurons in the intermediate layer 104 of the artificial neural network, the activation state of which differs in the case of the first input data from the activation state of which differs in the case of the second input data.

[0081] In this example, the first digital image 300 includes pixels that present the human as an unmasked first object 302. The second digital image 400 is, for example, identical to the first digital image 300, wherein at least a portion of the pixels in the second digital image 400 that present the first object 302 unmasked in the first digital image 300 are masked. In this example, the lower portion of the human presented in the bounding box 304 of the first digital image 300 is masked. This comparison determines the operating parameter based on the output channel that specifically identifies the masked portion. In this example, this is the lower portion of the human. If so, it can be specified that a test result indicating that the output channel is identified can be output. Otherwise, it can be specified that a test result indicating that the output channel is not identified can be output.

[0082] In a third aspect, the operating parameter represents information about a path through which first input data is mapped to a target via multiple intermediate layers. In this regard, information about a first path through which the first input data is mapped to the target is determined based on information about a first activation state from the multiple intermediate layers 104. Similarly, information about a second path through which second input data is mapped to the target is determined based on information about a second activation state from the multiple intermediate layers 104. In this respect, the operating parameter is determined based on information about the first path and information about the second path. These paths are incorporated to consider which activations are important for a particular target.

[0083] Following step 222, step 224 can be set. In step 224, it can be optionally checked whether step 202 should be performed or the method should be terminated. Preferably, the method specifies that the operating parameter is determined based on first and second input data from the data record, wherein the operating parameter is used to determine or to determine other operating parameters related to other input data from the data record, or to determine or not determine other operating parameters related to other input data from the data record. This is used to check whether the quality objective has been achieved. If the quality objective has not been achieved, the method continues using other input data.

[0084] The quality objective can be a pre-defined corridor or threshold for the behavior of the technical system under test or its components.

[0085] For example, coverage can be determined by providing a large amount of input data, particularly from data records. For this large amount of input data, multiple weighted activation states are determined. In this case, the coverage is determined based on multiple operating parameters, which are determined for the multiple weighted activation states. For example, the coverage can be used to check whether a identified neuron is marked as activated at least once.

[0086] For example, if the predetermined coverage of these neurons is indicated, then the quality target is achieved.

[0087] In the following text, the second method is based on Figure 5 To describe it.

[0088] The second method specifies that the operating parameter is determined for the first data record D1 in step 502. In this example, the first data record D1 is a marked data record, that is, a data record with marked data points.

[0089] Next, proceed to step 504.

[0090] Step 504 specifies that input data is extracted from the second data record D2 according to the operating parameters. In this example, the second data record D2 is an unlabeled data record, that is, a data record with unlabeled data points.

[0091] Next, proceed to step 506.

[0092] In step 506, the contribution of these input data to improving the operating parameter is determined. The contribution to improving the operating parameter is determined, for example, based on the weighted activation state of these input data. Selected data points are chosen based on their contribution to improving the operating parameter. Labels are provided for these selected data points. In this example, data points from a plurality of data points that have a large contribution to improving the operating parameter are selected from the second data record D2.

[0093] Next, proceed to step 508.

[0094] In step 508, the operating parameter is determined based on a plurality of data points selected from the second data record D2.

[0095] Next, in step 510, it is checked whether the quality objective of active learning has been achieved.

[0096] If the quality objective is achieved, the method ends. Otherwise, proceed to step 502. The quality objective is, for example, coverage.

[0097] A rule can be established: determine the distance between a data point and a reference data point. This distance is used to determine if the data point is an anomaly. For example, if the distance is greater than a threshold, an anomaly is considered present. In this case, the data point is only used as input data if it is determined to be normal.

[0098] In this active learning scheme, unlabeled data suitable for improving coverage is selected from the second data record D2 and then used to improve coverage.

[0099] It can be specified that a corresponding vector is determined for the output of unit 108 of artificial neural network 100, and this vector defines information about the activation state of neurons 104-1,...,104-i of unit 108. In this example, if the output parameter of neuron 108 exceeds a threshold or is within a pre-defined range, the activation state indicates that the neuron is activated.

[0100] Preferably, multiple vectors are determined for multiple units 108 of the artificial neural network 100. In the intermediate layer 104 of the artificial neural network, multiple neurons constitute units 108. The output of a unit 108 is represented by a vector. The outputs of multiple units 108 are represented by multiple vectors and mapped to an element of the state vector. Multiple elements of the state vector can be determined based on the different outputs of the units 108 caused by different multiple vectors. In this case, the state of the elements of the state vector is defined according to multiple vectors. Here, an element of the state vector indicates that the element of the state vector is activated if at least one element of one of the following vectors indicates activation, and the vector is analyzed to determine that element.

[0101] In the second method, the operating parameter is defined by the frequency of activation of elements of the state vector occurring with respect to at least the first and second input data. To determine the frequency of activation of an element of the state vector, the number of times that element is activated is summed, or the value of a function depending on the number of times that activation occurs is determined. Standardization may also be specified.

[0102] Therefore, activation is considered based on the architecture of the artificial neural network 100.

[0103] In the following text, based on Figure 6 To describe the third method.

[0104] The third method specifies that, in step 602, a first operating parameter is determined. To determine the first operating parameter, for at least one intermediate layer 104 of the artificial neural network 100, information about the first activation state of the neurons in the at least one intermediate layer 104 is determined based on the first input data.

[0105] Information about the first activation state is defined by at least one intermediate layer 104 of neurons that are activated and / or deactivated based on the first input data.

[0106] In this example, a first vector containing information about the first activation state is determined.

[0107] For example, the first input data defines reference data points. The first vector is determined based on the first input data. In this example, a reference distribution of information about the first activation state is determined using multiple first vectors v1 based on multiple reference data points.

[0108] Next, proceed to step 604.

[0109] In step 604, a second operating parameter is determined. This second operating parameter is determined in response to the second input data. To determine this second operating parameter, information regarding the second activation state of neurons in at least one intermediate layer 104 is determined based on the second input data.

[0110] Information about the second activation state is defined by at least one intermediate layer 104 of neurons that are activated and / or deactivated based on the second input data.

[0111] In this example, a second vector containing information about the second activation state is determined.

[0112] For example, the second input data defines the data points to be evaluated. The second vector is determined based on the second input data. In this example, multiple second vectors v2 are used to determine the distribution of information about the second activation state based on multiple data points.

[0113] Next, in step 606, the distance between the first vector and the second vector is determined. In this example, multiple distances between the first vector and the second vector are determined based on multiple data points and multiple reference data points.

[0114] Next, in step 608, the at least one weight and / or the operating parameter is determined based on at least one distance. In this example, the at least one weight and / or the operating parameter is determined based on the distance between the distribution and the reference distribution. It can be specified that the distance between the distribution and the reference distribution is determined instead of the distance between the first vector and the second vector.

[0115] In one respect, the distance between the distribution and the reference distribution is limited by the probability that the distribution lies within the reference distribution.

[0116] On the other hand, at least one prototype point is determined based on the reference distribution. In this example, the prototype point is a reference point used for distance calculation, which is determined based on the reference distribution, for example, through the k-nearest neighbor algorithm or centroid calculation. In this respect, at least one data point is determined based on the distribution. In this respect, the distance between the distribution and the reference distribution is defined by the distance between the data point and the prototype point.

[0117] The method then ends.

Claims

1. A method for determining an operating variable for testing a technical system for at least partially autonomous driving, actuator control and / or image processing, characterized in that For at least one intermediate layer (104) of an artificial neural network (100), information about an activation state of a neuron (104-1,..., 104-k) of the at least one intermediate layer (104) is determined from input data, wherein a digital image from a data record is used as input data of the artificial neural network (100), wherein at least one weight is provided for at least one of the neurons (104-1,..., 104-k) in addition to the weights of the artificial neural network (100), wherein a weighted activation state of the neuron (104-1,..., 104-k) of the at least one intermediate layer (104) is determined (216) from the at least one weight and from the information about the activation state, and wherein the operating variable is determined (222) from the weighted activation state, wherein, for carrying out the functional test, for at least one intermediate layer (104) of an artificial neural network (100), information about a first activation state of a neuron (104-1,..., 104-k) of the at least one intermediate layer (104) is determined (206) from first input data, wherein information about a second activation state of the neuron (104-1,..., 104-k) of the at least one intermediate layer (104) is determined (214) from second input data, and wherein the at least one weight is determined from the information about the first activation state and from the information about the second activation state.

2. The method of claim 1, wherein, The image processing is an image processing of such a technical system.

3. The method of claim 1, wherein, The information about the first activation state is defined by neurons (104-1,..., 104-k) of the at least one intermediate layer (104) which are activated and / or deactivated from the first input data; and / or the information about the second activation state is defined by neurons (104-1,..., 104-k) of the at least one intermediate layer (104) which are activated and / or deactivated from the second input data.

4. The method according to any one of claims 1 to 3, characterized in that, A first vector with information about the first activation state is determined (206), wherein a second vector with information about the second activation state is determined (214), and wherein the operating variable and / or the at least one weight is determined (222) from the first vector and the second vector.

5. The method according to any one of claims 1 to 3, characterized in that, A distance between the first vector and the second vector is determined.

6. The method according to any one of claims 1 to 3, characterized in that, The first input data define a reference data point, wherein a reference distribution of the information about the first activation state is determined from a large number of first input data, wherein the second input data define a data point to be evaluated, wherein a distribution of the information about the second activation state is determined from a large number of second input data, and wherein the operating variable and / or the at least one weight is determined from a distance of the distribution from the reference distribution. The image processing is an image processing of such a technical system.

7. The method of claim 6, wherein, The distance of the distribution from the reference distribution is defined in dependence on a probability that the distribution lies within the reference distribution.

8. The method of claim 6, wherein, At least one prototype point is determined in dependence on the reference distribution, wherein at least one data point is determined in dependence on the distribution, and wherein the distance of the distribution from the reference distribution is defined in dependence on a distance of the data point from the prototype point.

9. The method according to any one of claims 1 to 3, characterized in that, Information about a first path through which a first input data is mapped to a target is determined in dependence on information about a first activation state from a plurality of intermediate layers (104), wherein information about a second path through which a second input data is mapped to the target is determined in dependence on information about a second activation state from a plurality of intermediate layers (104), and wherein the operating parameter and / or the at least one weight is determined in dependence on the information about the first path and in dependence on the information about the second path.

10. The method according to any one of claims 1 to 3, characterized in that, A multitude of input data is provided from a data record, wherein the operating parameter is determined in dependence on a plurality of weighted activation states, which are determined in dependence on the multitude of input data.

11. The method according to any one of claims 1 to 3, characterized in that, For an output of a cell (108) of the artificial neural network (100), a vector is determined, which defines information about the weighted activation state of a neuron (104-1,..., 104i) of the cell (108), wherein the activation state indicates that the neuron is activated if an output parameter of the neuron (104-1,..., 104i) of the cell exceeds a threshold value or lies within a predefined range, wherein a multitude of vectors is determined for a multitude of cells (110) of the artificial neural network (100), wherein a plurality of vectors of the multitude of vectors defines a state of an element of a state vector, wherein the element of the state vector indicates that the element is activated if the information about the activation state of one of the plurality of vectors indicates that at least one neuron (104-1,..., 104-k) is activated, and wherein the operating parameter is defined by a frequency with which the element of the state vector is activated at least for a first input data and a second input data.

12. The method of claim 11, wherein, For determining the frequency with which the element of the state vector is activated, a number of occurrences of an activation of the element is summed or a value of a function depending on the number of occurrences of the activation is determined (510).

13. The method of claim 12, wherein, The operating parameter is determined (222) in dependence on at least one output channel of at least one intermediate layer (104) of the artificial neural network.

14. The method of claim 1 or 2, wherein, The operational quantity is determined (502) from a first data record, the first data record comprising labeled data points, wherein input data is extracted (504) from a second data record according to the operational quantity, wherein the second data record comprises unlabeled data points, wherein for the input data a contribution to improving the operational quantity is determined (506) according to a weighted activation state of the input data, wherein data points selected from the second data record are selected (506) according to their contribution to improving the operational quantity, wherein a label is provided for the selected data points, and wherein the operational quantity is determined (508) from the plurality of data points selected from the second data record.

15. The method of any one of claims 1 to 3, wherein, A distance of the data point to a reference data point is determined, wherein it is ascertained from the distance whether the data point is abnormal.

16. A device (110) for determining an operating variable for testing a technical system for at least partially autonomous driving, actuator control and / or image processing, characterized in that The device (110) is configured to implement the method according to any one of claims 1 to 15.

17. The device (110) according to claim 16, characterized by The image processing is an image processing for such a technical system.

18. A computer program product, characterised in that, The computer program product comprises a computer readable storage medium having stored thereon a computer program, wherein the computer program comprises computer readable instructions which, when executed by a computer, implement the method according to any one of claims 1 to 15.

Images