Method and system for validating trained artificial neural networks (KNN) based on test datasets

Abstract

Methods and systems for validating a trained artificial neural network (KNN) based on a test data set are described.

Description

Technical Field

[0001] The present invention relates to a method and system for validating a trained artificial neural network (KNN) based on a test data set. Background Art

[0002] Trained artificial neural networks (KNNs) usually have a high degree of redundancy in their weights. This means that many weights are not absolutely necessary to make the neural network work effectively, especially in the case of inference. Since the number of weights strongly affects the complexity of the network architecture on which the artificial neural network is based, and leads to the need for more and more computing power to execute the artificial neural network as the number of weights increases, but also requires more and more computing power to train it, there is a need to keep the number of weights per layer of the network as low as possible while still ensuring the efficiency and powerful functions of KNN.

[0003] In order to be able to evaluate the quality and / or performance of KNN, a validation method is desirable.

[0004] Subject matter of the invention

[0005] The object of the present invention is to provide a method and a system for validating a trained artificial neural network (KNN) based on a test data set.

[0006] This object is achieved by a computer-implemented method for validating a trained artificial neural network (KNN) based on a test data set according to the features of patent claim 1. This object is achieved by a system for validating a trained artificial neural network (KNN) based on a test data set according to the features of patent claim 10. Summary of the invention

[0007] A method for validating a trained artificial neural network (KNN) based on a test data set is described herein; the method comprises the following steps:

[0008] - providing a trained KNN having a network architecture, which represents a particularly non-linear chain function, by which a d-dimensional input space can be mapped into an N-dimensional output space, and which also comprises a plurality of weights;

[0009] -Provide test datasets;

[0010] - The KNN-based network architecture divides the d-dimensional input space into a plurality of cells, wherein the individual cells can be separated from one another by at least one hyperplane and / or superset with a specific weight;

[0011] - check: whether there is at least one data point of the test dataset in each of these cells, so as to validate the KNN in this way; and

[0012] - if no data point exists in at least one of the cells, generating (S5) at least one new data point based on the cell parameters of said at least one of the cells by means of a simulation model, so as to complete the test data set in this way

[0013] The invention decomposes or divides the input space into a plurality of cells. In particular, the predicted class is constant in each cell. The invention checks whether a data point exists in each cell. If this is the case, the algorithm or verification method preferably ends. If this is not the case, the simulation model is given the parameters of these cells without data points. The simulation model then preferably generates at least one new data point in the one or more relevant cells. If necessary, the KNN is iteratively retrained using the newly generated data or data points. Preferably, new test data are generated, based on which the retrained KNN can be evaluated or verified again.

[0014] It should be understood that the steps according to the present invention and other optional steps do not necessarily have to be performed in the order shown, but may also be performed in other orders. In addition, further intermediate steps may be provided. Each step may also include one or more sub-steps, without departing from the scope of the method according to the present invention.

[0015] In other words, the d-dimensional input space of KNN is decomposed or divided into a finite number of cells ("faces" in English) by this method. If the function of KNN is to solve a classification task, then in or within each of these cells, the output of KNN is preferably constant. If the function of KNN is to solve a regression task, then in or within each of these cells, the output of KNN is preferably linear. Pruning of KNN is performed by this method.

[0016] Hyperplane is a mathematical term defined in multidimensional space. In two-dimensional space, a hyperplane is a straight line, and in three-dimensional space, a hyperplane is a plane. Generally speaking, a hyperplane in n-dimensional space is a (n-1)-dimensional surface that divides the space into two parts. In KNN, a hyperplane is used to define a category or decision boundary (Entscheidungsgrenzen) by dividing the space into multiple regions corresponding to different categories or states.

[0017] Hyper-Menge is an extension of the concept of hyperplane. A superset is not a single hyperplane, but a collection of hyperplanes in space. These hyperplanes as a whole can be used to define complex spatial structures. Supersets are used in KNN to model more complex decision boundaries that cannot be simply described by a single hyperplane.

[0018] The input space (also called feature space) refers to the space in which the input data of the neural network exists. Each dimension of the input space corresponds to a specific feature or a specific attribute of the input data. For example, the pixel values ​​of an image in an image classification network can form the input space.

[0019] The output space refers to the space of values ​​that the neural network outputs. This space depends on the type of network. In a classification network with multiple classes, the output space can contain vectors representing probabilities for each class. In a recurrent neural network, the output space can represent numerical values.

[0020] In a preferred embodiment, KNN has a d-dimensional number of input variables. For example, such an input variable can have a sensor signal. The input variable can be preprocessed, such as normalization and / or transformation from a time range to a frequency range or from a frequency range to a time range and / or filtering and / or smoothing, etc. The expression "d-dimensional" means: preferably d different signals can be used as input variables. Therefore, an input space with a dimension of d is opened. The network architecture also has multiple bias terms (Biasterm). In KNN, the bias term is preferably a constant added to the weighted input of each neuron before applying a preferably nonlinear activation function. It can be interpreted as a shift (Verschiebung) that affects the activation of neurons. The bias here helps control the activation range and the overall ability of the network to adapt to data. Even if all weighted inputs are equal to zero, the bias term allows neurons to be activated. This enables the network to react more flexibly to different patterns and relationships in the data. In a multi-layer neural network, each neuron usually has its own bias value. Dividing the d-dimensional input space into a plurality of cells based on the network architecture comprises at least the following steps: determining a hyperplane and / or a superset based on the input variables, at least a subset of the plurality of weights, and a subset of the plurality of bias terms. How the input space and subsequent space of the network layer are specifically divided is described in detail in the accompanying drawings, and explicit reference is made to it here.

[0021] In a preferred embodiment, if at least one data point of the test data set is present in each cell, a validation result of the KNN is provided, based on which the performance and / or quality and / or quality of the KNN can be determined and / or evaluated. Based on the validation result, a decision can preferably be made as to whether the KNN can be used for reasoning, for example by comparing the validation result with a predetermined threshold. This improves the efficiency of the KNN. and fault reliability. In a preferred embodiment, the KNN is retrained based on the completed test data set. Through retraining, the performance of the KNN is continuously improved. This will produce a reduced error rate in the reasoning of the KNN.

[0022] In a preferred embodiment, an extended and / or new test data set is provided for the retrained KNN, and the retrained KNN is revalidated based on the test data set. In this way, the efficiency or performance of the KNN can be continuously and / or iteratively improved.

[0023] In a preferred embodiment, retraining is performed iteratively.

[0024] In a preferred embodiment, for a d-dimensional input space, the following applies: d=10, preferably d<10, particularly preferably d<6, where d is an element of the set of positive natural numbers. Therefore, KNN is preferably a network of small dimensions. For KNNs of higher dimensions, due to the large number of input variables, the division can only be performed with much higher computational effort.

[0025] In a preferred embodiment, KNN is part of a complex machine learning model. Thus, a part or component of the complex model can be optimized. This leads to an improvement in the overall model.

[0026] In a preferred embodiment, the simulation model is designed to generate at least one data point based on cell parameters, in particular based on hyperplane information and / or superset information, in particular by data augmentation. The simulation model can be, for example, a data augmentation model.

[0027] In a second aspect, a system for validating a trained artificial neural network (KNN) based on a test data set is described. The system has a providing device and an evaluating and / or computing device, wherein the providing device is designed to perform the following steps: providing a trained KNN with a network architecture, the network architecture represents a chain function, in particular a nonlinear one, by which a d-dimensional input space can be mapped to an N-dimensional output space, the network architecture also comprising a plurality of weights; providing a test data set, wherein the evaluating and / or computing device is designed to perform the following steps: dividing the d-dimensional input space into a plurality of cells based on the network architecture of the KNN, wherein the individual cells can be separated from each other by at least one hyperplane and / or superset, respectively, which is weight-specific; checking: whether at least one data point of the test data set exists in each of these cells, so as to validate the KNN in this way; and if a data point does not exist in at least one of the cells, generating at least one new data point based on the cell parameters of at least one of the cells by means of a simulation model, so as to complete the test data set in this way.

[0028] The embodiments mentioned for the method apply in the same or similar manner to the system.

[0029] A control and / or computing device is also claimed, which is designed to perform regression tasks and / or classification tasks by executing a trained, validated and / or optimized KNN provided according to the method. Purely as an example, the control and / or computing device can be included in an autonomous vehicle and / or a robotic system and / or an industrial machine. Such a control and / or computing device can be used, for example, as an embedded device in a facility or system. In general, the method is particularly suitable for artificial neural networks of small dimensions. For example, the currently validated and / or optimized KNN can be used as a virtual sensor with inputs of up to ten, preferably up to five dimensions. The KNN provided here can also be used for feature detection in larger KNNs, for example for autonomous edge detection. Other examples of virtual sensors in vehicles are: virtual temperature sensors (for example, which are located on the stator / rotor of the electric motor or at the injector magnet), injection system quality estimators, coking models and / or exhaust gas concentration virtual sensors. In principle, the method or the KNN provided by it can be applied to all virtual sensors that cannot be assembled, have high cost pressures, and have complex measurement structures for series products.

[0030] In addition, the currently validated and / or optimized KNN provided can be used to determine correction factors for calculating the reading of the current consumption of the engine in the vehicle. The current KNN here provides an alternative to the known methods that require "tuning (Einfahren) injectors" and may not always work. This behavior can be parameterized in a characteristic curve family (Kennfeld). In addition, the increased requirements of the EU require the use of alternative methods. Such an alternative is provided based on the compactness or low complexity of the currently validated and / or optimized KNN.

[0031] Furthermore, the currently validated and / or optimized and available KNN can be used to detect change points of discrete gyroscope signals, for example for fall detection. It is advantageous here that the input vectors are of small dimension and thus the application of the optimization method can be optimally performed to provide an optimized KNN.

[0032] In addition, the KNN presented here can be used as an edge filter in image processing. This can be explained using an excavator bucket as an example, wherein the KNN can be used as part of the excavator bucket control loop. The transfer function of the excavator joystick to the bucket cylinder position can be modeled here using such a KNN optimized here. Preferably, small-dimensional networks with simple network architectures, for example of size 3-15-15-1 and / or 2-15-7-1, are considered here. In the case of such a small KNN, the decomposition can be determined, for example, in less than 5 seconds.

[0033] In particular, due to its low dimensionality, the current KNN can be used as a surrogate function, where the KNN represents the solution of a partial differential equation, which cannot be independently calculated in real time.

[0034] According to the present invention, a computer program with a program code is also claimed so that when the computer program is executed on a computer, at least part of the method according to the present invention is performed in one of its embodiments. In other words, the computer program (product) comprises instructions which, when the program is executed by a computer, cause the computer to perform the method / method steps according to one of the embodiments of the present invention.

[0035] According to the invention, a computer-readable data carrier is also proposed with a program code of a computer program in order to perform at least part of the method according to the invention in one of its embodiments when the computer program is executed on a computer. In other words, the invention relates to a computer-readable (storage) medium comprising instructions which, when executed by a computer, cause the computer to perform a method / method steps according to one of its embodiments.

[0036] The described embodiments and refinements can be combined with one another as desired.

[0037] Further possible embodiments, further developments and implementations of the invention also include combinations of features of the invention which are described above or below with reference to the exemplary embodiments, which are not explicitly mentioned. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The accompanying drawings are intended to provide a further understanding of the embodiments of the present invention. They illustrate the embodiments and, together with the description, are used to explain the principles and concepts of the present invention.

[0039] Other embodiments and many of the mentioned advantages are derived with reference to the drawings. The elements shown in the drawings are not necessarily shown to scale with respect to each other.

[0040] in:

[0041] Figure 1 An exemplary hyperplane for partitioning the input space is shown;

[0042] Figure 2 shows a Figure 1 Representation of;

[0043] Figure 3 An exemplary representation of partitioning a two-dimensional open set is shown;

[0044] Figure 4 An exemplary representation of a one-dimensional open set is shown;

[0045] Figure 5 An exemplary representation of three 0-dimensional sets is shown;

[0046] Figure 6 An exemplary representation of the partitioning of the 10-dimensional input space of the first layer of KNN is shown;

[0047] Figure 7 An exemplary representation of the partitioning of the 10-dimensional input space of the second layer of KNN is shown;

[0048] Figure 8 An exemplary flow chart illustrating an embodiment of the present method is shown;

[0049] Fig. 9 An exemplary representation of the input space of the first layer of KNN is shown;

[0050] Fig.10 An exemplary representation of the input space of the first layer of KNN is shown;

[0051] Fig.11 An exemplary representation of the input space of a second or further layer of a KNN is shown;

[0052] Fig.12 An exemplary representation of the input space of a second or further layer of a KNN is shown.

[0053] In the drawings, unless otherwise indicated, the same reference numerals indicate the same or functionally equivalent elements, components or assemblies. DETAILED DESCRIPTION

[0054] refer to Figures 1 to 7 The partitioning of the input space is described according to its mathematical principles. Figures 8 to 12 Explain the present invention.

[0055] The partitioning according to the present method preferably considers a multilayer neural network KNN with ReLu nonlinearity as activation function. That is, the KNN defines the function F:R m1 →R mk+1 .

[0056] Applicable:

[0057] F(x)=A k φ(A k-1 φ(A k-2 ...φ(A 2 (φ(A 1 x+b 1 ))+b 2 )+…)+b k

[0058] The weight matrix is ​​A 1 ,...,A k , the bias term is b 1 ,...,b k In this case, A 1 is a matrix, i.e. a linear mapping between real vector spaces: A 1 :R m1 →R m2 , A 2 :R m2 →R m3 ,...,A k :R mk →R mk+1 is also a vector b in a real vector space 1 ∈R m2 ,...,b k ∈R mk+1 .

[0059] In general, F is preferably an m1-dimensional Euclidean space R m1 To mk+1-dimensional Euclidean space R mk+1 Link map (verketteteAbbildung) F:R m1 →Rmk+1 .

[0060] To reduce the number of symbols, set d = m 1 and n = m k+1 , i.e. the network F:R examined here d →R N is a d-dimensional Euclidean space R d In N-dimensional Euclidean space R N The functions in .

[0061] If regression is modeled, the description of the function F(x) is preferably completed. The KNN then preferably returns the value F(x) as an output value or a predicted value.

[0062] In classifying the input signal x into m k+1 In the case of the class, the output F(x) is preferably followed by a softmax function. Thus, F(x)=(f 1 (x),f 2 (x),...,f mk+1 The components of (z)) are preferably positive and normalized to 1, i.e. for all input variables x it applies that for all z and the sum f 1 (z)+...+f mk+1 (x) = 1, f 1 (x),...,f mk+1 (x)≥0.

[0063] The classification is preferably performed using the Argmax function, which returns a vector g 1 (x),...,g mk+1 The index of (x) at which g(x) is maximum.

[0064] After explaining the basis, the partitioning of the input space is described in more detail below. The layers of the KNN preferably divide the input space into a plurality of cells. In the case of a classification network, a distinction should preferably be made between the first layer (input layer), the middle layers (also subsequent layers or layers after the input layer) and the last layer (also output layer).

[0065] In the network, the weight matrix A 1 and the bias term b 1 In the first layer consisting of the pair and the ReLu nonlinear φ (activation function), the input vector x is multiplied by the weight matrix A 1 and the bias term b 1 Add them together and we get vector A 1 x+b 1is delivered to the component ReLuφ(A 1 x+b 1 ). At the weight level, this means:

[0066]

[0067] For ease of reading, we set d = m here 1 and m=m 2 .

[0068] The following also includes A 1 The column vector and vector h 1 ,...,h m .

[0069]

[0070] These vectors and the bias term each preferably define an open half-space.

[0071]

[0072] Furthermore, a hyperplane is preferably introduced for the purpose of partitioning.

[0073]

[0074] This system of half-spaces and hyperplanes preferably takes the input space R n This relationship is exemplified in Figure 1 and Figure 2 Shown in. Figure 1 and Figure 2 Here, three hyperplanes H generated according to the above equations are shown. 0 , H 1 , H 2。 exist Figure 2 In the figure, each hyperplane H is shown 0 , H 1 , H 2 and its open half-space. In addition, Figure 2 Three exemplary points (vertices) and their respective codes are shown in FIG.

[0075] Figure 3 Shown by the hyperplane H 0 , H 1 , H 2 Here, in Figure 4 A total of 7 cells are shown in , which define two-dimensional open sets respectively.

[0076] exist Figure 4In , a one-dimensional cell defining a one-dimensional relatively open set is shown.

[0077] exist Figure 5 , the 0th cell defining a zero-dimensional set, ie, a vertex or point, is shown.

[0078] In other words, for every element x∈R m1 All codes are assigned like this: code(x)∈{-1,0,1} m , so that in 〈h 1 ,x〉+b 1 1 <0, insert -1 at the kth position, 1 ,x〉+b 1 1 =0, insert 0 at the kth position, and 1 ,x〉+b 1 1 >0, +1 is inserted at the kth position. In this manner and method, each element of the input space is preferably uniquely defined by a code.

[0079] The first layer's hyperplane H 1 0 ,H 2 0 ,...,H m 0 Generates an n-dimensional cell (or n-plane). In this case, the n-th cell is the connected component of the complement of the union of hyperplanes. Figure 3 , which means that the three lines represent three hyperplanes, where the union of these lines defines a closed set (preferably as a set of points). The complement of this closed set is preferably an open set that divides the input space into a plurality of connected components (blue planes F 1 2 ,...,F 7 2 ).

[0080] The (relatively open connected component) of the edge of an n-face is the (n-1)th cell (see Figure 3 and Figure 4 ). Triangle F 4 2 Preferably there is an edge consisting of three edges, which are preferably defined by a bounding hyperplane or superset, respectively. These edges are preferably three first cells.

[0081] The kth cell is recursively defined as the edges of the k+1th cell until the 0th cell is generated at the end, which are preferably points of the input space.

[0082] Relationship with neural networks:

[0083] In each n-th face, the output of the first layer of the neural ReLu KNN can be viewed as a linear mapping.

[0084] From layer k to layer (k+1)

[0085] Each further layer of the ReLu KNN preferably further divides the nth cell generated by the previous layer. In this case, each nth cell should preferably be considered separately.

[0086] Output layer in regression:

[0087] In the case of regression, the partitioning process ends. In this case, the last or output layer is preferably a linear mapping onto the output of the penultimate layer. Preferably, there is no further partitioning of the input space.

[0088] Output layer in classification:

[0089] The previous step yields the partitioning P of the input space into the nth cell (in the input layer), the (n-1)th cell (in a further layer), and so on. k These n-th cells are also divided in the last layer, the output layer. In this case, preferably all n-th cells are traversed, wherein the following steps can be performed:

[0090] 1. Determine the matrix-vector pair L, l that describes the linear relationship in the nth cell. For example, this is done in the following way:

[0091] a. For example, determine the internal points of the nth cell by forming the common center point of the vertices of the nth cell

[0092] b. For each layer, preferably an activated feature is determined. In addition, the linear mapping and the bias term are preferably determined interactively in the following manner:

[0093] i. Formation

[0094] ii. Determine the vector Negative rows.

[0095] iii. Form new matrix-vector pairs in Corresponding to the matrix A 1 , preferably with the only difference being that the rows in the vector is negative, and These lines are set to zero. In this way and method, for a selected point x it applies:

[0096]

[0097] iv. Now this is performed similarly for further layers of KNN:

[0098] v. preferably formed The following rows are determined, in which the vector is negative.

[0099] vi. Form new matrix-vector pairs in Corresponding to the matrix A 2 , but in the vector These rows are negative and These rows are set to zero.

[0100] vii. L and l can be introduced in such a way that they are applicable in the cell under examination:

[0101] Lx+l=A k φ(A k-1 φ(A k-2 ...φ(A 2 (φ(A 1 x+b 1 ))+b 2 )+…)+b k .

[0102] 2. Now, determine the new matrix-vector pair (H, d) from the matrix-vector pair L, l in the following way:

[0103] a. Form a zero-initialized matrix H of size (0.5*number of classes*(number of classes-1))×d and a zero-initialized vector d of length 0.5*number of classes*(number of classes-1).

[0104] b. Traverse all possible combinations of two elements of the output class.

[0105] i. For two categories 1 and 2, there is only one possible combination (1,2)

[0106] ii. For three categories 1, 2, and 3, there are three possible combinations: (1,2), (1,3), and (2,3)

[0107] iii. For the four categories 1, 2, 3, 4, there are six possible combinations (1,2), (1,3), (1,4), (2,3), (2,4), (3,4), and so on.

[0108] c. For each combination i, j, form the following difference between the i-th column vector and the j-th column vector of L, and similarly, form the difference between the i-th entry of the bias (Eintrag) and the j-th entry of the bias.

[0109] 3. Now, the cell is further divided by this matrix and the bias term b.

[0110] 4. The construction makes the class of the network determined by the Argmax function constant in each cell. The matrix H and the bias term identify the following points in the input space at which F(x) has two components of the same size, which are the class boundaries of the neural network.

[0111] Thus, the above describes a partitioning or decomposition of the input space into a plurality of cells or faces, wherein KNN is constant in each cell (ie, each nth face) of the partitioning or decomposition.

[0112] Figure 6 and Figure 7 A further example of the above partitioning is shown in . An exemplary partitioning of the input space is shown using 2->10->5->4 classification KNN. Figure 6 The partitioning of the input space by the first layer is shown. The ten hyperplanes generated by the weight matrix and bias terms are shown. Figure 7 Shown is the partitioning of the input space after the second layer. Figure 7 Each second cell divided in is further divided by five hyperplanes of the second layer. In principle, the second layer may also have no (dividing) effect in the cell.

[0113] Figure 8 A flow chart of an embodiment of the method is shown. In any embodiment, the method may be at least partially performed by a system 1, which may include a plurality of components not shown in detail, such as one or more providing devices and / or at least one evaluation and calculation device. It should be understood that the providing device may be designed together with the evaluation and calculation device or may be different from it. In addition, the system may include a storage device and / or an output device and / or a display device and / or an input device.

[0114] According to the present invention, the computer-implemented method comprises at least the following steps:

[0115] - providing (S1) a trained KNN having a network architecture, the network architecture representing a chained function, in particular a non-linear one, by which a d-dimensional input space can be mapped to an N-dimensional output space, and the network architecture also comprising a plurality of weights;

[0116] - Provide (S2) a test dataset;

[0117] - The KNN-based network architecture divides (S3) the d-dimensional input space into a plurality of cells, wherein the cells are separable from each other by at least one hyperplane and / or superset with a specific weight;

[0118] - checking (S4): whether there is at least one data point of the test data set in each of these cells, so as to verify the KNN in this way; and

[0119] If no data point exists in at least one of the cells, at least one new data point is generated (S5) by means of the simulation model based on the cell parameters of the at least one of the cells, in order to complete the test data set in this way.

[0120] Figures 9 to 12 The method is shown, for example, for a partitioning of the input space by a KNN with two hidden layers and for four output classes (network architecture: 2->10->10-4).

[0121] Fig. 9 KNN with a representation of a partitioned input space is shown. Figures 10 to 12 A modified KNN is shown compared to KNN. The penultimate layer of KNN preferably decomposes the input space into 2 faces, 1 face, etc. In order to be able to distinguish the test data, the last layer is preferably also considered. The circles preferably represent the test data of KNN. The constant classes of the lookup table are seen. It can also be seen how these classes are matched to the data.

Claims

1. Methods for validating trained artificial neural networks (KNN) based on a test dataset, The method comprises the following steps: - providing (S1) a trained KNN having a network architecture, the network architecture representing a chained function, in particular a non-linear one, by which a d-dimensional input space can be mapped to an N-dimensional output space, and the network architecture further comprising a plurality of weights; - Provide (S2) a test dataset; - based on the network architecture of the KNN, the d-dimensional input space is divided (S3) into a plurality of cells, wherein the cells are separable from each other by at least one hyperplane and / or superset with a specific weight; - checking (S4): whether there is at least one data point of the test data set in each cell thereof, so as to verify the KNN in this way; and - If no data point exists in at least one of the cells, generating (S5) at least one new data point based on the cell parameters of the at least one of the cells by means of a simulation model, in order to complete the test data set in this way.

2. The method according to claim 1, wherein the KNN has a d-dimensional number of input variables, wherein the network architecture further includes a plurality of bias terms, and wherein dividing the d-dimensional input space into a plurality of cells based on the network architecture comprises at least the following steps: - determining a hyperplane and / or a superset based on the input variables, at least a subset of the plurality of weights and a subset of the plurality of bias terms.

3. A method according to any one of the preceding claims, wherein: If at least one data point of the test data set exists in each cell, a verification result of the KNN is provided, and the quality of the KNN can be determined based on the verification result.

4. The method according to any one of the preceding claims, wherein the KNN is retrained based on a completed test data set. 5 . The method according to claim 4 , wherein an extended and / or new test data set is provided for the retrained KNN, and the retrained KNN is validated based on the test data set. The method according to claim 4 , wherein the retraining is performed iteratively. 7 . The method according to claim 1 , wherein for the d-dimensional input space: d=10, preferably d<10, particularly preferably d<6, wherein d is an element of the set of positive natural numbers.

8. The method according to any one of the preceding claims, wherein the KNN is part of a complex machine learning model.

9. The method according to any one of claims 1 to 8, wherein the simulation model is designed to generate at least one data point based on the cell parameters, in particular based on hyperplane information and / or superset information, in particular by data enhancement.

10. A system for validating a trained artificial neural network (KNN) based on a test data set (1), The system has a providing device and an evaluating and / or calculating device, wherein the providing device is designed to perform the following steps: providing (S1) a trained KNN having a network architecture, wherein the network architecture represents a chain function, which is particularly nonlinear, by which a d-dimensional input space can be mapped to an N-dimensional output space, and the network architecture also includes multiple weights; providing (S2) a test data set, wherein the evaluating and / or calculating device is designed to perform the following steps: dividing (S3) the d-dimensional input space into multiple cells based on the KNN network architecture, wherein the individual cells can be separated from each other by at least one hyperplane and / or superset that is respectively weight-specific; checking (S4): whether at least one data point of the test data set exists in each of the cells, so as to verify the KNN in this way; and if a data point does not exist in at least one of the cells, generating (S5) at least one new data point based on the cell parameters of at least one of the cells by a simulation model, so as to complete the test data set in this way.

11. Computer program having a program code for carrying out at least part of the method according to any one of claims 1 to 9 when the computer program is executed on a computer. 12 . A computer-readable data carrier having a program code of a computer program in order to carry out at least parts of the method according to claim 1 when the computer program is executed on a computer.

Images