Device, memory medium, computer program and computer-implemented method for validating data-based model

Abstract

To provide a device, a method and a program for validating a data-based model for classifying an object.SOLUTION: There is provided a method for validating a data-based model 206 for classifying an object 208 into a class for an object type 210 or a function type for a driver assistance system of a vehicle 200. The classification is determined using the data-based model depending on a digital signal, especially a digital image 202, especially a radar spectrum or a Lidar spectrum or a segment 204 of one of the spectra. A reference classification for the object is determined using a reference model depending on the digital signal 202. It is checked, depending on the classification and the reference classification, whether or not the classification of the data-based model for the object is correct, and the data-based model is validated or not validated, depending on whether or not the classification of the data-based model for the object is correct.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to an apparatus, a storage medium, a computer program and a computer-implemented method for validating a data-based model. [Background technology]

[0002] Driver assistance systems such as emergency braking assistance and automatic distance / speed control can be realized by video and / or radar sensors. Objects encoded in the data of these sensors can be recognized by object recognition and classified by object type recognition.

[0003] Data-based models can be used for object type recognition. A prerequisite for using data-based models in safety-critical applications is to validate the data-based model and, further, for example, to create a representative data set for validating or training the data-based model. Summary of the Invention [Problem to be solved by the invention]

[0004] Disclosure of the Invention The methods and devices according to the independent claims allow for the validation of data-based models and the creation of representative data sets therefor. [Means for solving the problem]

[0005] A computer-implemented method for validating a data-based model for classifying objects, particularly into classes relating to object type or, for a driver assistance system of a vehicle, into classes relating to function type, comprises: That is, the classification is determined by the data-based model depending on the digital signal, in particular on the digital image, in particular on the radar spectrum or the lidar spectrum, or on a spectral segment of one of these spectra; a reference classification for the object is determined by the reference model depending on the digital signal; whether the classification of the data-based model for the object is correct or not is checked depending on the classification and the reference classification; the data-based model is validated or not depending on whether the classification of the data-based model for the object is correct or not; the classification and the reference classification are preferably determined for a set of digital signals, this set of digital signals being assigned to various distances between the object and a reference point, in particular a vehicle or a sensor that detects the set; a confidence criterion, in particular the distance between the object and the reference point, is determined for each digital signal from this set; the classification of the data-based model for the object is correct for the digital signal; and if the confidence criterion for this digital signal satisfies a condition, in particular that the distance is within a reference distance to the reference point, the data-based model is validated. As the confidence increases, for example as the distance becomes shorter, the reference model reliably recognizes the correct object type. The validation of a data-driven model must be proved with respect to its adherence to the intended function. This method allows for statistical demonstration with respect to a variety of real-world situations. The validation proves that the data-driven model did not lead to an erroneous decision, or even that the decision of the data-driven model was better than that of the reference model, or confirms that this is not the case. The use of a confidence criterion allows for particularly reliable validation.

[0006] It is assumed here that if the confidence criterion satisfies a condition, in particular that the distance is within a reference distance, and the classification differs from the reference classification, the set of digital signals and the reference classification are stored in correspondence with each other, and if not, the digital signals are discarded and / or not stored, thereby recognizing incorrect classifications and creating a data set that is particularly well suited for training with little effort.

[0007] Preferably, a value pair is determined for the set, the value pair including a first value and a second value, the first value representing the distance within which the reference classification for the object is correct, and the second value representing the distance within which the data-based model's classification for the object is correct, or the interval between this distance and the reference distance. The data-based model preferably correctly classifies objects at least within this distance, like the reference model. The first value and the second value are quantities that contain the necessary information for this purpose and can be easily evaluated in a validation test.

[0008] Preferably, a memory location in memory is determined for the value pair, and the value stored in this memory location is changed depending on the value of the value pair. Instead of storing the first and second values ​​themselves, only one value is stored. This is a particularly efficient way to store accumulated information for validation purposes, especially for easily evaluable quantities.

[0009] Preferably, the data-dependent model is validated in dependence on the position stored in this memory location.

[0010] Preferably, classifications of the digital signals and reference classifications are determined for multiple sets of digital signals, and the classification of the data-based model for the object is checked for correctness. The digital signals represent a sequence of individual records occurring at various distances during approaches to the object. These records can be radar, lidar, or video records, or spectra of these records. These records can also be signals derived therefrom. Spectra are one possibility, but point clouds or other derived signals can also be used. The classifications of multiple sets of digital signals represent classifications for multiple approaches of this type. This ensures a large number of different, statistically relevant situations in order to reliably validate the data-based model.

[0011] It is envisioned that for each set from the multiple sets, a value pair including a first value and a second value for the respective set is determined, a memory location is determined for each set for the determined value pair for the set, and the value stored in the memory location is changed depending on the values ​​of the value pair, thereby providing a large number of statistically relevant results that can be used to validate the data-based model.

[0012] It is envisaged here that for each digital signal a position is detected and / or stored, in particular by a satellite navigation system, and the distance is determined depending on this position, so that relevant area-specific data is determined.

[0013] It is contemplated that if the data-based model fails validation, the data-based model may be trained anew, trained with other data, and / or another data-based model may be used.

[0014] The envisioned configuration is that if the data-based model is successfully validated, it will be used in object classification systems, particularly driver assistance systems.

[0015] An apparatus for validating a data-based model for classifying objects includes at least one processor and at least one memory and is configured to implement the above-described method.

[0016] A computer program may be provided that includes machine-readable instructions, which when executed by a computer, cause the above-described method to be carried out.

[0017] A storage medium, in particular a fixed storage medium, may be provided on which the computer program is stored.

[0018] Further advantageous embodiments will become apparent from the following description and drawings. [Brief explanation of the drawings]

[0019] [Figure 1] FIG. [Figure 2] FIG. 1 is a diagram illustrating object recognition. [Figure 3] FIG. 1 illustrates an exemplary method. [Figure 4] FIG. 10 illustrates an array containing entries for validation. [Figure 5] FIG. 1 illustrates an exemplary approaching maneuver toward an object. DETAILED DESCRIPTION OF THE INVENTION

[0020] 1, an apparatus 100 for validating a data-based model is shown. The data-based model is configured to classify an object. The apparatus 100 includes at least one processor 102 and at least one memory 104. Optionally, the apparatus 100 includes at least one sensor 106 and a system 108 for satellite navigation.

[0021] The at least one memory 104, in this embodiment, includes a main memory and a persistent memory, the main memory providing faster access than the persistent memory in this embodiment.

[0022] In this embodiment, the at least one sensor 106 includes a radar sensor. The radar sensor transmits radio frequency signals and receives reflections from stationary and moving objects. These signals are received by the radar sensor's antenna, converted into electrical signals by electronics, and transformed into digital signals by an analog-to-digital converter. Primary signal processing, such as an FFT, transforms the time signals into frequency space.

[0023] The at least one processor 102 and the at least one memory 104 are connected with a data connection in this embodiment. The at least one sensor 106 and / or the system 108 are communicatively connected with the at least one processor 102 by a data connection. The at least one sensor 106 and / or the system 108 can be connected to the device 100 externally or can be integrated within the device 100.

[0024] At least one processor 102 and at least one memory 104 are configured to perform object recognition, object type recognition, and methods or steps therein described below.

[0025] FIG. 2 depicts a schematic diagram of object recognition.

[0026] In this embodiment, the apparatus 100 is located in a vehicle 200. For object recognition, a spectrum 202 of a signal received from at least one radar sensor 106 is provided. In this embodiment, object type recognition is performed in a segment 204 of the spectrum 202 by a data-based model 206. In this embodiment, the data-based model 206 includes an artificial neural network, which is configured, for example, as a convolutional neural network.

[0027] Segment 204, in this example, includes object 208. Data-based model 206, in this example, determines object type 210 of object 208.

[0028] Object recognition, in the sense of whether an object is present or absent in segment 204, can be performed in various ways, for example, a threshold detector can be used, and the distance between the recognized object and the sensor can be determined, for example, by time-of-flight measurements or phase shifts.

[0029] Object recognition and object type recognition are used in this embodiment for driver assistance.

[0030] Important for the quality of driver assistance is the quality of the object type recognition. The object type recognition can be configured to recognize the following object types: cars, motorcycles, pedestrians, and manhole covers. The object types cars, motorcycles, and pedestrians can be assigned to a class called "cannot be ignored and passed through." The object type manhole cover can be assigned to a class called "can be ignored and passed through." The object type recognition can also be configured to recognize other object types. Other classes can also be provided. For example, it is envisaged to provide one class for each object type.

[0031] The quality of object type recognition is estimated, for example, by the correctness of the object type recognition with respect to the distance to the object to be recognized. The greater the distance at which the object is correctly recognized, the better the quality, for example, because this allows the driving characteristics to be adapted to the recognized situation earlier.

[0032] The technical realization of object type recognition can be achieved in various ways. In this embodiment, the data-based model 206 is implemented as an artificial neural network. The data-based model 206 to be validated can also be part of a hybrid model, which in this case refers to a combination of classical signal processing and the data-based model 206. Classical signal processing concepts can be used as a reference model for validating the data-based model 206, or other already established data-based models can be used.

[0033] The reference model may also be a hybrid model, ie a combination of classical signal processing and at least one data-based model.

[0034] Validation of the data-based model 206 can be performed by comparing the results achieved by the data-based model 206 with the results achieved by a reference model, which preferably has a provable predetermined classification quality.

[0035] An exemplary flow of this method is described with reference to FIG. 3. This method exploits the fact that classification quality in the near field is higher than at greater distances. The actual object type of a detected object does not change over time. Therefore, for quality purposes and to identify important data for training, the change in recognized object type during proximity, i.e., while the vehicle 200 is driving toward the actual object, can be exploited. What is important here is the record of a classification result that differs at greater distances from the classification result at close distances. A close distance, for example, means a distance of 3 to 30 meters from the vehicle 200 or at least one sensor 206 to the actual object. From this distance, it can be assumed that the object type predicted by the reference model is correct due to the convergence properties of the reference model.

[0036] The method according to the invention is designed such that, for cost reasons, the required data memory, for example the main memory, with fast access times is small.

[0037] The method starts when an object is first recognized, for example, by an object detector.

[0038] In step 302, sensor data is recorded from a sensor. In this embodiment, the sensor is a radar sensor. This allows the function to receive new sensor data. The sensor data is used to determine a spectrum. In this embodiment, one frame containing the spectrum is determined.

[0039] Step 304 is then performed.

[0040] In step 304, the object is recognized, for example, in a spectrum.

[0041] In step 304, the current segment of the spectrum is determined. In this embodiment, the segment is an excerpt from the spectrum that contains the object. The current segment is stored in a variable S_akt. In this embodiment, the variable S_akt stores one frame that contains the current segment.

[0042] In step 304, the current distance is estimated, which in this example is the distance from the sensor to the object, and is stored in the variable d_akt.

[0043] Step 306 is then performed.

[0044] In step 306, the current segment S_akt is classified by the data-based model 206 on the one hand and by the reference model on the other hand.

[0045] The classification result of the reference model is stored in the variable OT_akt_base for the current object type. The classification result of the data-based model being validated is stored in the variable OT_akt_val for the current object type. The reference model may include the established object recognition algorithm. The data-based model may include the algorithm being validated.

[0046] Step 308 is then performed.

[0047] In step 308, variables that will be used in the subsequent steps are initialized.

[0048] For the data-based model, a variable OT_rel_val for the relevant object type, a variable S_rel_val for the relevant segment, and a variable d_rel_val for the relevant distance are initialized.

[0049] For the reference model, the variables OT_rel_base for the related object type, S_rel_base for the related segment, and d_rel_base for the related distance are initialized. Also, the variable entries for the number of entries is initialized. In this embodiment, the variables are stored in the following correspondence: OT_rel_base=OT_akt_base OT_rel_val=OT_akt_val S_rel_base=S_akt S_rel_val=S_akt d_rel_base=d_akt d_rel_val=d_akt entries=0

[0050] Step 310 is then performed.

[0051] Step 310 forms the start of the main loop.

[0052] In step 310, the following variables are stored in the following correspondence: OT_old_base=OT_akt_base OT_old_val=OT_akt_val S_old_base=S_akt S_old_val=S_akt d_old_base=d_akt d_old_val=d_akt

[0053] For the data-based model, the current object type is stored in the variable OT_old_val, the current sequence in the variable S_old_val, and the current distance in the variable d_old_val.

[0054] For the reference model, the current object type is stored in the variable OT_old_base, the current sequence in the variable S_old_base, and the current distance in the variable d_old_base.

[0055] Step 312 is then performed.

[0056] In step 312, the sensor data is recorded from the sensor, so that the function receives new sensor data. The sensor data is used to determine a spectrum. In this embodiment, one frame containing the spectrum is determined.

[0057] In step 314, the current segment of the spectrum is determined. In this embodiment, the segment is an excerpt from the spectrum that contains the object. The current segment is stored in the variable S_akt. In this embodiment, the variable S_akt stores the frame that contains the current segment.

[0058] In step 314, the current distance is estimated, which in this example is the distance from the sensor to the object, and is stored in the variable d_akt.

[0059] Step 316 is then performed.

[0060] The current segment S_akt is classified by the reference model in step 316. The reference model classification result is stored in the variable OT_akt_base for the current object type.

[0061] Step 318 is then performed.

[0062] In step 318, it is checked whether the current object type and the temporarily stored object type for the reference model match. In this embodiment, it is checked whether object type OT_akt_base != object type OT_old_base.

[0063] If the object types do not match, step 320 is performed, otherwise step 322 is performed.

[0064] By comparing these object types, it is possible to recognize interchange of object types.

[0065] In step 320, that is, when a change has occurred, the temporarily stored data is stored as associated data. In this embodiment, the variables are associated and temporarily stored as follows: OT_rel_base=OT_old_base S_rel_base=S_old_base d_rel_base=d_old_base

[0066] If the object types are the same, the previously stored associated data is retained. The associated data is preferably stored in a main memory, for example a volatile memory.

[0067] In step 322 , the current segment S_akt is classified by the data-based model 206 .

[0068] The classification result of the data-based model being validated is stored in the variable OT_akt_val for the current object type.

[0069] Step 324 is then performed.

[0070] In step 324, for the data-based model being validated, it is checked whether the current object type matches the temporarily stored object type. In this embodiment, it is checked whether object type OT_akt_val != object type OT_old_val.

[0071] If the object types do not match, step 326 is performed, otherwise step 328 is performed.

[0072] By comparing these object types, it is possible to recognize the replacement of object types.

[0073] In step 326, that is, when a replacement has occurred, the temporarily stored data is stored as relevant data. In this embodiment, the variables are stored in association as follows. OT_rel_val = OT_old_val S_rel_val = S_old_val d_rel_val = d_old_val

[0074] If these object types are equal, the previously stored relevant data is continuously retained.

[0075] In step 328, a comparison is made between the current distance to the object and a threshold value. For example, it is checked whether d_akt < SHORTDIST, where SHORTDIST is a stored constant. In this embodiment, the constant SHORTDIST represents a value for a distance between 3 m and 30 m. In this embodiment, it is checked whether the object is present at a short distance. In the case of a short distance, object recognition by the reference model, that is, object recognition by the established algorithm, is considered reliable. If this short distance has not been reached, the main loop starts anew from step 310.

[0076] What is assumed here is to configure it such that in step 328, further, after finally updating the relevant data, it is checked whether a predetermined time has elapsed. If this time has not elapsed, in this embodiment, regardless of whether the short distance has been reached, the main loop starts anew from step 310.

[0077] For example, in step 328, the current time is determined and the difference between the current time and the last time an update occurred is determined. The current time is determined, for example, by the function time_now(). The last time an update occurred is stored, for example, in a variable lastUpdate. This variable is initialized, for example, to zero on the first iteration.

[0078] In this embodiment, if the close distance has been reached and the difference is greater than a threshold, step 330 is performed. The threshold is, for example, a constant RETRIGGER. The constant RETRIGGER may be a time in the range of 10 ms to 1 s. In this embodiment, step 310 is performed otherwise.

[0079] In step 330, the last time an update was performed is set to the value of the current time. In this embodiment, lastUpdate=time_now() is set.

[0080] If close range is reached and therefore the actual object type is identified, the algorithm being validated can be configured to store relevant data and / or insert an entry into the validation array.

[0081] An exemplary procedure for storing relevant data for an algorithm to be validated is referred to below as corner case detection.

[0082] An exemplary procedure for inserting an entry into the validation array is hereinafter referred to as validation.

[0083] Both procedures are performed in parallel in this embodiment and are described in detail below. After step 330, this embodiment performs step 332 to start corner case detection and step 336 to start validation.

[0084] Corner case detection Basically, it is assumed that the segment stored in S_rel_val is relevant. At that time, it is also assumed that the algorithm to be validated for validity has higher classification quality as the distance becomes shorter. However, it is not an essential prerequisite that the algorithm provides a reliable classification when the short distance, in this embodiment, d_akt < SHORTDIST is reached.

[0085] In step 332, it is checked whether the object type recognized by the reference model matches the object type from the relevant data for the data-dependent model 206 to be validated for validity. For example, it is checked whether OT_rel_val!= OT_akt_base. If both object types match, the main loop starts from step 310 and is executed. Otherwise, step 334 is performed. This ensures that segments that have been incorrectly switched to the wrong object type are not stored even though the data-dependent model 206 to be validated for validity has classified the correct object type in a previous iteration.

[0086] In step 334, the relevant data is stored. Preferably, in step 334, the relevant data is stored in the fixed storage memory.

[0087] If the object types are different, the relevant segments are identified and the corresponding relevant data is stored for later use.

[0088] After step 334, in this embodiment, the main loop starts from step 310 and is performed.

[0089] It is envisaged that in a parallel task not shown in Figure 3, data stored in persistent storage memory is transmitted to the computer infrastructure as soon as a sufficient amount of relevant data is available.

[0090] When new data arrives in the computer infrastructure from persistent storage memory, the training process can be initiated.

[0091] During the training process, in this embodiment, a new data-based model 206 is determined. It is contemplated that this model is compiled to generate new firmware and provided to the sensor, e.g., via firmware-over-the-air. It is further contemplated that the new firmware is updated in the sensor, variables are newly initialized, and the method is started anew.

[0092] Validation In this embodiment, validation checks whether the data-based model 206 is suitable for the intended use of the model. A special role in validation is played by situations in which the data-based model 206 being validated fails to classify the correct result.

[0093] In step 336, it is checked whether this situation exists. In this embodiment, it is checked whether OT_akt_val != OT_akt_base. If this situation exists, step 338 is performed. If not, step 340 is performed.

[0094] In step 338, in this embodiment, the ccc value of OTC_val is set to 0 because d_rel_val, in this case, belongs to the previous replacement of the object type, but is therefore not the correct object type. Step 340 is then performed.

[0095] The fundamental metric for the quality of an object recognition algorithm is the distance from which an object can be consistently classified correctly. This is called continuous correct classification, ccc. The ccc value is determined in this example by the function ccc(). For the reference model, the ccc value is determined by the function ccc(OTC_base). For the data-based model 206 being validated, the ccc value is determined by the function ccc(OTC_val). The reference model, in this example, has sufficient classification quality to be provable. In this example, what applies to the data-based model 206 being validated is that it has a ccc value at least as high as the reference model in all relevant situations below the distance threshold DIST_REL.

[0096] To be able to provide this proof, the ccc values ​​of the reference model and the differences Δccc between the ccc values ​​of the reference model and the ccc values ​​of the data-based model 206 being validated are stored in a two-dimensional array, which can be visualized as shown in FIG.

[0097] On the x-axis, the difference Δccc is shown in meters. A range from -200 meters to +200 meters is shown in Figure 4. A number of ranges are defined on the x-axis. One range has an area 402 in the x-direction, which will be referred to below as BIN_SIZE.

[0098] On the y-axis, the ccc value of the reference model is shown in meters. In FIG. 4, a range from 0 meters to 200 meters is shown. In this embodiment, for example, at the boundary 404 at a distance DIST_REL of 150 meters or less, the vicinity area is reached.

[0099] To be able to store the array efficiently, the ccc values are assigned to individual BINs. Each BIN has a size of BIN_SIZE. Thus, the array is, for example, of size (200*2 / BIN_SIZE)×(200 / BIN_SIZE), and an increment of the array occurs at the following (x,y) positions of this array by entries for ccc(OTC_base), ccc(OTC_val).

Number

[0100] Since the data-dependent model 206 to be subjected to the validity check must have at least the classification quality of the established reference model when the distance is shorter than the distance 404, it is desirable that all entries within the range 0 < x < 200 and 0 < y < DIST_REL be smaller than the threshold value, preferably 0. This is the lower right range of FIG. 4. What the entries in this area mean is that the difference between ccc(OTC_base) and ccc(OTC_val) is positive, and thus, the data-dependent model 206 to be subjected to the validity check has a relatively poor ccc value. ccc(OTC_base) and ccc(OTC_val) form a value pair. This value pair includes the first value ccc(OTC_base) representing the following distance, that is, within this distance, the reference classification for the object is appropriate. This value pair includes the second value ccc(OTC_val) representing the following distance, that is, within this distance, the classification of the data-dependent model 206 for the object is appropriate. The difference ccc(OTC_base) - ccc(OTC_val) represents the interval between this distance and the reference distance.

[0101] On the other hand, entries where -200 < x ≤ 0 and 0 < y < DIST_REL are desirably high. This is the lower left range in FIG. 4. What the entries in this range mean is that the data-dependent model 206 to be validated has a higher ccc value than the established reference model.

[0102] For distances greater than or equal to DIST_REL, a relatively high classification quality of the data-dependent model 206 to be validated is similarly desirable, although this is not necessarily the case.

[0103] In step 340, the following sizes are respectively determined for the related data. bin_base = floor(d_rel_base / BIN_SIZE) delta = d_rel_base - d_rel_val bin_val = floor(delta / BIN_SIZE)

[0104] Next, step 342 is performed.

[0105] In step 342, the array is updated. For example, the function ccc_matrix(bin_bas, bin_val)++ is executed. By this function, at the location defined by bin_bas and bin_val, the entry in the array is incremented by 1. As a result, the value stored in this memory location is changed depending on the value of the value pair.

[0106] Moreover, in this embodiment, the number of entries to the array is counted. In this embodiment, the variable entries is incremented by 1, that is, entries++.

[0107] Next, step 344 is performed.

[0108] In step 344, a check is made to see if the number of entries in the array exceeds a threshold. In this embodiment, a check is made to see if the variable entries>MAX_ENTRIES. If the number of entries exceeds the threshold, step 346 is performed. If not, the validation ends.

[0109] In step 346, the array thus generated when MAX_ENTRIES is exceeded is transmitted to the computer infrastructure.

[0110] This array is an efficient representation of the ccc values, and the assumption is that the data-based model 206 will be validated by this array, and so on.

[0111] The validation is then completed.

[0112] It is contemplated that if the data-based model 206 fails validation, the data-based model 206 may be trained anew, trained with other data, and / or another data-based model may be used.

[0113] It is envisioned that if the data-based model 206 is successfully validated, then the data-based model 206 is configured for use in an object classification system, particularly in a driver assistance system.

[0114] It is contemplated that the method may be implemented with multiple vehicles, and that the data-based model 206 may be validated with a constellation of those vehicles.

[0115] These sequences are used, for example, for statistical validation of the data-based model 206 .

[0116] An exemplary approaching maneuver toward an object is shown diagrammatically in Figure 5. The x-axis indicates the distance to the object as a negative value. The y-axis indicates the object type. In this example, an object of arbitrarily selected object type class 3 is the object of interest.

[0117] The object types predicted by the established reference model are depicted as triangles at different distances, and the object types predicted by the data-based model 206 being validated are depicted as circles at different distances.

[0118] In this example, the reference model consistently classifies objects correctly from a distance of 8 m. The data-based model 206 being validated consistently classifies objects correctly already from 10 m.

[0119] In this example, when the neighborhood is reached, for example at a distance of 8 meters, the correct object type is identified, and at a distance of 11 meters, the final incorrect classification is transmitted by the data-based model being validated. In this example, ccc(OTC_base)=8 and ccc(OTC_val)=10, which results in an increment of the validation array at (8,-2).

[0120] The data-based model 206 being validated can be configured to identify data when it incorrectly classifies an object type. This data is, for example, data that an established reference model classifies differently. This data is particularly important for training the data-based model 206, for example, a neural network for classification, because it reveals weaknesses in object recognition in its current state.

[0121] Instead of using the distance d_akt and the threshold SHORTDIST to determine that the data-based model 206 being validated has correctly classified an object, a confidence metric for object recognition by the reference model can additionally or alternatively be used, such as provided by the reference model and based, for example, on the duration of stable classification by the reference model.

[0122] Similarly, it is envisioned that the ccc intervals between the reference model and the data-based model 206 being validated are determined as the distance to the object increases. For example, when an object that was previously within a region where proper classification was possible subsequently moves out of this region, a ccc interval beyond which ccc is no longer possible is determined. This can be accomplished by following the procedure described above. Corner case detection can also be performed in this case.

[0123] Instead of using the distance to the classified object as a criterion for a reliable classification result of the established reference model, other confidence criteria can be used. For example, if a stable or unchanging classification result occurs for a predetermined period longer than a threshold, e.g., t_stable, the classification result of the established reference model can be considered reliable. Therefore, subsequent runs that are not closely spaced can also be used for validation, regardless of the distance to the object.

[0124] An exemplary object classification is based on spectral segments. Instead of being based on spectral segments, object classification may be based on other input quantities. For example, this approach can also be used in location-based object recognition algorithms, which replace or complement object classification based on spectral segments. In this case, corresponding data, i.e., positions, are stored as associated data instead of spectra.

[0125] In the embodiment described so far, corner case detection checks whether the object types, e.g., OT_rel_val and OT_akt_base, are different, thereby ensuring that data that resulted in a correct classification is not stored in persistent memory. Alternatively, it is envisioned that when the recognized object type, e.g., OT_akt_base and OT_akt_val, is not equal, the object type currently recognized by the reference model, e.g., OT_akt_val, is stored in persistent memory instead of the object type previously recognized by the reference model, e.g., OT_rel_val. This is advantageous because the data-based model 206 would, in this case, also produce an incorrect classification result for the current segment.

[0126] It is contemplated that in addition to the data already described, the GPS positions of the individual data detections may be stored and provided by the vehicle via a bus system. The GPS positions provide data that can be used to train the data-based model 206 in a location-specific manner.

[0127] The object type comparison can be replaced by other functions, such as for example automatic emergency braking or automatic emergency avoidance intervention, which means that the reaction of the function to the respective recognized object type is used.

Claims

Claim 1 A computer-implemented method for validating a data-dependent model (206) that classifies an object (208) into a class specific to an object type (210) or into a class related to a function type for a driver assistance system of a vehicle (200), the method comprising: the classification being determined (322, 324, 326) by the data-dependent model (206) depending on a digital signal, in particular a digital image (202), in particular in a radar spectrum or a Lidar spectrum or a segment (204) of one of the spectra; a reference classification for the object (208) being determined (316, 318, 320) by a reference model depending on the digital signal (202); checking (336) whether the classification of the data-dependent model (206) for the object (208) is appropriate depending on the classification and the reference classification; the data-dependent model (206) being validated or not validated depending on whether the classification of the data-dependent model (206) for the object (208) is appropriate; wherein in the computer-implemented method: the classification and the reference classification are determined for a set of digital signals, the set of digital signals being assigned to various distances between the object and a reference point, in particular the vehicle or a sensor that detects the set; a confidence criterion, in particular the distance between the object and the reference point, being determined (314) for each digital signal from the set; the data-dependent model is validated (328) if the classification of the data-dependent model for the object is appropriate in the digital signal and the confidence criterion of the digital signal meets a condition, in particular the condition that the distance is within a reference distance to the reference point; characterized in that it is a computer-implemented method. Claim 2 If the confidence criterion meets the condition, in particular the condition that the distance is within the reference distance, and the classification is different from the reference classification, the set of digital signals and the reference classification are stored in association with each other (346), otherwise the digital signal is discarded and / or not stored; The method according to claim 1. Claim 3 For the set, a value pair including a first value and a second value is obtained, the first value representing the following distance, i.e., the distance within which the reference classification for the object is appropriate, and the second value representing the following distance, i.e., the distance within which the classification of the data-dependent model for the object is appropriate, or the interval between the distance and the reference distance. The method according to claim 1.

4. A memory location in the memory is obtained for the value pair, and the value stored in the memory location is changed depending on the values of the value pair. The method according to claim 3.

5. The data-dependent model is verified for validity depending on the location stored in the memory location. The method according to claim 4.

6. For a plurality of sets of digital signals, the classification and reference classification of the digital signals are obtained, and it is checked whether the classification of the data-dependent model for the object is appropriate. The method according to claim 1.

7. For each set from the plurality of sets, a value pair including a first value and a second value for the individual set is obtained, a memory location for the value pair obtained for the set is obtained for each set, and the value stored in the memory location is changed depending on the values of the value pair. The method according to claim 6.

8. For each digital signal, one position is detected and / or stored, in particular by a satellite navigation system (108), and the distance is obtained depending on the position. The method according to claim 1.

9. If the verification of the validity of the data-dependent model (206) fails, the data-dependent model (206) is newly trained, trained with other data, and / or another data-dependent model is used. The method according to claim 1.

10. If the verification of the validity of the data-dependent model (206) is successful, the data-dependent model (206) is used in an object classification system, in particular in a driver assistance system. The method according to claim 1.

11. In an apparatus (100) for verifying the validity of a data-dependent model for classifying an object The apparatus includes at least one processor (102) and at least one memory (104), and is characterized in that the apparatus is configured to implement the method according to any one of claims 1 to 10. An apparatus (100) for validating a data-dependent model for classifying objects. **Claim 12** In a computer program, the computer program includes machine-readable instructions, and is characterized in that when the machine-readable instructions are executed by a computer, they are for implementing the method according to any one of claims 1 to 10. **Claim 13** In a storage medium, particularly a fixed storage medium, the storage medium is characterized in that the computer program according to claim 12 is stored therein.

Images