Apparatus, storage medium, computer program, and computer-implemented method for validating data-dependent models.

Abstract

To provide a device, a method and a program for validating a data-based model for classifying an object.SOLUTION: There is provided a method for validating a data-based model 206 for classifying an object 208 into a class for an object type 210 or a function type for a driver assistance system of a vehicle 200. The classification is determined using the data-based model depending on a digital signal, especially a digital image 202, especially a radar spectrum or a Lidar spectrum or a segment 204 of one of the spectra. A reference classification for the object is determined using a reference model depending on the digital signal 202. It is checked, depending on the classification and the reference classification, whether or not the classification of the data-based model for the object is correct, and the data-based model is validated or not validated, depending on whether or not the classification of the data-based model for the object is correct.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present invention relates to an apparatus, a storage medium, a computer program, and a computer-implemented method for validating a data-dependent model.

Background Art

[0002] Driver assistance systems such as emergency brake assistance and distance / speed automatic control can be realized by video sensors and / or radar sensors. Objects encoded in the data of these sensors can be recognized by object recognition and classified by object type recognition.

[0003] A data-dependent model can be used for object type recognition. In applications critical to safety, a prerequisite for using a data-dependent model is to validate the data-dependent model, and further, for example, to create a typical data set for validating or training the data-dependent model.

Summary of the Invention

Problems to be Solved by the Invention

[0004] Disclosure of the Invention According to the method and apparatus described in the independent claims, it is possible to validate a data-dependent model and create a typical data set therefor.

Means for Solving the Problems

[0005] A computer-implemented method for validating a data-dependent model for classifying objects, particularly in terms of object type classes or function types for vehicle driver assistance systems, is configured as follows: That is, classification is determined by a data-dependent model depending on a digital signal, particularly a digital image, particularly a radar spectrum or Lidar spectrum, or a segment of one of these spectra; a reference classification for an object is determined by a reference model depending on the digital signal; the validity of the classification of the data-dependent model for an object is checked depending on the classification and the reference classification; the data-dependent model is validated or not validated depending on whether the classification of the data-dependent model for an object is valid; the classification and reference classification are preferably determined for a set of digital signals, which are assigned to various distances between an object and a reference point, particularly a vehicle or a sensor that detects the set; for each digital signal from this set, a confidence criterion, particularly the distance between the object and the reference point, is determined; the data-dependent model is validated if the classification of the data-dependent model for an object is valid in the digital signal and the confidence criterion for this digital signal satisfies the condition, particularly that the distance is within the reference distance to the reference point. As confidence increases, for example as the distance decreases, the reference model reliably recognizes the correct object type. Validation of data-dependent models requires proof of compliance with their intended function. This method allows for statistical argumentation regarding various real-world situations. Validation proves that the data-dependent model did not lead to incorrect judgments, or even that its judgments were superior to those of the reference model, or confirms that this is not the case. Using confidence criteria enables exceptionally reliable validation.

[0006] The assumption here is that if the confidence criteria meet the conditions, particularly that the distance is within the reference distance, and the classification differs from the reference classification, then the set of digital signals and the reference classification are associated with each other and stored; otherwise, the digital signals are discarded and / or not stored. This allows for the recognition of incorrect classifications and the creation of a dataset particularly well-suited for training with minimal effort.

[0007] Preferably, for this set, a value pair is obtained that includes a first value and a second value, where the first value represents the following distance, i.e., the distance within which the standard classification of an object is appropriate, and the second value represents the following distance, i.e., the distance within which the classification of a data-dependent model of an object is appropriate, or the interval between this distance and the standard distance. It is desirable that the data-dependent model appropriately classifies objects within at least this distance, as in the standard model. The first and second values ​​contain the necessary information for this purpose and are quantities that can be easily evaluated in validation testing.

[0008] Preferably, a memory location is determined for each value pair, and the value stored in this memory location is modified depending on the value of the value pair. Instead of storing the first and second values ​​themselves, only one value is stored. This is a particularly efficient method for storing accumulated information, especially with respect to easily quantifiable quantities, for validation purposes.

[0009] Preferably, the data-dependent model is validated based on the location stored in this memory location.

[0010] Preferably, for multiple sets of digital signals, classification and reference classification of these digital signals are required, and the appropriateness of the classification of the data-dependent model concerning the object is checked. The digital signals represent a sequence of individual recordings that occur at various distances during approach to an object. These recordings can be radar recordings, Lidar recordings, or video recordings, or spectra of such recordings. These recordings can also be signals derived therefrom. While spectra are one possibility, point clouds or other derived signals can also be used. The classification of multiple sets of digital signals represents classifications for a large number of such approaching events. This ensures a large number of statistically relevant and diverse situations for the purpose of reliably validating the data-dependent model.

[0011] The assumption here is that for each set from a large number of sets, a value pair containing a first and second value for that set is determined, a memory location for the value pair determined for that set is determined, and the value stored in this memory location is modified depending on the value of the value pair. This provides a large amount of statistically relevant results that can be used to validate the data-dependent model.

[0012] The assumption here is that one position is detected and / or stored for each digital signal, particularly by a satellite navigation system, and the distance is determined based on this position. This allows for the acquisition of region-specific relevant data.

[0013] The intended configuration here is that if the validation of the data-dependent model fails, the data-dependent model is retrained, trained with other data, and / or another data-dependent model is used.

[0014] The assumption here is that, if the validation of the data-dependent model is successful, the data-dependent model will be used in the object classification system, particularly in the driver assistance system.

[0015] An apparatus for validating a data-dependent model for classifying objects includes at least one processor and at least one memory, and is configured to carry out the method described above.

[0016] A computer program can be provided that includes machine-readable instructions, and when these machine-readable instructions are executed by the computer, the above-described method is carried out.

[0017] A storage medium, particularly a fixed storage medium, can be provided in which computer programs are stored.

[0018] Further advantageous embodiments will become apparent from the following description and drawings. [Brief explanation of the drawing]

[0019] [Figure 1] This is a diagram illustrating the device. [Figure 2] This is a schematic diagram illustrating object recognition. [Figure 3] This diagram illustrates an exemplary method. [Figure 4] This figure shows the sequence containing entries for validation testing. [Figure 5] This diagram illustrates an example of movement approaching an object. [Modes for carrying out the invention]

[0020] FIG. 1 schematically shows an apparatus 100 for validating a data-dependent model. The data-dependent model is configured to classify objects. The apparatus 100 includes at least one processor 102 and at least one memory 104. Optionally, the apparatus 100 includes at least one sensor 106 and a satellite navigation system 108.

[0021] In this embodiment, the at least one memory 104 includes a main memory and a fixed storage memory. The main memory provides faster access than the fixed storage memory in this embodiment.

[0022] In this embodiment, the at least one sensor 106 includes a radar sensor. The radar sensor transmits high-frequency signals and receives reflections from stationary and moving objects. These signals are received by the radar sensor's antenna, converted into electrical signals by an electronic device, and shifted to digital signals by an analog / digital converter. Through primary signal processing, such as FFT, the time signal is shifted to the frequency domain.

[0023] In this embodiment, the at least one processor 102 and the at least one memory 104 are connected by a data connection line. The at least one sensor 106 and / or the system 108 are connected to the at least one processor 102 by a data connection line for communication. The at least one sensor 106 and / or the system can be connected to the apparatus 100 from outside the apparatus 100 or integrated and implemented within the apparatus 100.

[0024] The at least one processor 102 and the at least one memory 104 are configured to perform object recognition, object type recognition, and the methods or steps in the methods described below.

[0025] FIG. 2 depicts a diagram schematically showing object recognition.

[0026] In this embodiment, the device 100 is located inside the vehicle 200. For object recognition, a spectrum 202 of signals received from at least one radar sensor 106 is provided. In this embodiment, object type recognition is performed in one segment 204 of the spectrum 202 by a data-dependent model 206. In this embodiment, the data-dependent model 206 includes an artificial neural network, which is configured, for example, as a convolutional neural network.

[0027] In this embodiment, segment 204 includes object 208. In this embodiment, the object type 210 of object 208 is determined by the data-dependent model 206.

[0028] Object recognition, specifically determining whether an object exists within segment 204 or not, can be performed using various methods. For example, a threshold detector can be used. The distance between the recognized object and the sensor can then be determined, for example, by measuring propagation time or phase shift.

[0029] Object recognition and object type recognition are used in this embodiment for driver assistance.

[0030] A crucial aspect of driver assistance quality is the quality of object type recognition. Object type recognition can be configured to recognize the following object types: private cars, motorcycles, pedestrians, and manhole covers. The object types of cars, motorcycles, and pedestrians can be assigned to the "cannot be ignored and passed" class. The object type of manhole cover can be assigned to the "can be ignored and passed" class. Object type recognition can also be configured to recognize other object types. Other classes can also be created. For example, one class can be created for each object type.

[0031] The quality of object type recognition can be estimated, for example, by the accuracy of object type recognition relative to the distance to the object to be recognized. The greater the distance at which accurate recognition occurs, the better the quality, for example. This is because it allows for earlier adjustment of driving characteristics to match the recognized situation.

[0032] The technical realization of object type recognition is possible through various methods. In this embodiment, the data-dependent model 206 is implemented as an artificial neural network. The data-dependent model 206 to be validated can also be part of a hybrid model. In this case, a hybrid model refers to a combination of classical signal processing and the data-dependent model 206. Classical signal processing concepts can be used as a reference model for validating the data-dependent model 206, or other already established data-dependent models can be used.

[0033] The reference model can also be a hybrid model, i.e., a combination of classical signal processing and at least one data-dependent model.

[0034] The validity of the data-dependent model 206 can be verified by comparing the results achieved by the data-dependent model 206 with the results achieved by the reference model. The reference model preferably has a predetermined verifiable classification quality.

[0035] An exemplary flow of this method will be explained with reference to Figure 3. This method utilizes the fact that the quality of classification in the immediate vicinity is higher than at greater distances. The actual object type of the detected object does not change over time. Therefore, for the purpose of determining quality and identifying important data for training, it is possible to utilize the fact that changes in the recognized object type occurred during approach, that is, while the vehicle 200 was approaching and driving towards the real object. What is important here is the record of the classification result that was different at a greater distance from the classification result at a shorter distance. A shorter distance is, for example, a distance of 3 to 30 meters from the vehicle 200 or at least one sensor 206 to the real object. From this distance, it can be assumed that the object type predicted by the reference model is appropriate due to the convergence characteristics of the reference model.

[0036] In the method according to the present invention, the required data memory, such as main memory, which has a fast access time, is configured to be small for cost reasons.

[0037] This method starts, for example, when an object is first detected by an object detector.

[0038] In step 302, sensor data from the sensor is recorded. In this embodiment, the sensor is a radar sensor. As a result, this function receives new sensor data. The spectrum is obtained using the sensor data. In this embodiment, one frame containing the spectrum is obtained.

[0039] Next, step 304 is performed.

[0040] In step 304, the object is recognized. For example, the object is recognized in the spectrum.

[0041] In step 304, the current segment of the spectrum is determined. In this embodiment, the segment is an excerpt from the spectrum that contains the object. The current segment is stored in the variable S_akt. In this embodiment, one frame containing the current segment is stored in the variable S_akt.

[0042] In step 304, the current distance is estimated. In this embodiment, this distance is the distance from the sensor to the object. The current distance is stored in the variable d_akt.

[0043] Next, step 306 is performed.

[0044] In step 306, the current segment S_akt is classified by data-dependent model 206 on the one hand and by the reference model on the other.

[0045] The classification result of the reference model is stored in the variable OT_akt_base for the current object type. The classification result of the data-dependent model being validated is stored in the variable OT_akt_val for the current object type. The reference model may include an established object recognition algorithm. The data-dependent model may include an algorithm being validated.

[0046] Next, step 308 is performed.

[0047] In step 308, variables that will be used in subsequent processes are initialized.

[0048] For the data-dependent model, the variables OT_rel_val for related object types, S_rel_val for related segments, and d_rel_val for related distances are initialized.

[0049] For the reference model, the variables OT_rel_base for related object types, S_rel_base for related segments, and d_rel_base for related distances are initialized. In addition, the variable entries for the number of entries is initialized. In this embodiment, the variables are stored in the following correspondence. OT_rel_base=OT_akt_base OT_rel_val=OT_akt_val S_rel_base=S_akt S_rel_val=S_akt d_rel_base=d_akt d_rel_val=d_akt entry=0

[0050] Next, step 310 is performed.

[0051] Step 310 marks the start of the main loop.

[0052] In step 310, the following variables are stored in the following correspondence: OT_old_base=OT_akt_base OT_old_val=OT_akt_val S_old_base=S_akt S_old_val=S_akt d_old_base=d_akt d_old_val=d_akt

[0053] In the data-dependent model, the current object type is stored in the variable OT_old_val, the current sequence in the variable S_old_val, and the current distance in the variable d_old_val.

[0054] For the reference model, the current object type is stored in the variable OT_old_base, the current sequence in the variable S_old_base, and the current distance in the variable d_old_base.

[0055] Next, step 312 is performed.

[0056] In step 312, sensor data is recorded. This allows the function to receive new sensor data. The spectrum is then calculated using this sensor data. In this embodiment, one frame containing the spectrum is obtained.

[0057] In step 314, the current segment of the spectrum is determined. In this embodiment, the segment is an excerpt from the spectrum containing the object. The current segment is stored in the variable S_akt. In this embodiment, the frame containing the current segment is stored in the variable S_akt.

[0058] In step 314, the current distance is estimated. In this embodiment, this distance is the distance from the sensor to the object. The current distance is stored in the variable d_akt.

[0059] Next, step 316 is performed.

[0060] In step 316, the current segment S_akt is classified by the reference model. The classification result of the reference model is stored in the variable OT_akt_base for the current object type.

[0061] Next, step 318 is performed.

[0062] In step 318, a check is performed to determine whether the current object type matches the temporarily stored object type for the reference model. In this embodiment, it is checked whether object type OT_akt_base != object type OT_old_base.

[0063] If these object types do not match, step 320 is performed. Otherwise, step 322 is performed.

[0064] By comparing these object types, we can recognize when object types are swapped.

[0065] In step 320, that is, when a swap has occurred, the temporarily stored data is stored as related data. In this embodiment, the variables are temporarily stored in the following correspondence. OT_rel_base=OT_old_base S_rel_base=S_old_base d_rel_base=d_old_base

[0066] If these object types are the same, the previously temporarily stored associated data is retained. The associated data is preferably stored in main memory, for example, in volatile memory.

[0067] In step 322, the current segment S_akt is classified by the data-dependent model 206.

[0068] The classification result of the data-dependent model being validated is stored in the variable OT_akt_val for the current object type.

[0069] Next, step 324 is performed.

[0070] In step 324, the data-dependent model being validated is checked to see if the current object type matches the temporarily stored object type. In this embodiment, it is checked whether object type OT_akt_val != object type OT_old_val.

[0071] If these object types do not match, step 326 is performed. Otherwise, step 328 is performed.

[0072] By comparing these object types, it is possible to recognize the replacement of object types.

[0073] In step 326, that is, when a replacement has occurred, the temporarily stored data is stored as relevant data. In this embodiment, the variables are stored in the following associated manner. OT_rel_val = OT_old_val S_rel_val = S_old_val d_rel_val = d_old_val

[0074] If these object types are equal, the previously stored relevant data is continuously retained.

[0075] In step 328, a comparison is performed between the current distance to the object and a threshold value. For example, it is checked whether d_akt < SHORTDIST, where SHORTDIST is a stored constant. The constant SHORTDIST represents a value for a distance of 3 m to 30 m in this embodiment. In this embodiment, it is checked whether the object is present at a short distance. In the case of a short distance, object recognition by the reference model, that is, object recognition by the established algorithm, is considered reliable. If this short distance has not been reached, the main loop starts anew from step 310.

[0076] What is assumed here is that in step 328, further, after finally updating the relevant data, it is configured to check whether a predetermined time has elapsed. If this time has not elapsed, in this embodiment, regardless of whether the short distance has been reached, the main loop starts anew from step 310.

[0077] For example, in step 328, the current time is determined, and the difference between the current time and the time of the last update is calculated. The current time can be obtained, for example, by the function time_now(). The time of the last update is stored, for example, in the variable lastUpdate. This variable is initialized to zero, for example, in the first iteration.

[0078] In this embodiment, if the proximity has been reached and the above difference is greater than the threshold, step 330 is performed. The threshold is, for example, a constant RETRIGGER. The constant RETRIGGER can be a time in the range of 10 ms to 1 s. In this embodiment, in all other cases, step 310 is performed.

[0079] In step 330, the value of the current time is set to the time when the last update occurred. In this example, lastUpdate=time_now() is set.

[0080] If a short distance has been reached and the actual object type has been identified, the system can be configured to store relevant data for the algorithm being validated and / or to insert an entry into the validation sequence.

[0081] In the following, an exemplary procedure for storing relevant data for an algorithm to be validated will be referred to as corner case detection.

[0082] In the following, an exemplary procedure for inserting an entry into a validation sequence will be referred to as "validation."

[0083] Both procedures are executed in parallel in this embodiment, and these will be described in detail below. Following step 330, in this embodiment, step 332 is performed to start corner case detection and step 336 is performed to start validation.

[0084] Corner case detection Basically, it is assumed that the segment stored in S_rel_val is relevant. At that time, it is also assumed that the algorithm to be validated for validity also has higher classification quality as the distance becomes shorter. However, it is not an essential prerequisite that the algorithm already provides a reliable classification when the short distance, in this embodiment, d_akt < SHORTDIST is reached.

[0085] In step 332, it is checked whether the object type recognized by the reference model matches the object type from the relevant data for the data-dependent model 206 to be validated for validity. For example, it is checked whether OT_rel_val!= OT_akt_base. If both object types match, the main loop starts from step 310 and is executed. Otherwise, step 334 is performed. This ensures that segments that have been misclassified into an incorrect object type are not stored even though the data-dependent model 206 to be validated for validity had already classified the correct object type in a previous iteration.

[0086] In step 334, the relevant data is stored. Preferably, in step 334, the relevant data is stored in the fixed memory.

[0087] If the object types are different, the relevant segments are identified and the corresponding relevant data is stored for later use.

[0088] After step 334, in this embodiment, the main loop starts from step 310 and is performed.

[0089] The assumption here is that, in parallel tasks not shown in Figure 3, if a sufficient amount of relevant data exists, the data stored in fixed memory will be immediately transmitted to the computer infrastructure.

[0090] When new data arrives in the computer infrastructure from fixed memory, the training process can be initiated.

[0091] In the training process, in this embodiment, a new data-dependent model 206 is required. It is assumed that this model is compiled to generate new firmware, which is then supplied to the sensor, for example, via firmware over-the-air. Furthermore, it is assumed that the new firmware is updated on the sensor, the variables are reinitialized, and the process is restarted.

[0092] Validity testing In this embodiment, validation is performed to confirm whether the data-dependent model 206 is suitable for the intended use of this model. A special role in the validation is played when the data-dependent model 206 being validated fails to classify the appropriate results.

[0093] In step 336, it is checked whether this situation has occurred. In this embodiment, it is checked whether OT_akt_val != OT_akt_base. If this situation has occurred, step 338 is performed. Otherwise, step 340 is performed.

[0094] In step 338, in this embodiment, the ccc value of OTC_val is set to 0. This is because, although d_rel_val is a replacement for the object type immediately preceding it in this case, it is not a valid object type. Next, step 340 is performed.

[0095] The fundamental metric for the quality of an object recognition algorithm is distance, which represents the distance at which objects could be consistently classified correctly. This is called continuous correct classification (ccc). In this embodiment, the ccc value is obtained by the function ccc(). For the reference model, the ccc value is obtained by the function ccc(OTC_base). For the data-dependent model 206 being validated, the ccc value is obtained by the function ccc(OTC_val). In this embodiment, the reference model has sufficient classification quality to be provable. In this embodiment, applying this to the data-dependent model 206 being validated means that this model has a ccc value that is at least exactly the same as the reference model, below the distance threshold DIST_REL, in all relevant situations.

[0096] To enable this proof, the ccc value of the reference model and the difference Δccc between the ccc value of the reference model and the ccc value of the data-dependent model 206 being validated are stored in a two-dimensional array. This array can be visualized as shown in Figure 4.

[0097] The difference Δccc is shown in meters on the x-axis. Figure 4 shows the range from -200 meters to +200 meters. Numerous ranges are defined on the x-axis. One range has an occupied area of ​​402 in the x-direction, which will be referred to as BIN_SIZE below.

[0098] On the y-axis, the ccc values of the reference model are shown in meters. In FIG. 4, a range from 0 meters to 200 meters is shown. In this embodiment, for example, at the boundary 404 at a distance DIST_REL of 150 meters or less, it reaches the vicinity region.

[0099] To be able to store the array efficiently, the ccc values are assigned to individual BINs. Each BIN has a size of BIN_SIZE. Thus, the array is, for example, of size (200*2 / BIN_SIZE)×(200 / BIN_SIZE), and an increment of the array occurs at the following (x,y) positions of this array by entries for ccc(OTC_base), ccc(OTC_val).

Number

[0100] Since the data-dependent model 206 to be subjected to the validity check must have at least the classification quality of the established reference model when it is at a distance shorter than the distance 404, it is desirable that all entries within the range 0 < x < 200 and 0 < y < DIST_REL be smaller than the threshold value, preferably 0. This is the lower right range of FIG. 4. What the entries in this region mean is that the difference between ccc(OTC_base) and ccc(OTC_val) is positive, and thus, the data-dependent model 206 to be subjected to the validity check has a relatively poor ccc value. ccc(OTC_base) and ccc(OTC_val) form a value pair. This value pair includes the first value ccc(OTC_base) representing the following distance, that is, within this distance, the reference classification for the object is appropriate. This value pair includes the second value ccc(OTC_val) representing the following distance, that is, within this distance, the classification of the data-dependent model 206 for the object is appropriate. The difference ccc(OTC_base) - ccc(OTC_val) represents the interval between this distance and the reference distance.

[0101] On the other hand, entries where -200 < x ≤ 0 and 0 < y < DIST_REL are desirably high. This is the lower left range in FIG. 4. What the entries in this range mean is that the data-dependent model 206 to be validated has a higher ccc value than the established reference model.

[0102] For distances greater than or equal to DIST_REL, a relatively high classification quality of the data-dependent model 206 to be validated is similarly desirable, although this is not necessarily so.

[0103] In step 340, the following magnitudes are determined for the relevant data respectively. bin_base = floor(d_rel_base / BIN_SIZE) delta = d_rel_base - d_rel_val bin_val = floor(delta / BIN_SIZE)

[0104] Next, step 342 is implemented.

[0105] In step 342, the array is updated. For example, the function ccc_matrix(bin_bas, bin_val)++ is executed. By this function, at the location defined by bin_bas and bin_val, the entry in the array is incremented by 1. Thereby, the value stored in this memory location is changed depending on the value of the value pair.

[0106] Moreover, in this embodiment, the number of entries into the array is counted. In this embodiment, the variable entries is incremented by 1, that is, entries++.

[0107] Next, step 344 is implemented.

[0108] In step 344, it is checked whether the number of entries in the array exceeds a threshold. In this example, it is checked whether the variable entries > MAX_ENTRIES. If the number of entries exceeds the threshold, step 346 is performed. Otherwise, the validation check ends.

[0109] In step 346, when the number of entries exceeds MAX_ENTRIES, the resulting array is transmitted to the computer infrastructure.

[0110] This sequence is an efficient representation of the ccc value. The assumption here is that the data-dependent model 206 will be validated using this sequence.

[0111] Next, the validation test is completed.

[0112] The assumption here is that if the validation of data-dependent model 206 fails, data-dependent model 206 will be newly trained, trained with other data, and / or another data-dependent model will be used.

[0113] The assumption here is that, if the validation of the data-dependent model 206 is successful, the data-dependent model 206 will be used in the object classification system, particularly in the driver assistance system.

[0114] The assumption here is that this method is implemented by multiple vehicles. Furthermore, the assumption here is that the data-dependent model 206 is validated by the sequence of those vehicles.

[0115] These sequences can be used, for example, to test the statistical validity of data-dependent model 206.

[0116] Figure 5 schematically shows an exemplary movement towards an object. The x-axis indicates the distance to the object as a negative value. The object type is written on the y-axis. In this embodiment, an object of arbitrarily selected object type class 3 is targeted.

[0117] The object types predicted by the established baseline model are depicted as triangles at different distances. The object types predicted by the data-dependent model 206, which is being validated, are depicted as circles at different distances.

[0118] In this embodiment, the reference model continuously classifies objects correctly from a distance of 8m. The data-dependent model 206, which is the subject of validation testing, has already continuously classified objects correctly from a distance of 10m.

[0119] In this embodiment, when the object reaches the vicinity region, for example at a distance of 8 meters, the correct object type is identified, and at a distance of 11 meters, the data-dependent model being validated transmits the last incorrect classification. In this embodiment, ccc(OTC_base)=8 and ccc(OTC_val)=10. That is, as a result, the validation sequence is incremented at (8, -2).

[0120] The data-dependent model 206, which is the subject of validation, can be configured to identify data where it has incorrectly classified an object type. This data may be, for example, data that an established reference model has classified differently. This data is particularly important for training the data-dependent model 206, for example, a neural network for classification, because it reveals weaknesses in object recognition in the current state each time.

[0121] Instead of using the distance d_akt and threshold SHORTDIST to determine whether the data-dependent model 206, which is under validation, has correctly classified the object, a confidence criterion for object recognition by a reference model can be used additionally or selectively. This confidence criterion may be provided by the reference model and may be based, for example, on the duration of stable classification by the reference model.

[0122] Similarly, the assumption here is that as the distance to the object increases, the ccc interval is determined for both the reference model and the data-dependent model 206, which is the object being validated. When an object that was continuously able to be classified correctly then moves out of this area, for example, a ccc interval beyond which ccc is no longer possible can be determined. This is carried out using the procedure described above. Corner case detection can also be performed in this case.

[0123] Instead of using the distance to the classified object as a criterion for reliable classification results of an established reference model, other confidence criteria can be used. For example, if stable, i.e., unchanging, classification results are obtained over a predetermined period longer than a threshold, e.g., t_stable, the classification results of the established reference model can be considered reliable. Therefore, regardless of the distance to the object, follow-up driving that does not approach at close intervals can also be used for validation.

[0124] The exemplary object classification is based on spectral segments. Instead of being based on spectral segments, object classification may be based on other input quantities. For example, this approach can also be used in location-based object recognition algorithms that replace or complement spectral segment-based object classification. In this case, the corresponding data, i.e., the position, is stored as relevant data instead of the spectrum.

[0125] In the embodiments described so far, corner case detection checks whether the object types, for example, OT_rel_val and OT_akt_base, are different, thereby ensuring that data resulting in a correct classification is not stored in fixed memory. Alternatively, it is assumed that when the recognized object types, for example, OT_akt_base and OT_akt_val, are not equal, the object type currently recognized by the reference model, e.g., OT_akt_val, is stored in fixed memory, rather than the object type previously recognized by the reference model, e.g., OT_rel_val. This is advantageous because, in this case, the data-dependent model 206 would also yield an incorrect classification result for the current segment.

[0126] The intended configuration here is that, in addition to the previously described data, individual GPS positions for data detection are stored. This can be provided from the vehicle via the bus system. By using GPS positions, data is provided that allows the data-dependent model 206 to be trained in a region-specific manner.

[0127] Object type comparison can be replaced by other functions, such as automatic emergency braking or automatic emergency avoidance intervention. This means that the function's reaction to each recognized object type is used.

Claims

1. A computer-implemented method for validating a data-dependent model (206) that classifies objects (208) into classes relating to object types (210), or into classes relating to function types for driver assistance systems of a vehicle (200), The classification is determined by the data-dependent model (206) (322, 324, 326), depending on the digital signal, particularly the digital image (202), particularly the radar spectrum or the Lidar spectrum, or a segment (204) of one of the spectra. Depending on the digital signal (202), a reference model is used to determine the reference classification of the object (208) (316, 318, 320). Depending on the classification and the standard classification, it is checked whether the classification of the data-dependent model (206) for the object (208) is appropriate (336). The data-dependent model (206) is validated or not validated depending on whether the classification of the data-dependent model (206) with respect to the object (208) is appropriate. In computer-implemented methods, The aforementioned classification and the aforementioned reference classification are determined for a digital signal set, and the digital signal set is assigned to various distances between the object and a reference point, particularly the vehicle or the sensor that detects the set. For each digital signal from the set, a confidence criterion is determined, in particular the distance between the object and the reference point (314), The data-dependent model is validated if the classification of the object is appropriate in the digital signal and the confidence criteria of the digital signal are met, in particular the condition that the distance is within the reference distance to the reference point (328). A computer-implemented method characterized by the following features.

2. If the confidence criterion satisfies the conditions, particularly the condition that the distance is within the reference distance, and the classification is different from the reference classification, then the set of digital signals and the reference classification are stored in association with each other (346); otherwise, the digital signals are discarded and / or not stored. The method according to claim 1.

3. For the set, a value pair is obtained that includes a first value and a second value, wherein the first value represents the following distance, i.e., the distance within which the standard classification of the object is appropriate, and the second value represents the following distance, i.e., the distance within which the classification of the data-dependent model of the object is appropriate, or the interval between that distance and the standard distance. The method according to claim 1.

4. A memory location in memory is determined for the aforementioned value pair, and the value stored in the memory location is modified depending on the value of the value pair. The method according to claim 3.

5. The aforementioned data-dependent model is validated based on the location stored in the memory location. The method according to claim 4.

6. For multiple sets of digital signals, the classification and reference classification of the digital signals are required, and it is checked whether the classification of the data-dependent model relating to the object is appropriate. The method according to claim 1.

7. For each set from the aforementioned set, a value pair containing a first value and a second value for that set is determined, a memory location for the value pair determined for that set is determined, and the value stored in the memory location is modified depending on the value of the value pair. The method according to claim 6.

8. For each digital signal, one position is detected and / or stored, particularly by a satellite navigation system (108), and the distance is determined depending on the position. The method according to claim 1.

9. If the validation of the data-dependent model (206) fails, the data-dependent model (206) is newly trained, trained with other data, and / or another data-dependent model is used. The method according to claim 1.

10. If the validation of the data-dependent model (206) is successful, the data-dependent model (206) will be used in the object classification system, particularly in the driver assistance system. The method according to claim 1.

11. In a device (100) for validating a data-dependent model for classifying objects, The device comprises at least one processor (102) and at least one memory (104), and is configured to carry out the method described in any one of claims 1 to 10. A device (100) for validating a data-dependent model for classifying objects.

12. In computer programs, The computer program includes machine-readable instructions, and is characterized in that, when the machine-readable instructions are executed by a computer, the method described in any one of claims 1 to 10 is performed.

13. In storage media, particularly fixed storage media, A storage medium characterized in that the computer program described in claim 12 is stored in the storage medium.

Images