Method and apparatus for providing training data records for classification model for object recognition of ultrasonic sensor system in mobile device
By selecting the closest training data set from different configurations of multiple ultrasonic sensor systems and training classification models, the problems of poor applicability and complex acquisition of training data records in the prior art are solved, and the accuracy and efficiency of object recognition are improved.
Patent Information
- Application Number
- CN202380044098.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2022-06-01
- Filing Date
- 2023-04-24
- Publication Date
- 2025-01-10
AI Technical Summary
The classification model training data recording of existing ultrasonic sensor systems is difficult to apply to systems with different configurations, and the acquisition of new training data records is complex and time-consuming, resulting in low object recognition accuracy.
By collecting training data records from different configurations of multiple ultrasonic sensor systems, selecting the data set closest to the new configuration, training the classification model based on these data, and using verification data records to optimize model accuracy, reducing measurement workload.
The classification model suitable for the new ultrasonic sensor system was effectively trained, which improved the accuracy and efficiency of object recognition and reduced the collection workload of training data records.
Smart Images
Figure CN119301478A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to an ultrasonic sensor system for a mobile device, such as a motor vehicle, and in particular to a method for providing training data records for training a classification model for classifying objects based on input data records, which correspond to ultrasonic sensor data and / or are derived from these ultrasonic sensor data. Background Art
[0002] Vehicles are often equipped with ultrasonic sensor systems for object detection. These ultrasonic sensor systems often have a plurality of ultrasonic transducers for different detection areas in which environmental objects are to be detected. Such ultrasonic sensor systems are often arranged on the front and / or rear bumper of the vehicle.
[0003] In addition to locating environmental objects by evaluating the sensor data of the ultrasonic transducers, the sensor data can also be used to classify the environmental objects. In particular, a highly differentiated classification of environmental objects is to be achieved, in particular to determine whether the environmental objects are likely or unlikely to be driven over by the mobile device, i.e., whether they are relevant for a collision.
[0004] The classification of environmental objects is usually carried out with the aid of classification models. These classification models can be designed to be data-based, ie the classification models can be trained with the aid of machine learning methods with the aid of training data records.
[0005] Ultrasonic sensor systems of different vehicle types may differ in their configuration, i.e., in the number, arrangement and type of ultrasonic transducers used, so that a training data record for a classification model of an ultrasonic sensor system cannot be easily used to train a classification model in an ultrasonic sensor system having a different configuration. However, the acquisition of training data records for a new configuration of an ultrasonic sensor system is complex and requires a considerable amount of time before the ultrasonic sensor system is put into operation. Summary of the invention
[0006] According to the invention, a method for providing a training data record for training a classification model for a new configuration of an ultrasonic sensor system of a mobile device, as well as a device and a classification method for object recognition by means of an ultrasonic sensor system in a mobile device are provided.
[0007] Further refinements are described in the dependent claims.
[0008] According to a first aspect, a method, in particular a computer-implemented method, is provided for training a data-based classification model for the configuration of a new ultrasonic sensor system having a plurality of ultrasonic transducers using a training data record, wherein the classification model indicates at least one class of object properties for one or more environmental objects, wherein the training data record assigns input data records from the sensor data of the ultrasonic transducers and / or from sensor data features derived therefrom to classification vectors for the one or more environmental objects, the method comprising the following steps:
[0009] - providing training data sets for a plurality of configurations of the ultrasonic sensor system, wherein the training data sets each comprise one or more training data records for a respective measuring situation;
[0010] - selecting those configurations of the ultrasonic sensor system which are closest to the configuration of the new ultrasonic sensor system at least in terms of configuration features;
[0011] - selecting a training data record and a validation data record from a training data set assigned to the selected configuration of the ultrasonic sensor system;
[0012] - Based on the selected training data records and validation data records, the classification model is trained for a configuration of a new ultrasonic sensor system.
[0013] When using data-based classification models for object recognition with ultrasonic sensor systems, it is difficult to obtain a sufficient number of training data records. Usually, the training data records are determined by recording sensor signals for various environmental situations. To this end, these environmental situations must be created and measurements performed with the aid of the ultrasonic sensor system for which the classification model is to be created. The data-based classification model can include neural networks, probabilistic regression models, decision trees trained by gradient boosting algorithms, and the like.
[0014] For example, each of the measurement situations provided for creating a training data record can be determined by one or more environmental objects having the same or different object properties (such as height, width, etc.) and having the same or different distances and / or orientations relative to the ultrasonic sensor system.
[0015] The training data records respectively determined from different measurement situations respectively provide input data records, which include sensor data and / or features aggregated from these sensor data, such as the maximum signal amplitude, a histogram of the number of runners detected relative to the acquired environmental objects, the proportion of point / line / single echo detections of environmental objects, the proportion of detections with specific signal patterns with respect to amplitude, multiple reflections, etc. (for example, signal patterns indicating pedestrians), the average number of ultrasonic transducers acquiring specific environmental objects.
[0016] The environmental objects to be classified are classified accordingly and a classification vector is provided as a label, which for one or more environmental objects respectively comprises a plurality of object classes, which respectively indicate the presence (or the model reliability of this presence) of a specific predefined object property. The classification vector forms a label for the corresponding training data record assigned to the measurement situation.
[0017] For a specific measurement situation, the input data record of the training data record depends largely on the configuration of the ultrasonic sensor system under consideration. The configuration of the ultrasonic sensor system results from configuration features that indicate, among other things, the number, arrangement and type of ultrasonic transducers used. In particular, the arrangement can be determined by features of the sensor installation height, the vertical installation angle of the ultrasonic transducers, the offset of the respective installation height of the ultrasonic transducers and the horizontal installation angle of the ultrasonic transducers.
[0018] The method described above now provides that a training data record for training a corresponding classification model is provided for a new configuration of the ultrasonic sensor system by using training data records that have already been acquired for other configurations of the ultrasonic sensor system, even if these training data records were determined for different configurations of the ultrasonic sensor system.
[0019] The method is based on training data sets acquired for a plurality of different configurations of the ultrasonic sensor system. Each training data set is assigned to a configuration and comprises training data records for different measurement situations with the relevant configuration of the ultrasonic sensor system.
[0020] It can be provided that the configuration feature is determined by one or more arrangement features and one or more sensor type features, wherein the configuration of the ultrasonic sensor system is selected by selecting a plurality of most closely adjacent configuration feature points which extend a region which contains the configuration feature points of the configuration of the new ultrasonic sensor system.
[0021] Regarding new configurations of ultrasonic sensor systems, the closest configuration of ultrasonic sensor systems that is already equipped with a training data set is first selected, i.e., the most similar configurations of these ultrasonic sensor systems are selected. This selection is made based on configuration features, which may indicate the arrangement of ultrasonic transducers characterized by the arrangement feature and the number of ultrasonic transducers in the relevant ultrasonic sensor system and the sensor type determined by the sensor type feature.
[0022] These arrangement features may include, for example, the installation height of the ultrasonic transducer from the ground, the vertical installation angle of the ultrasonic transducer, the offset of the installation heights between the ultrasonic transducers, and the horizontal installation angle of the ultrasonic transducer.
[0023] In particular, as configurations of the ultrasonic sensor system, those configurations of the ultrasonic sensor system that are most closely adjacent in terms of the configuration feature are selected, which extend the smallest possible area in which the new configuration of the ultrasonic sensor system is located. In particular, the minimum number of the selected most closely adjacent configurations of the ultrasonic sensor system is the number of the considered configuration features increased by one.
[0024] With respect to configuration features, the closest configuration may be determined as the angular difference of the vectors in the feature space determined by these configuration features.
[0025] Alternatively, the closest configuration can be determined as the Euclidean distance between points in the feature space determined by the configuration features or as the Euclidean distance about the normalized configuration features. In addition, the configuration features can also be weighted so that the configuration features determined when determining the distance are given priority.
[0026] For each of the selected configurations, a training set of training data records is provided, which training sets each include training data records for different measurement situations.
[0027] From all training data sets for the selected configuration, training data records and validation data records are selected for a specific measurement situation or for similar measurement situations. Similar measurement situations can be characterized, for example, based on object features such as the distance to the ultrasonic sensor device and the height of environmental objects, wherein the similarity between the considered configurations can be described by being below a specified maximum distance (Euclidean distance) with respect to the object features. In this way, a series of training data records and validation data records are obtained for different measurement situations, with which a new classification model can be trained for a new configuration of the ultrasonic sensor system.
[0028] Furthermore, the classification model can be retrained with further training data records for the configuration of new ultrasonic sensor systems.
[0029] It can be provided that, if the accuracy of the model determined with the aid of a validation data record assigned to a specific measurement situation or a measurement situation similar to the specific measurement situation falls below a specified threshold, a further training data record for the specific measurement situation is determined by measurement.
[0030] If, for a measurement case, the evaluation with the validation data record assigned to the corresponding measurement case yields a model accuracy that is too low, for example the rate of correct classifications falls below a specified threshold rate of, for example, 95%, provision can be made for a training data record to be determined in addition thereto in a conventional manner for a new configuration of the ultrasonic sensor system for the corresponding measurement case. However, since it is expected that the model accuracy will be sufficient for at least some of these measurement cases, it is possible to dispense with the determination of training data for the measurement case and to significantly reduce the overall effort for determining the training data record.
[0031] In particular, the measurement situation can be determined by measurement features, wherein similar measurement situations are determined in such a way that the distance between a measurement feature point determined by one or more measurement features of a measurement situation and a measurement feature point of a specific measurement situation does not exceed a specified maximum distance, wherein in particular measurement features are determined which characterize the relative position of environmental objects of the measurement situation and one or more object properties.
[0032] According to one embodiment, the configuration of the ultrasonic sensor system can be selected for different measurement situations by selecting a plurality of most closely adjacent configuration feature points, which are determined, in addition to one or more layout features and one or more sensor type features, by one or more measurement features (in particular, the relative position of one or more environmental objects relative to the new ultrasonic sensor system) and / or object properties (in particular, the height), so that the configuration feature points extend to an area that contains the configuration feature points of the configuration of the new ultrasonic sensor system including one or more measurement features.
[0033] In this embodiment, a separate number of closest configurations of the ultrasonic sensor system can be selected for each measurement case, wherein the selection of the closest adjacent configurations also takes into account at least one measurement feature, such as the height of an environmental object and the distance to the ultrasonic sensor system, in addition to the configuration features. In this way, different closest configurations of the ultrasonic sensor system can be derived for each measurement case, from which the assigned training data record and validation data record are selected for the relevant measurement case. BRIEF DESCRIPTION OF THE DRAWINGS
[0034] Subsequently, the embodiments are described in more detail with reference to the accompanying drawings, wherein:
[0035] Figure 1 A schematic diagram of a vehicle having an ultrasonic sensor system with an arrangement of ultrasonic transducers is shown;
[0036] Figure 2 A flow chart illustrating a method for determining a training data record for training a classification model for object recognition in an ultrasonic sensor system is shown; and
[0037] Figure 3 Shown are diagrams illustrating different designs of ultrasonic sensor systems with respect to two exemplary design features as a basis for selecting a training data record for a new design of an ultrasonic sensor system. DETAILED DESCRIPTION
[0038] Figure 1 A schematic diagram of a vehicle 1 in a vehicle environment in which one or more environmental objects U may be present is shown. The vehicle 1 as an example of a mobile device comprises an ultrasonic sensor system 2, which is arranged on a bumper 4. The ultrasonic sensor system 2 comprises a plurality of ultrasonic transducers 5 for transmitting ultrasonic signals having signal pulses and for receiving ultrasonic signals reflected on environmental objects U. The arrangement of the ultrasonic transducers 5 and the type of the ultrasonic transducers 5 determine the configuration of the ultrasonic sensor system 2. Different types of vehicles 1 have different ultrasonic sensor systems 2 with different configurations.
[0039] A control unit 6 is provided for evaluating the sensor signals of the ultrasonic transducers 5 of the ultrasonic sensor system 2. In the control unit 6, in addition to a localization model for localizing environmental objects relative to the vehicle 1, a data-based classification model 61 is implemented. The data-based classification model can include a neural network, a probabilistic regression model, a data-based decision tree trained by a gradient boosting algorithm, etc. For this purpose, the sensor signals of the ultrasonic transducers 5 are evaluated in a manner known per se by means of an ultrasound-based localization method in order to create a virtual map of the environment in the control unit 6 and to enter the positions of the detected environmental objects U there.
[0040] The classification model 61 is trained according to the configuration of the ultrasonic sensor system 2. The classification model is trained to classify the environmental objects U with respect to the object characteristics (especially the height of the environmental objects), mainly for distinguishing whether the relevant environmental objects can be driven over by the vehicle 1, that is, whether they are relevant to a collision. For this purpose, the classification model 61 is trained to determine a classification result for each environmental object U, which assigns object characteristics to each environmental object U identified in the environment.
[0041] The detected surrounding objects are assigned classification results by means of the classification model 61. These classification results classify the surrounding objects U according to corresponding relevant object properties, in particular according to a height class.
[0042] The data-based classification model 61 assigns a classification vector to an input data record, which may include a signal time series of a sensor signal of an ultrasonic transducer and / or signal features derived or aggregated therefrom. The classification vector includes elements that quantify the possible categories of the classification result for each identified environmental object. Here, the value of the element indicates the probability that the object characteristic assigned to the category is realized by the environmental object U associated with the category. Through the argmax function of the elements of the classification vector that are respectively assigned to the environmental object, a specific category can be output for model evaluation as a classification result. Here, the value of the element determined by argmax corresponds to the classification confidence.
[0043] When creating a data-based classification model 61 for a new configuration of an ultrasonic sensor system, training data records have usually had to be determined in a complex manner by measuring measurement situations. To this end, measurement situations are simulated and corresponding signal time series of sensor signals are recorded, from which input data records are generated and assigned to classification vectors, which indicate to each of the environmental objects provided in the measurement situation the relevant object properties of the environmental object, usually in the form of a one-hot coded vector. In order to reduce the measurement effort, the training data records are then combined with the training data. Figure 2 , describing a method which enables training data records to be derived from other configurations of the ultrasonic sensor system which have already been measured and to be used for training the classification model for a new configuration of the ultrasonic sensor system.
[0044] Figure 2 The method can be implemented as software or hardware in a conventional data processing device and in particular be carried out offline, ie outside the vehicle in which the classification model 61 is to be used.
[0045] In step S1, a database is first provided in which a training data set with training data records for respectively different measurement situations is present for a plurality of configurations of the ultrasonic sensor system 2. The measurement situations are usually determined by measurement features which indicate the relative position of one or more environmental objects with respect to the ultrasonic sensor system 2 or the distance from the ultrasonic sensor system and the orientation associated therewith as well as properties of the environmental objects U, such as the height of one or more environmental objects U.
[0046] Different configurations of the ultrasonic sensor system 2 are determined by characteristic configuration features, which may include, for example, layout features and sensor type features. These layout features may include, for example, the installation height of the ultrasonic transducer, the vertical installation angle of the ultrasonic transducer, the offset of the installation height between the ultrasonic transducers, the horizontal installation angle of the ultrasonic transducer, and the number of ultrasonic transducers. Considering the sensor type in the sensor type feature may be particularly important because the ultrasonic transducers 5 may have different sensitivities and reception characteristics.
[0047] In step S2 , corresponding configuration features of the new configuration of ultrasonic sensor system 2 are determined.
[0048] In step S3, the number of the closest configurations of the ultrasonic sensor system 2 is determined, wherein the number corresponds to at least the number of dimensions of the configuration feature space increased by 1. Figure 3 As schematically shown in FIG. 1 , for two design features M1, M2, such as the installation height and the vertical installation angle as design features, a plurality of configuration feature points K of the ultrasonic sensor system 2 already measured in the database and a new configuration feature point N for the configuration of the ultrasonic sensor system 2 for which a training data record is to be provided are derived. The closest configuration feature points K of the ultrasonic sensor system 2 already measured are determined such that these configuration feature points define an area in the configuration feature space, within which the new configuration feature point N is located.
[0049] In step S4, training data records are now selected for a specific measurement situation or measurement situations similar thereto, which are assigned to the selected configuration of the ultrasonic sensor system 2, in order to select training data records and validation data records therefrom, in particular by random selection (e.g. 80% training data records and 20% validation data records), for subsequent model training of the classification model. This is performed for different measurement situations, in particular for all available measurement situations. Measurement situations similar to the specific measurement situation can be determined based on the angular distances of feature vectors determined by relevant measurement feature points or the Euclidean distances of relevant measurement feature points, which are each determined by one or more measurement features, for example all measurement feature points whose distance to the measurement feature points of the specific measurement situation does not exceed a specified maximum angle or maximum distance.
[0050] In step S5 , the model training is then carried out using the training data records determined in this way.
[0051] In step S6, with the aid of the validation data records, it can now be determined individually for each measurement case whether the trained classification model has sufficient accuracy. The accuracy can be determined in particular by the ratio of the accurate / correct classification of the corresponding environmental objects to the total number of checks (the number of validation data records considered for the measurement case).
[0052] If it is ascertained in step S7 that the proportion of correct classifications is below a specified accuracy threshold (option: yes), the trained classification model is not reliable for this specific measurement case and further training data records must be determined in step S8 for this specific measurement case or measurement cases similar thereto by means of conventional methods, in particular by measurements on a real system. The classification model can then be retrained in step S9 with the aid of these further training data records. If it is ascertained in step S7 that the proportion of correct classifications is above a specified accuracy threshold (option: no), the trained classification model is reliable for this specific measurement case and steps S8 and S9 are not carried out. Steps S7 to S9 are carried out for each measurement case.
[0053] The dependence on the configuration features is equally pronounced for all environmental objects, so that in some measurement cases, changing configuration features have no major influence on the classification result, while in other measurement cases, significant misclassifications can already occur with slight changes in the position of the environmental objects. The selection of the closest configuration of the ultrasonic sensor system can therefore also take into account one or more object properties of the environmental objects for each individual measurement case in addition to the design features and can be performed individually for each considered measurement case.
[0054] The number of selected closest configurations of the ultrasonic sensor system can also be determined as a function of the sensitivity of the classification result to the measurement situation.
[0055] Furthermore, the configuration feature can also take into account the loading state of the vehicle, wherein two or more loading states can be taken into account accordingly. The loading state influences the effective installation height and the effective vertical installation angle of the ultrasonic transducer, so that the loading state can be taken into account when selecting the corresponding closest configuration of the ultrasonic sensor system.
Claims
1. A method, in particular a computer-implemented method, for training a data-based classification model (61) for the configuration of a new ultrasonic sensor system (2) having a plurality of ultrasonic transducers (5) using a training data record, wherein: The classification model (61) indicates at least one class of object properties for one or more environmental objects (U), wherein a training data record assigns input data records from sensor data of the ultrasonic transducer (5) and / or from sensor data features derived therefrom to a classification vector for one or more environmental objects (U), the method comprising the following steps: - providing (S1) a training data set for a plurality of configurations of the ultrasonic sensor system (2), wherein the training data set comprises one or more training data records for each corresponding measurement situation; - selecting (S3) a configuration of the ultrasonic sensor system (2) which is closest, at least in terms of configuration features, to the configuration of the new ultrasonic sensor system (2); - selecting (S4) a training data record and a validation data record from the training data set assigned to the selected configuration of the ultrasonic sensor system (2); - Based on the selected training data records and validation data records, the classification model (61) is trained (S5) for the configuration of the new ultrasonic sensor system (2).
2. The method according to claim 1, wherein: The classification model (61) is retrained (S8) using further training data records for the configuration of the new ultrasonic sensor system (2).
3. The method according to claim 2, wherein: If the accuracy of the model determined with the aid of a validation data record assigned to a specific measurement situation or a measurement situation similar to the specific measurement situation is below a specified threshold value, a further training data record is determined for the specific measurement situation by measurement.
4. The method according to claim 3, wherein: The measuring situation is determined by measuring features, wherein similar measuring situations are determined in such a way that the distance between a measuring feature point determined by one or more measuring features of a measuring situation and a measuring feature point of the specific measuring situation does not exceed a specified maximum distance, wherein in particular measuring features are determined which characterize the relative position of environmental objects of the measuring situation and one or more object properties.
5. The method according to any one of claims 1 to 4, wherein: Configuration features are determined by one or more arrangement features and one or more sensor type features, wherein the configuration of the ultrasonic sensor system is selected by selecting a plurality of most closely adjacent configuration feature points which extend a region containing the configuration feature points of the configuration of the new ultrasonic sensor system (2).
6. The method according to claim 5, wherein: The configuration of an ultrasonic sensor system (2) is selected for different measurement situations by selecting a plurality of most closely adjacent configuration feature points (K), which are determined by one or more measurement features, in particular the relative position and / or object properties, in particular the height, of one or more environmental objects (U) relative to the new ultrasonic sensor system (2), in addition to the one or more arrangement features and the one or more sensor type features, so that the configuration feature points extend to a region that contains the configuration feature points (N) of the configuration of the new ultrasonic sensor system (2) including the one or more measurement features.
7. The method according to any one of claims 1 to 6, wherein: The classification model is designed as a decision tree based on the data.
8. An apparatus for performing the method according to any one of claims 1 to 7.
9. A computer program product comprising instructions which, when executed by at least one data processing device, cause the data processing device to perform the steps of the method according to any one of claims 1 to 7.
10. A machine-readable storage medium comprising instructions which, when executed by at least one data processing device, cause the data processing device to perform the steps of the method according to any one of claims 1 to 7.