Method and device for providing classification result for object identification using ultrasonic-based sensor system in movement mechanism
Patent Information
- Application Number
- JP2023027973
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2022-02-28
- Filing Date
- 2023-02-27
- Publication Date
- 2026-02-27
AI Technical Summary
Existing ultrasonic sensor systems for vehicles require extensive resource consumption and high costs due to the need for training classification models using a large number of signal features from ultrasound received signals, which are highly dependent on acquisition situations, leading to inefficient classification of surrounding objects.
The method employs multiple classification models trained on different subsets of signal features based on specific acquisition situations, such as the position and movement of surrounding objects relative to the vehicle, to reduce resource consumption and improve classification efficiency.
This approach significantly reduces the cost of creating training databases and enhances the reliability of object classification by using the most relevant signal features for each situation, thereby improving the accuracy and efficiency of object identification.
Smart Images
Figure 00000000_0000_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to an ultrasonic sensor system for a moving mechanism such as an automobile, and more particularly to an efficient classification of surrounding objects related to collisions.
Background Art
[0002] Vehicles generally include an ultrasonic sensor system for object detection. This ultrasonic sensor system often has a plurality of ultrasonic sensor mechanisms each having a plurality of ultrasonic transducers for various capture ranges in which surrounding objects are to be detected. The evaluation of the ultrasonic reception signal is usually performed to recognize the position of the surrounding object. For this purpose, a dedicated classification model is used to distinguish between surrounding objects related to collisions and surrounding objects that can be traveled over. Classical approaches analyze the signal strength of low objects from the perspective of threshold values, while more complex approaches attempt to develop decision trees or use machine learning techniques for such classification tasks.
[0003] The use of a data-based classification model for object identification of surrounding objects requires a large number of features derived from the ultrasonic reception signal. To train such a classification model, ultrasonic reception signals for scenario examples are recorded, classification features are determined, and a label indicating whether the surrounding object is an object that can be traveled over or an object related to a collision is assigned to the classification features. The training data thus determined is then used for training the data-based classification model. Possible classification models can include, for example, neural networks or random forest models.
Summary of the Invention
[0004] The present invention provides a method for object identification of surrounding objects of an ultrasonic-based sensor system in a moving mechanism according to claim 1, as well as corresponding apparatus and ultrasonic sensor systems according to other independent claims.
[0005] Further forms are presented in the dependent claims. A method for operating an ultrasonic sensor system equipped with an ultrasonic sensor mechanism and for determining the object characteristics of surrounding objects, wherein each classification model is provided for determining a classification result that presents modeled object characteristics of surrounding objects for one of a plurality of ranges of surrounding object capture conditions, each classification model for evaluation is trained on different subsets of the set of signal features extracted from the ultrasonic received signal of the ultrasonic sensor mechanism to provide the classification results and accompanying quality indicators of the corresponding classification model, and the steps are as follows: - A step of capturing the ultrasonic received signal using the ultrasonic transducer of the ultrasonic sensor mechanism. - A step to determine the set of signal features from the ultrasonic received signal. - A step of determining one or more classification results and associated quality indications by evaluating one or more classification models using a corresponding subset of the set of signal features, - A method is provided that includes the step of determining object properties depending on one or more classification results and accompanying quality indications.
[0006] This method involves further steps: - In particular, a step to determine the status of capturing surrounding objects by positioning using an ultrasonic sensor mechanism, - A step of selecting a classification model based on the established capture status at any given time. - A step to determine the classification result using the classification model at the time, - A step in which object characteristics are determined based on the classification results and accompanying quality indications. It may include.
[0007] To train a classification model using a data-driven classification method, ultrasonic data of relevant surrounding objects is required. These surrounding objects are captured using an ultrasonic sensor system, and the resulting ultrasonic received signals are evaluated. The evaluation of the ultrasonic received signals is performed by extracting signal features from the transition of the ultrasonic received signals and assigning the extracted signal features to the important object characteristics of the surrounding objects, specifically whether the surrounding objects are drivable or collision-related. This classification corresponds to labels in the training data, which is used to assign labels to vectors based on signal features and to train the classification model.
[0008] Signal characteristics may include quantities collected from the ultrasonic received signal, such as maximum amplitude, the time of maximum amplitude, the phase position between two received signals, and further attributes, such as the amplitude measured at any given moment, the object spread, the detection rate, the average number of received signal peaks / transmitted impulses (multiple reflectivity), the number of ultrasonic transducers that can receive surrounding objects, the signal stability while the moving mechanism is in motion, the quotient of the distance traveled and the measured object spacing, the number of intersections found of the echo circles of adjacent ultrasonic transducers, and so on.
[0009] The ultrasonic received signal is highly dependent on the acquisition conditions, which are determined by the mounting position of the ultrasonic transducer, the geometric positions of surrounding objects, the relative velocities of surrounding objects, and / or ambient conditions such as temperature or ground conditions. To adequately consider these factors in a classification model requires numerous signal features, and in this regard, depending on the acquisition conditions, different subsets of signal features are of particular importance for the classification result, while the remaining signal features have only secondary significance. A single classification model trained on all possible signal features would have a high order of magnitude and therefore high resource consumption in subsequent applications. The cost of creating such a classification model would also be high, as the data bank consisting of training data would have to consider all combinations of influences as space-fillingly as possible.
[0010] Therefore, according to the method described above, various classification models are trained and applied depending on the capture situation. This capture situation may include, for example, the position of the surrounding object to be captured, the speed of the moving mechanism (vehicle), and similar factors. In this case, depending on the capture situation, different subsets of signal features may be used to train the corresponding classification model. This makes it possible to mask signal features that are not important or only slightly important for object identification in the given capture situation, and thus a classification model to be trained and provided is provided for the significantly reduced dimensionality of the signal features. For example, in the case of an ultrasonic sensor system for a vehicle equipped with 12 ultrasonic transducers, it is meaningful to divide the capture range as part of the capture situation into, for example, four capture situations. These capture situations correspond to relative movement in the medium to far range in front of the bumper, movement in the near range in front of the bumper, while stationary, and movement in the lateral range of the vehicle.
[0011] For acquisition scenarios involving movement within a medium to long range, surrounding objects can be detected by all ultrasonic transducers, and ultrasonic received signals can be collected at different distances. In addition, reflections from the ground, such as from moldings between surrounding objects and the ground, can be detected. This acquisition scenario can be defined, in particular, by all ranges where surrounding objects are spaced at intervals greater than a predetermined interval value.
[0012] Furthermore, the capture condition can be defined as movement within a range near the front of the vehicle's bumper (as an example of a moving mechanism). Within this range, information near the ground is lost, which has a significant impact on the signal characteristics because the multiple reflectivity of the surrounding objects is low. Therefore, low-reflectivity objects can no longer be detected based on signal characteristics related to multiple reflectivity. This capture condition can be defined, in particular, by all ranges where the surrounding objects have intervals smaller than a predetermined interval value.
[0013] Further detection conditions exist when the vehicle is stationary, i.e., when relative movement between the vehicle and surrounding objects has ceased. All signal features based on changes in the ultrasonic received signal due to multiple distances or during movement are unusable in these detection conditions.
[0014] Further capture conditions exist when surrounding objects are to the side of the vehicle. Here, surrounding objects are only perceived by a portion of the ultrasonic transducer, but instead are perceived from various detection angles. In addition, the detection point on surrounding objects may change as the vehicle passes by. The distance to surrounding objects remains constant or changes only in conjunction with the detection angle and cannot be used as a signal feature on its own. This makes it impossible to use all signal features that must be evaluated at various distances. This capture condition can be defined, for example, by determining all ranges that are only within one range of the detection range of a single ultrasonic transducer. In the simplest case, for evaluation, the capture condition can be recognized based, among other things, on location and speed recognition, and a classification model for attribution can be selected based on this.
[0015] Generally, for a classification model, a quality indication is determined in addition to the classification result. The quality indication is determined from the height of the argmax element value of the classification vector, that is, from the value of the highest-value element in the classification vector.
[0016] To obtain appropriate classification results and associated quality indicators, each classification model may be evaluated by each subset of the set of signal features, and the object properties are determined by the classification result in which the quality indicator shows the highest quality.
[0017] When all classification models are used to determine the classification result during the operation of an ultrasonic sensor system, the quality indicators of all classification models are available, and therefore the classification result with the highest quality indicator, i.e., the quality indicator indicating the highest reliability of the classification result, is always used.
[0018] When it is confirmed that one surrounding object leaves a capture situation, the last determined classification result and associated quality indicators by the classification model assigned to that capture situation may be temporarily stored, and the determination of the object properties depends on the classification result and associated quality indicators obtained by the classification model at that time, as well as the temporarily stored classification result and associated quality indicators.
[0019] In particular, as long as the quality indicator attached to the temporarily stored classification result indicates a higher quality than the quality indicator of the classification result determined by the classification model at that time, object properties may be determined by the temporarily stored classification result.
[0020] In an alternative embodiment, under one capture situation, only the classification model assigned to the corresponding capture situation may be computed. Each classification result is used and stored along with its associated quality indication. Upon moving to a further capture situation, only the classification model assigned to the corresponding capture range is used, and the classification result and corresponding quality indication can be determined there. However, as long as the quality indicated by the quality indication for the current evaluation of the classification model does not exceed the quality indicated by the quality indication for the previous evaluation, the classification result previously determined by the previous classification model is used as the important classification result along with its corresponding quality indication.
[0021] Furthermore, when it is confirmed that one surrounding object leaves one capture situation, the signal characteristics based on the historical transition of the ultrasonic received signal can be reset. When transitioning from one detection state to another, signal features are generally calculated only for the classification model at that time, and are reset when moving to a new classification model. Therefore, signal features based on historical ultrasonic received signals are recalculated. Signal features not used in the current classification model are maintained as invalid values, which essentially affects the quality indication of the classification model.
[0022] By using multiple classification models, the creation cost of the training data bank can be significantly reduced. Based on further aspects, an apparatus for implementing one of the above methods is provided.
[0023] Hereinafter, embodiments will be described in more detail based on the accompanying drawings.
Brief Description of the Drawings
[0024] [Figure 1] It is a schematic diagram of a vehicle equipped with an ultrasonic sensor system including an ultrasonic sensor mechanism. [Figure 2] It is a flowchart for illustrating the operation method of the ultrasonic sensor system of FIG. 1.
Mode for Carrying Out the Invention
[0025] FIG. 1 shows a schematic diagram of a vehicle 1 within the vehicle's surroundings where one or more surrounding objects U exist. The vehicle 1, as an example of a moving mechanism, includes an ultrasonic sensor system 2 having an ultrasonic sensor mechanism 3 disposed on the front bumper 4.
[0026] The ultrasonic sensor mechanism 3 includes a plurality of ultrasonic transducers 5 for transmitting ultrasonic signals by signal impulses and receiving ultrasonic signals reflected by surrounding objects U within the capture range E.
[0027] A control unit 6 is provided for evaluating the ultrasonic reception signals (sensor signals) of the ultrasonic transducers 5 of the ultrasonic sensor mechanism 3. In the control unit 6, a plurality of classification models 61a, 61b, 61c, 61d are implemented, which are separately trained for various capture situations with respect to the ultrasonic sensor mechanism 3.
[0028] The ultrasonic received signal is further evaluated in the control unit 6 in a known manner using an ultrasonic-based localization method to create a virtual map of the surroundings and plot the positions of the detected surrounding objects U therein. Each detected surrounding object U is assigned a classification result using the respective classification models 61a, 61b, 61c, and 61d. The classification result classifies the surrounding object U based on important characteristics for vehicle travel. For example, the classification result may indicate that the surrounding object U is traversable and related to collisions. This characteristic corresponds to the height of the surrounding object from the ground.
[0029] The data-based classification models 61a, 61b, 61c, and 61d assign one classification vector to one input quantity vector containing signal features from the ultrasonic received signal of the ultrasonic transducer. This classification vector contains one element of the classification result for each possible class. The argmax function can output a specific class as the classification result for model evaluation. In this case, the value of the element determined by argmax corresponds to a quality indicator of the classification result.
[0030] To classify recognized surrounding objects, according to the present invention, multiple classification models 61a, 61b, 61c, and 61d are used for different capture situations. For this purpose, for each surrounding object U, one capture situation is determined, and the classification model to be used can be selected accordingly.
[0031] The capture conditions may include the relative position of the surrounding object with respect to the ultrasonic sensor system 2 and, optionally, the relative velocity of the surrounding object U with respect to the ultrasonic sensor system. Further factors may include, for example, surface properties, wetting, and similar factors.
[0032] In the following, the relative position and relative velocity of the surrounding object U are used as the capture conditions. In the exemplary embodiment shown, there are four capture conditions for one surrounding object U: - A moving surrounding object U is located within a medium to far range M in front of it, i.e., a surrounding object having a relative position greater than D from the ultrasonic sensor system. - The moving surrounding object U is within a nearby range N from the ultrasonic sensor system, where the distance D between the surrounding objects is less than a predetermined distance. - It is specified that a moving surrounding object U is within one of two lateral ranges S positioned alongside the travel path in the direction of travel, thereby not all ultrasonic transducers can receive ultrasonic signals from the corresponding surrounding object U, and - The vehicle is stationary, or there is no relative movement between the surrounding object U and the ultrasonic sensor system 2. They are distinguished.
[0033] Figure 2 shows a flowchart illustrating the process performed in the control unit 6 of the ultrasonic sensor system 2. In step S1, the relative position and relative velocity of the surrounding object U with respect to the ultrasonic sensor system 2 are first captured using a position determination model.
[0034] In step S2, for each captured surrounding object U, one corresponding classification model is selected from classification models 61a, 61b, 61c, and 61d to determine whether the corresponding surrounding object U is related to a collision or can be driven over. An assigned classification model 61a, 61b, 61c, or 61d is selected, and in this classification model, the capture status of the surrounding object U is assigned to the corresponding classification model 61a, 61b, 61c, or 61d.
[0035] The selected classification model is used in step S3 to select a corresponding classification result, namely, an assertion that the surrounding object is related to collision or is traversable. To this end, the classification model utilizes and evaluates the signal features of the ultrasonic received signal from the ultrasonic transducer. The signal features can be quantities collected from the ultrasonic received signal, for example, characterizing the temporal changes in the ultrasonic received signal.
[0036] Each classification model is trained in a corresponding manner to evaluate itself using a subset of signal features determined from the ultrasonic received signal in order to obtain classification results.
[0037] Furthermore, in step S4, not only the classification result but also the quality indication is determined, which can be read, for example, from the degree of attribute of the classification result to a particular class. That is, in various exemplary embodiments, the surrounding object U can be characterized as collision-related or traversable, which may correspond to two different classes. In other words, the corresponding classification vector shows an output value between 0 and 1 for each class, where 1 corresponds to a certain estimation of attribute to one class, and 0 corresponds to the maximum possible uncertainty of attribute to this class.
[0038] Step S5 checks whether the capture status has changed and whether there are any capture statuses to which a different classification model has been assigned. If so (option: yes), this method continues in step S6; otherwise, it returns to step S1.
[0039] In step S6, the evaluation is performed in accordance with the newly assigned classification model for the changed capture range. Prior to this, the last classification result performed by the previous classification model is temporarily stored, along with the quality indicators associated with this classification result.
[0040] In step S7, the quality indicator of the new classification result is compared with the temporarily stored quality indicator. If the current evaluation by the new classification model results in a classification result with a lower quality indicator (option: yes), in step S8, the temporarily stored classification result is used as the final classification result. Otherwise (option: no), that is, if the quality indicator of the evaluation in the new classification model is higher than the temporarily stored quality indicator, in step S9, the newly finalized classification result is used as the final classification result.
[0041] When a capture status transitions to one assigned to a different classification model, signal features are generally calculated only for that specific classification model. The underlying historical progression of the ultrasonic received signal is reset when transitioning to a further classification model, and therefore the signal features based on the historical ultrasonic received signal are recalculated. Signal features not used in the current classification model are retained as invalid values, which essentially affects the quality indication of that classification model.
[0042] In an alternative embodiment, classification models for all capture situations may be evaluated in parallel during operation. In this case, the classification result that yields the best quality indication is used. This allows for greater reliability of the classification model that shows higher reliability (quality indication) during evaluation, particularly within the boundary range between two capture situations. [Explanation of symbols]
[0043] 1 vehicle 2. Ultrasonic Sensor System 3. Ultrasonic sensor mechanism 4. Front bumper 5. Ultrasonic transducer 6. Control Unit 61a, 61b, 61c, 61d classification models D predetermined interval E Capture range M: Medium to far range N Nearby range S horizontal range U surrounding objects
Claims
1. A method for operating an ultrasonic sensor system (2) with an ultrasonic sensor mechanism (3) and for determining object properties of a surrounding object (U), wherein a classification model (61 a, 61 b, 61 c, 61 d) is provided for determining a classification result representing a modeled object property of the surrounding object (U) for one of a plurality of acquisition situations of the surrounding object (U), and each classification model (61 a, 61 b, 61 c, 61 d) for evaluation is trained with a different subset of a set of signal features extracted from the ultrasonic received signal of the ultrasonic sensor mechanism (3) in order to provide a classification result of the corresponding classification model (61 a, 61 b, 61 c, 61 d) and an associated quality indication, the method comprising the following steps: a step (S1) of capturing said ultrasonic reception signal by the ultrasonic transducer (5) of said ultrasonic sensor mechanism (3); determining said set of signal features from said received ultrasound signals; - determining (S3) one or more classification results and associated quality indications by one or more evaluations of the classification models (61a, 61b, 61c, 61d) using the corresponding subsets of the set of signal features depending on the acquisition situation, - determining (S4) said object characteristics depending on said classification result(s) and an associated quality indication; A method comprising:
2. Further steps: - determining the capture of said surrounding object (U) by localization, in particular using said ultrasonic sensor arrangement (3); a step (S2) of selecting a classification model from said plurality of classification models (61a, 61b, 61c, 61d) at any given time depending on said determined capture situation, determining a classification result using said selected classification model at that time, - determining (S4) said object characteristics depending on said classification results and an attached quality indication; The method of claim 1 , comprising:
3. 3. The method of claim 2, wherein when it is determined that a surrounding object (U) is leaving a capture situation, the classification result and the associated quality indication last determined by the classification model assigned to the capture situation are temporarily saved, and the determination of the object characteristics is determined depending on the classification result and the associated quality indication obtained by the classification model at that time and the temporarily saved classification result and the associated quality indication.
4. 4. The method of claim 3, wherein the object characteristics are determined by the temporarily stored classification result while an accompanying quality indication of the temporarily stored classification result indicates a higher quality than the quality indication of the classification result determined by the current classification model.
5. The method of claim 3 , wherein the signal characteristics based on the historical progression of the ultrasonic reception signal are reset when it is confirmed that a surrounding object leaves a capture situation.
6. The method according to claim 2 , wherein the capture situation depends on the position of the surrounding object (U) and / or the velocity of the surrounding object (U).
7. 2. The method of claim 1, wherein each of the classification models is evaluated with a respective subset of the set of signal features to obtain the corresponding classification result and the associated quality indication, and the object characteristic is determined by the classification result for which the quality indication indicates the highest quality.
8. Apparatus configured to carry out one of the methods according to any one of claims 1 to 7.
9. A computer program for causing a computer to execute the method described in any one of claims 1 to 7.
10. A computer-readable recording medium having recorded thereon a computer program for causing a computer to execute the method according to any one of claims 1 to 7.