Method for classifying object to be detected by means of at least one ultrasonic sensor taking into account phase information of detected signal
By extracting the phase information of the signal from the ultrasonic sensor and combining it with a neural network for classification, the problem of insufficient accuracy caused by relying solely on amplitude information in existing technologies is solved, achieving more efficient object classification and differentiation.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- ROBERT BOSCH GMBH
- Filing Date
- 2024-07-30
- Publication Date
- 2026-04-24
AI Technical Summary
Existing ultrasonic sensors only utilize amplitude information in object detection and classification, resulting in insufficient accuracy and robustness, and making it difficult to effectively distinguish object categories.
By processing the signals detected by the ultrasonic sensor, phase information is extracted and combined with a neural network for object classification, including time-frequency transformation of the signal, phase unwrapping and gradient calculation, forming a feature vector that is input into the neural network.
The accuracy and robustness of ultrasonic sensors in object classification have been improved, enabling them to better distinguish object categories and support complex and safety-related driving functions.
Smart Images
Figure CN121925571A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to a method for classifying objects to be detected using at least one ultrasonic sensor, taking into account the phase information of the detected signal. Background Technology
[0002] Ultrasonic sensors are used in automotive and industrial applications for distance measurement, such as in parking assistance systems in vehicles, or more generally for detecting objects in the environment.
[0003] Ultrasonic sensors operate based on the known pulse-echo principle. An electrical signal excites a transducer to vibrate its diaphragm, and this vibration is radiated as sound. This sound travels through the air until it encounters an object in the environment. The object's surface reflects the sound, which occurs as backscattering in the direction towards the ultrasonic sensor. If the backscattered sound encounters the diaphragm, it causes the diaphragm to vibrate, generating an electrical signal on the piezoelectric element.
[0004] Currently, ultrasonic sensors using existing technology measure the propagation time of sound from its emission to its return. Using the known speed of sound, the distance to the object that is backscattering the sound can be determined.
[0005] Although the backscattered sound pressure is a time-curved parameter in its physical properties, the time signal has not been fully utilized to date, primarily for cost reasons and for the sake of simple data detection, storage, and transmission. After some signal preprocessing steps (e.g., filtering), amplitude values and associated correlation values are formed separately using thresholding methods, employing simple or more complex and adaptive approaches. These correlation values representatively represent the maximum values in the sound pressure time signal. Thus, the echo of the object is detected by comparing the received amplitude of the sound with a threshold.
[0006] Typically, only echoes with amplitudes above a threshold are considered significant and relevant, and are further evaluated. Existing ultrasonic sensors transmit only these few amplitude or correlation values, often referred to as echo values. This can impair accuracy in distinguishing objects or determining object size.
[0007] Furthermore, it should be mentioned that the schemes known from the prior art for distinguishing object categories have so far only considered amplitude-based signal features. As an example of the corresponding prior art, the publication of Jona Eisele, André Gerlach, Marcus Maeder, and Steffen Marburg (2023) entitled "Convolutional neural network with dataaugmentation for object classification in automotive ultrasonic sensing" (Journal of the Acoustical Society of America, April 1, 2023; 153 (4): 2447, https: / / doi.org / 10.1121 / 10.0017922) is mentioned.
[0008] Therefore, the object of the present invention is to provide a solution by which the performance of ultrasonic sensors can be improved in an efficient and reliable manner, thereby improving the detection of objects in the vehicle's environment using ultrasonic sensors. Summary of the Invention
[0009] This task is solved by a method that uses at least one ultrasonic sensor to classify the object to be detected, as described in the independent claims.
[0010] According to the first aspect, this disclosure relates to a method for classifying at least one object to be detected using at least one ultrasonic sensor, comprising the following steps: In the first step, a first signal is sent to the object by the at least one ultrasonic sensor.
[0011] In the second step, a second signal is received by the at least one ultrasonic sensor, wherein the second signal represents a signal backscattered from the object.
[0012] In the third step, the second signal is processed into a first digital signal.
[0013] In the fourth step, the first digital signal is converted into a complex value signal representation based on a time-frequency transformation or modulation method to obtain the second digital signal.
[0014] In the fifth step, the second digital signal is separated into a first feature vector and / or at least into a second feature vector, wherein the first feature vector contains multiple amplitude values of the second digital signal, and the at least second feature vector contains multiple phase values of the second digital signal.
[0015] In the sixth step, the range of phase jump values of the second eigenvector is converted into a continuous curve (=Phasen-Unwrapping).
[0016] In the seventh step, the gradient of the phase curve in the time direction or the frequency direction is determined.
[0017] In the eighth step, a third feature vector is created based on the determined gradient.
[0018] In the ninth step, the at least third feature vector is fed into the neural network as the first input parameter.
[0019] In the tenth step, object category information of the at least one object is determined using the neural network, wherein the neural network generates an output parameter based on the at least first input parameter, the output parameter indicating the probability value of at least one defined object category of the object.
[0020] A fundamental idea of this invention is to utilize ultrasonic sensors not only to determine the distance between the sensor and the object, but also to use the sensor data detected by the ultrasonic sensors for object classification. During classification, the type or characteristic attributes of the object are determined from the backscattered ultrasonic signals.
[0021] Based on this classification, sensors can determine groups of objects, or so-called object categories, rather than just a single individual or a single representative of a detected object. Typical object categories could include: people, poles, trees, curbs, small objects, manhole covers, etc. Similarly, aggregated categories (e.g., drivable / impassable) can be classified.
[0022] Understanding categorization can be leveraged to generate numerous possibilities for more advanced driving functions. For example, distinguishing between static and mobile objects, or drivable and non-drivable objects, is highly helpful. From categorization knowledge, additional information supporting driving functions can be derived directly or indirectly, or in combination with other information or sensors. This information is particularly relevant and applicable to complex or safety-related driving functions, such as more advanced or fully automated driving.
[0023] The present invention improves the performance of ultrasonic sensors. This is achieved, in particular, by additionally considering phase information in the processing of sensor data, rather than employing only amplitude-based features as known to date in the prior art, for classification or to improve the robustness of classification. This results in a more robust object classification model produced in a more efficient manner.
[0024] Crucially for this invention, the phase, in particular, contains precise timing information, thereby enabling better temporal analysis of individual reflections superimposed within backscattering, specific to the object. The corresponding distance of each individual reflection reveals insights into the object's geometry. Especially in noisy signals, employing phase information leads to improved recognition rates.
[0025] For example, in the field of speech recognition, phase is crucial for improving robustness at low signal-to-noise ratios. Similarly, the phase of an echo contains material-dependent absorption characteristics. Furthermore, so-called circular waves (Kriechwellen) in object backscattering cause interference, which depends on the object's shape and perimeter. These circumferential waves are important for many typical objects in this application area of vehicle environment sensing, such as poles (traffic signs), trees, parking posts, and bicycle racks. The phase transitions generated by interference, and correspondingly object-specific, can be identified particularly well in the derivative of the phase.
[0026] In one configuration of this method, the step of processing the second signal into a first digital signal includes filtering the first digital signal. This improves the signal-to-noise ratio (S / R) of the first digital signal.
[0027] One configuration of the method involves a modulation method constructed as in-phase and quadrature (I&Q) modulation. This enables the efficient and application-relevant conversion of a first digital signal into a complex-valued signal representation, which is mapped by at least one one-dimensional eigenvector.
[0028] In one configuration of this method, the complex-valued signal representation is mapped to a time-frequency representation. This, in particular, enables frequency-related feature extraction from a two-dimensional feature vector.
[0029] One configuration of this method feeds the first feature vector as a second input parameter into the neural network. This further improves the object classification results.
[0030] According to a second aspect, this disclosure relates to a detection system that classifies objects to be detected using at least one ultrasonic sensor, the detection system being configured to perform the method according to the invention, wherein the at least one ultrasonic sensor is capable of use in a vehicle.
[0031] According to a third aspect, this disclosure relates to a computer program containing machine-readable instructions that, when executed on one or more computers and / or computing instances, cause the one or more computers or computing instances to perform a method according to the invention.
[0032] According to the fourth aspect, this disclosure relates to a machine-readable data carrier and / or downloadable product having the computer program.
[0033] According to the fifth aspect, this disclosure relates to one or more computer and / or computing instances having the computer program and / or having the machine-readable data carrier and / or downloadable product.
[0034] Other measures to improve the invention will now be described in more detail with reference to the accompanying drawings, in light of preferred embodiments of the invention. Attached Figure Description
[0035] The attached diagram shows: Figure 1 A schematic flowchart of a method 100 for classifying an object 10 to be detected using at least one ultrasonic sensor 50. Figure 2 A schematic diagram of a detection system 200 for performing method 100 according to an embodiment of the present invention, and Figure 3 A schematic diagram illustrating object classification using a neural network 9 in the case of using method 100 according to an embodiment of the present invention. Detailed Implementation
[0036] Figure 1 A schematic flowchart of a method 100 for classifying at least one object 10 to be detected using at least one ultrasonic sensor 50 is shown.
[0037] In step 102, the at least one ultrasonic sensor 50 sends a first signal 1 to the object 10.
[0038] In step 104, a second signal 2 is received by the at least one ultrasonic sensor 50, wherein the second signal 2 represents a signal backscattered from the object 10.
[0039] In step 106, the second signal 2 is processed into a first digital signal 3. Optionally, the step of processing the second signal 2 into the first digital signal 3 may include filtering the first digital signal 3 to improve the S / R ratio.
[0040] In step 108, the first digital signal 3 is converted into a complex value signal representation 20 based on a time-frequency transformation or modulation method to obtain the second digital signal 4. Optionally, the complex value signal representation is mapped to a time-frequency representation.
[0041] Optionally, the modulation method is configured as in-phase and quadrature (I&Q) modulation.
[0042] In step 110, the second digital signal 4 is separated into a first feature vector 5 and / or into at least one second feature vector 6, wherein the first feature vector 5 contains a plurality of amplitude values 21 of the second digital signal 4, and the at least one second feature vector 6 contains a plurality of phase values 23 of the second digital signal 4. Therefore, according to the present invention, it is also conceivable to separate or extract only the at least one second feature vector 6 from the second digital signal 4.
[0043] In step 112, the jump value range of phase 23 of the second feature vector 6 is transformed into a continuous curve 24.
[0044] In step 114, the gradient 8 of the curve of phase 24 is determined, preferably in the time direction or the frequency direction.
[0045] In step 116, a third feature vector 7 is created based on the determined gradient 8.
[0046] In step 118, the at least third feature vector 7 is fed into the neural network 9 as the first input parameter 11.
[0047] In step 120, when using the neural network 9, object category information 12 of the at least one object 10 is determined, wherein the neural network 9 generates an output parameter 15 based on at least a first input parameter 11, the output parameter indicating the probability value of at least one defined object category of the object 10.
[0048] Optionally, in addition to feeding the first input parameter 11 into the neural network 9, the first feature vector 5 is also fed into the neural network 9 as a second input parameter 12. This can improve the accuracy of the probability value to be obtained for at least one defined object category of the said object.
[0049] Figure 2 A schematic diagram of a detection system 200 for performing method 100 according to an embodiment of the present invention is shown. Here, the detection system 200 includes an ultrasonic sensor 50 that sends a first signal 1 to an object 10 to be detected and classified. The first signal 1 is reflected by the object 10 and arrives at the ultrasonic sensor 50 as a second signal 2, which is processed as described above according to method 100 of the present invention to classify the object 10.
[0050] In general, it should be noted in this context that the ultrasonic time signal (= original signal) backscattered and detected by the object 10 is used.
[0051] Figure 3This diagram illustrates object classification using a neural network 9 in the case of using method 100 according to an exemplary embodiment of the present invention. Here, a first input parameter 11 and / or (at least) a second input parameter 12 are fed to the neural network 9.
[0052] The two input parameters 11 and 12 are processed by the neural network 9 according to the method 100 of the present invention as explained above, such that the output parameter 15 is output by the neural network 9, which represents the object category information 12 of the detected object 10.
[0053] Therefore, it is conceivable to feed only one of these two input parameters 11 and 12 to the neural network 9. Thus, for example, it is possible to feed only the input parameter that contains at least the second feature vector 6 with the corresponding phase information of the second digital signal 4 to the neural network 9.
[0054] Here, the neural network 9 may be a component of the ultrasonic sensor 50, but it may also be located outside the ultrasonic sensor 50, for example, as part of a cloud-based or network-based connection, or as part of an external connection to a data processing device (such as a control device in a vehicle).
[0055] Other aspects and embodiments of the invention are described in detail below.
[0056] In summary, and as explained at the beginning, according to the present invention, the ultrasonic time signal (raw signal) backscattered and sensed by at least one object is processed, including its phase information.
[0057] Thus, a particularly robust classification of object categories is achieved by combining the following steps: a. Detect the raw time signal at the transducer element of the ultrasonic sensor. b. Convert the signal into a complex value using modulation methods or time-frequency representation. c. Extract amplitude and phase values separately. d. Unwrap (= the transformation shown in step 112 of this method) and differentiate the phase value in the time and / or frequency directions. e. Input the processed amplitude feature vector sequence and phase feature vector sequence into a neural network (NN) for classification. At the end of the method, the probability of being assigned to one of the object classes is provided as an output parameter of the neural network. This parameter achieves higher statistical reliability in assigning to an object class than earlier methods.
[0058] Regarding a) detecting the raw time signal at the transducer element of the ultrasonic sensor. The proposed method relies on the ability to detect and evaluate the raw time signal without the need for common processing steps such as forming an amplitude envelope or thresholding. As mentioned above, the proposed processing of the detected time signal can be performed using known methods.
[0059] Regarding b) converting it into a complex value signal representation The obtained time signal segment can be converted into a complex-valued time-frequency representation using time-frequency transforms such as the Short-Time Fourier Transform (STFT) or, particularly advantageously, the Continuous Wavelet Transform (CWT). Especially in CWT, scaling in the time direction is suitable for data reduction using linear interpolation. Comparable resolution in both the time and frequency directions should be achieved. Alternatively, it can be converted into a complex-valued signal representation using modulation methods (particularly advantageously, I&Q modulation).
[0060] Regarding c) Extracting amplitude and phase values separately From the representation of complex-valued signals, on the one hand, the amplitude is represented by modulo operation, and on the other hand, the phase is represented by the argument (arctan2) to form a real-valued eigenvector or sequence of eigenvectors (more precisely: at least a phase vector is formed, with optional additions to form an amplitude vector). The range of values for the amplitude representation is logarithmically compressed to highlight low-energy signal components.
[0061] Regarding d), unwinding (= the transformation shown in step 112 of this method) and differentiating the phase value in the time and frequency directions. Phase representation is difficult to interpret due to its range of values in the interval (-π, π]. Therefore, according to the invention, unwrapping is applied in the time direction. Here, 2π is added for each phase transition. In the phase signal thus processed, changes in the phase curve are considered to be important and relevant features. For this purpose, the phase derivative is formed in the eigenvectors of continuous phases in two different ways (or only one option can be used! Gradient determination in the time or frequency direction). This is done in the time direction to extract the so-called local instantaneous frequency, and in the frequency direction to extract the local group delay. A total of three real-valued eigenvector sequences are generated: one with respect to amplitude, and two with respect to the phase derivative in the time and frequency directions. However, alternatively, for example, only one of the phase eigenvector sequences can be calculated and used.
[0062] Regarding e), the processed feature vector sequence is input into a neural network (NN) for classification. At least one sequence of extracted feature vectors is arranged channel by channel and fed as tensors into a neural network for classification. Convolutional Neural Networks (CNNs) are particularly well-suited for this purpose.
Claims
1. A method (100) for classifying at least one object (10) to be detected using at least one ultrasonic sensor (50), the method comprising the following steps: - A first signal (1) is sent (102) to the object (10) by the at least one ultrasonic sensor (50); - The second signal (2) is received (104) by the at least one ultrasonic sensor (50), wherein, The second signal (2) represents the signal backscattered from the object (10). - Process the second signal (2) (106) into a first digital signal (3); - The first digital signal (3) is converted (108) into a complex value signal representation (20) based on time-frequency transformation or modulation method to obtain the second digital signal (4). - Separate (110) the second digital signal (4) into a first feature vector (5) and / or at least into a second feature vector (6), wherein the first feature vector (5) contains a plurality of amplitude values (21) of the second digital signal (4), and the at least second feature vector (6) contains a plurality of phase values (23) of the second digital signal (4). - Convert the jump value range of the phase (23) of the second eigenvector (6) into a continuous curve (24) (112). - Determine (114) the gradient (8) of the curve of the phase (24) in the time direction or frequency direction; - Create (116) third eigenvector (7) based on the determined gradient (8); - The at least third feature vector (7) is fed (118) into the neural network (9) as the first input parameter (11); - When using the neural network (9), determine (120) the object category information (12) of the at least one object (10), wherein the neural network (9) generates an output parameter (15) based on the at least first input parameter (11), the output parameter describing the probability value of at least one defined object category of the object (10).
2. The method (100) according to claim 1, wherein, The step of processing the second signal (2) into a first digital signal (3) includes filtering the first digital signal (3) to improve the S / R ratio.
3. The method (100) according to any one of the preceding claims, wherein, The modulation method is constructed as in-phase and quadrature (I&Q) modulation.
4. The method (100) according to any one of the preceding claims, wherein, The complex numerical signal representation is mapped to a time-frequency representation.
5. The method (100) according to any one of the preceding claims, wherein, The first feature vector (5) is fed into the neural network (9) as the second input parameter (12).
6. A detection system (200) for classifying an object (10) to be detected using at least one ultrasonic sensor (50), said detection system being configured to perform the method (100) according to any one of claims 1 to 5, wherein, The at least one ultrasonic sensor (50) can be used in a vehicle.
7. A computer program comprising machine-readable instructions that, when executed on one or more computers and / or computing instances, cause the one or more computers or computing instances to perform the method according to any one of claims 1 to 5.
8. A machine-readable data carrier and / or downloadable product having the computer program according to claim 7.
9. One or more computers and / or computing instances having a computer program as claimed in claim 7 and / or a machine-readable data carrier and / or downloadable product as claimed in claim 8.