Method for training artificial intelligence module, sensor system and vehicle
By processing the training dataset of ultrasonic array sensors and artificial intelligence modules, the problem of insufficient detection accuracy in ultrasonic signal analysis and evaluation in vehicle environments is solved, enabling accurate classification of near-field obstacles and saving resources.
Patent Information
- Application Number
- CN202510580116.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2024-05-07
- Filing Date
- 2025-05-07
- Publication Date
- 2025-11-07
AI Technical Summary
Existing technologies for analyzing and evaluating ultrasonic signals in vehicle environments suffer from insufficient detection accuracy, particularly in the near-field region where obstacle classification is difficult, and the high cost of components necessitates more efficient solutions.
By using an ultrasonic array sensor and a computer-implemented artificial intelligence module, the training dataset generates and processes ultrasonic signal reflection data. Beamforming, time-frequency analysis, and source map construction are utilized, combined with neural networks for object classification, thereby improving detection accuracy.
It significantly improves the environmental sensing performance in the near-field area of the vehicle, can accurately distinguish multiple objects at the same distance, and reduces the resource consumption and computational workload of the sensor system.
Smart Images
Figure CN120915802A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to a computer-implemented method for training an artificial intelligence module for determining objects in an environment of a vehicle, a sensor system and a vehicle. BACKGROUND
[0002] Currently, there are a large number of different solutions for analyzing ultrasonic signals in the field of vehicles. Due to the increasing requirements for detection accuracy and the increasing quality expectations, the demand for innovative and robust analysis methods for ultrasonic signals continues to increase.
[0003] In the field of vehicles, the weight is constantly being reduced in order to reduce consumption and competition is increasing, which leads to cost pressure, so the demand for cheaper and more efficient components for vehicles is stronger. SUMMARY
[0004] The computer-implemented method for training an artificial intelligence module for determining objects in an environment of a vehicle according to the application has the following advantages over known methods: The detection accuracy of the object classification by means of ultrasonic sensors, in particular ultrasonic array sensors, can be significantly improved. Thus, in particular, the performance of the environmental sensing in the near-field region of the vehicle can be significantly increased, in particular in terms of object classification of obstacles surrounding the vehicle. Furthermore, preferably, a potential plurality of objects having the same distance, in particular with respect to the vehicle, can be distinguished.
[0005] According to the application, this is achieved in that the computer-implemented method for training an artificial intelligence module for determining objects in an environment of a vehicle has the following steps:
[0006] - providing a measurement value dataset on a data carrier, wherein the measurement value dataset has at least one data entry on a reflection of an ultrasonic signal in an air sound region and at least one data entry on an object class,
[0007] - generating a modified training dataset on the basis of the measurement value dataset, wherein generating the modified training dataset comprises the following steps:
[0008] - forming an input dataset on the basis of the data entry on the reflection of the ultrasonic signal in the air sound region,
[0009] - forming an output dataset on the basis of the data entry on the object class,
[0010] - wherein the method further has the following steps:
[0011] - training the artificial intelligence module on the basis of the modified training dataset.
[0012] In other words, the object can be recognized on the basis of the reflection of the ultrasound signal by means of a trained artificial intelligence module. Here, the ultrasound signal can be an ultrasound array signal which is determined by means of beamforming, in particular. Furthermore, preferably, an ultrasound sensor array can be used to determine input features which can be used for the object classification by means of the artificial intelligence module. Here, the ultrasound sensor array can comprise a plurality of ultrasound transducer elements, in particular. For example, a first input feature can be a time-resolved signal. Here, in particular, the time-resolved signal of the acoustic pressure scattered back at the output of each transducer element can be used as a feature. Furthermore, preferably, the time-resolved signal of the acoustic pressure scattered back from the delay-and-sum calculation, in particular from the beamforming, can be used. In addition to the time signal, a complex baseband or amplitude height curve of the representation of the time signal can also be exhibited. Further features can comprise, in particular, a calculation of a time-frequency representation, for example a short-time Fourier transform or a wavelet transform. Further features can comprise, in particular, a calculation of an acoustic source map by means of beamforming methods, for example a delay-and-sum (Delay & Sum) calculation, a Bartlett method and / or a multiple signal classification (Music) method. Furthermore, preferably, the input features can also comprise an extraction of a bounding box or an object contour segmentation from the source map consisting of the previous features. Furthermore, preferably, further input features can also comprise a calculation of individual object points with directional and distance features from the source map.
[0013] Below, preferred embodiments of the application are shown.
[0014] Furthermore, preferably, the method has, for some steps:
[0015] - constructing a time course on the basis of the data entries on the reflection,
[0016] - extending the input data set by means of the time course.
[0017] The advantage of this embodiment is that the received ultrasound signal can be placed in a time course in order thus to increase the persuasiveness.
[0018] Furthermore, preferably, the method has, for some steps:
[0019] - constructing a time-frequency proportion on the basis of the constructed time course and the data entries on the reflection,
[0020] - extending the input data set by means of the time-frequency proportion.
[0021] The advantage of this configuration is that the relationship of the received ultrasound data signals can be taken into account when training the artificial intelligence module by virtue of the configured proportion, which makes it possible to further improve the detection accuracy of the object.
[0022] Furthermore, it is preferable for the measurement value data set to have data entries on a plurality of reflections of an ultrasound signal emitted by means of the ultrasound array in the air sound field, wherein the method has the following steps:
[0023] - configuring a source map on the basis of the data entries on the plurality of reflections,
[0024] - extending the input data set by means of the source map.
[0025] The advantage of this embodiment is that the correlation of the ultrasound signals, in particular a multidimensional correlation, can always be taken into account by configuring the source map.
[0026] Furthermore, it is preferable for the method to have the following steps:
[0027] - determining at least one bounding volume and / or object contour on the basis of the configured source map,
[0028] - extending the input data set by means of the bounding volume and / or the object contour.
[0029] The advantage of this embodiment is that different object classes can be excluded, in particular on the basis of the determined bounding volume and / or object contour. For example, when a rectangular contour is recognized, objects with a circular contour, for example balls, can be excluded for further object classification.
[0030] Furthermore, it is preferable for the source map to have at least one vector describing a reflection path and / or a reflection orientation, wherein the method has the following steps:
[0031] - determining an object reference, in particular an object center point, on the basis of the source map with the at least one vector,
[0032] - extending the input data set by means of the determined object reference point.
[0033] The advantage of this embodiment is that the data can be processed in a significantly simpler manner, since the position in three-dimensional space can be determined simply as a result of the vector with the object center point.
[0034] A further aspect of the application relates to a sensor system having an ultrasonic sensor unit, wherein the ultrasonic sensor unit is configured to emit at least one ultrasonic signal into an air acoustic domain, wherein the ultrasonic sensor unit is configured to receive the emitted ultrasonic signal in the air acoustic domain, wherein the sensor system can be connected to an artificial intelligence module which has been trained with a method as described above and below, wherein the sensor system is configured to determine a class of an object in the environment of the sensor system by means of the artificial intelligence module and the received ultrasonic signal.
[0035] The advantage of this embodiment is that the detection accuracy of the sensor system and its object classification can be significantly improved by means of the artificial intelligence module. Here, in particular, the object classification of objects which have substantially the same distance, in particular, with respect to the sensor system, can be improved.
[0036] Furthermore, preferably, the ultrasonic sensor unit has an ultrasonic array with a plurality of ultrasonic elements, wherein the ultrasonic sensor unit is configured to emit a set of ultrasonic waves comprising the ultrasonic signal by means of the ultrasonic array, wherein the ultrasonic sensor unit is configured to adjust the orientation of the set of ultrasonic waves.
[0037] The advantage of this embodiment is that the set of ultrasonic waves can provide a spatial resolution, for example, by means of beamforming. Thus, in particular, a source map can be calculated. Here, in particular, an acoustic source map can be created by means of the adjustment of the set of ultrasonic waves, so that a two- or three-dimensional sound pressure or other variables can be displayed thereby. As possible calculation methods, Delay & Sum, Capon, Clean SC, Multiple Signal Classification (MUSIC) or similar methods can be utilized, in particular.
[0038] Furthermore, preferably, the ultrasonic sensor unit is configured to create a source map by adjusting the orientation of the set of ultrasonic waves.
[0039] The advantage of this embodiment is that the processing of the ultrasonic signal can be significantly simplified by means of the source map.
[0040] Furthermore, preferably, the sensor system is configured to derive a contour of the object on the basis of the received ultrasonic signal.
[0041] An advantage of this embodiment is that the number of detected objects can be determined by determining contours, bounding volumes or similar features. Here, in particular, an estimate of the number and the spatial position of the determined objects can be determined from the contours of the determined objects. Here, the bounding volumes and / or contours are in particular simple geometric figures which can encompass complex two- or multi-dimensional objects in the source map. The contours are in particular outlines, for example lines, which describe the object or its edges as outer lines and can be determined by means of image segmentation. Here, the bounding volumes and / or contours can in particular be determined on the basis of the source map and thus also other visualizations of the vehicle environment can be transmitted. Furthermore, preferably, simple methods can be used when calculating the bounding boxes or contours. Here, in particular, groups of points from the bounding volumes or contours can be brought together. Furthermore, preferably, the maximum can also be selected as the first point and then other groups of points can be selected as the magnitude decreases. Furthermore, preferably, methods can be used which use separate neural networks for calculating the bounding volumes or contours, for example Mask-R-CNN or SegNet.
[0042] Furthermore, preferably, the sensor system is configured to determine a center point of the object on the basis of the received ultrasound signals.
[0043] An advantage of this embodiment is that the computational effort can be significantly reduced, for example, without determining bounding volumes or contours, and thus resources can be saved. Here, in particular, the number of objects can be determined by determining the center point of the object.
[0044] Furthermore, preferably, the ultrasound sensor unit is configured to emit and receive a second set of ultrasound waves, wherein the ultrasound sensor unit is configured to adjust the orientation of the second set of ultrasound waves on the basis of the determined object contour and / or the determined object center point.
[0045] An advantage of this embodiment is that the object classification can be limited only to the region in the environment of the sensor system in which it has been assumed or detected that there is an object, so that more resources can thus be saved.
[0046] Furthermore, preferably, the sensor system is configured to adjust the source map on the basis of the received second set of ultrasound waves.
[0047] An advantage of this embodiment is that the detection of objects can be checked so that false positives or other detection errors can thus preferably be ruled out. Here, the second beamforming can be performed, in particular, on the basis of the source map. Preferably, the number of partial source maps that are intercepted corresponds to the number of objects, wherein for each object the source map can be limited to the region belonging to it, taking into account the bounding volume or contour. For the case that an object center point is present, the source map can be limited to the determined region. Preferably, the processing by means of the classification network can be significantly simplified in accordance with the division of the source map.
[0048] Furthermore, preferably, the sensor system is set up to determine the class of an object in the environment of the sensor system by means of the artificial intelligence module, the adjusted source map, the determined object contour, and / or the determined object center point.
[0049] An advantage of this embodiment is that the detection accuracy by means of the artificial intelligence module can be significantly improved, since the input for the classification, in particular with the adjusted source map or the partial source map, is significantly simplified. Preferably, a neural network or a similar network can be used for the classification. Preferably, for this a convolutional neural network and other network architectures, such as MLPs, RNNs, Transformers, Auto-Encoders, or combinations thereof, can be used.
[0050] A further aspect of the application relates to a vehicle having a sensor system as described above and below and / or having an artificial intelligence module that has been trained by means of a method as described above and below. BRIEF DESCRIPTION OF DRAWINGS
[0051] Embodiments of the application are described in detail below with reference to the attached drawing figures. The drawing figures show:
[0052] Figure 1 and Figure 2 a schematic diagram for illustrating the working principle of a sensor system according to an embodiment,
[0053] Figure 3 a sensor system according to an embodiment,
[0054] Figure 4 a vehicle according to an embodiment,
[0055] Figure 5 and Figure 6 a flowchart for illustrating a method according to an embodiment.
[0056] Preferably, all identical elements, units, and / or steps are provided with identical reference signs in all of the drawing figures. DETAILED DESCRIPTION
[0057] Figure 1 A schematic 500 for illustrating the working principle of the sensor system 300 is shown. The schematic 500 comprises an ultrasound sensor unit 502. The ultrasound sensor unit 502 can inter alia transmit and receive ultrasound signals, which are inter alia used for beamforming 504. Based on the beamforming 504 a bounding volume and a contour of an object can be determined. Based on the determined bounding volume and contour 506 a second beamforming 508 can be performed, inter alia by means of the ultrasound sensor unit 502. Furthermore, preferably, a source map determined by means of the first beamforming 504 can be adjusted in a step 510. Based on the second beamforming 508 and in the adjusted source map 510 a probability of an object classification can be output in a step 514 by means of a classification model 512.
[0058] Figure 2 A schematic 600 for illustrating the working principle of the sensor system 300 is shown. Here, the schematic 600 has inter alia an ultrasound sensor unit 602. A center point 608 can be determined inter alia by means of a first beamforming 604. Based on the determined center point 608 a second beamforming 606 can be performed by means of the ultrasound sensor unit 602. Furthermore, preferably, a source map 610 can be adjusted based on the first beamforming 604 and the determined center point 608. Based on the adjusted source map 610 and in the second beamforming 606 an object can be classified 614 by means of a classification model 612.
[0059] Figure 3 A sensor system 300 according to an embodiment is shown. The sensor system 300 has an ultrasound sensor unit 400, wherein the ultrasound sensor unit 400 is configured to emit at least one ultrasound signal into an acoustic domain, wherein the ultrasound sensor unit 400 is configured to receive the emitted ultrasound signal in the acoustic domain, wherein the sensor system 300 can be connected with an artificial intelligence module 202 that has been trained by means of a method 100 as described above and below, wherein the sensor system 300 is configured to determine a class of an object in an environment of the sensor system 300 by means of the artificial intelligence module 202 and the received ultrasound signal. Preferably, the sensor system 300 can comprise the artificial intelligence module 202 or a data connection to the artificial intelligence module 202 can be established, so that the class of the object can be determined.
[0060] Figure 4 A vehicle 200 according to an embodiment is shown. Preferably, the vehicle 200 has a sensor system 300 as described above and below. Furthermore, preferably, the vehicle 200 can have an artificial intelligence module 202 that has been trained by means of a method 100 as described above and below.
[0061] Figure 5 A flow chart illustrating steps of a method 100 according to an embodiment is shown. The method 100 for training an artificial intelligence module 202 for determining an object in an environment of a vehicle 200 comprises the following steps:
[0062] - providing S1 a measurement value dataset on a data carrier, wherein the measurement value dataset has at least one data entry on a reflection of an ultrasound signal in an air sound domain and at least one data component on an object class,
[0063] - generating S2 a modified training dataset based on the measurement value dataset, wherein generating the modified training dataset comprises the following steps:
[0064] - forming S3 an input dataset based on the data entry on a reflection of an ultrasound signal in an air sound domain,
[0065] - forming S4 an output dataset based on the data entry on an object class,
[0066] - wherein the method 100 further has the following steps:
[0067] - training S5 the artificial intelligence module 202 based on the modified training dataset.
[0068] Figure 6 A flow chart illustrating steps of a method 100 according to an embodiment is shown. Preferably, the method 100 has the same steps S1 to S5 as already explained with reference to the method 100. Figure 5 Furthermore, preferably, the method 100 further has a step of constructing S6 a time course and a step of extending S7 the input dataset. Furthermore, preferably, the method 100 has a step of constructing S8 a time- frequency proportion and a step of extending S9 the input dataset. Furthermore, preferably, the method 100 has a step of constructing S10 a source map and a step of extending S11 the input dataset by means of the source map. Preferably, the method 100 has a step of ascertaining S12 an enclosing volume or an object contour and, preferably, a step of extending S13 the input dataset by means of the enclosing volume and / or the object contour. Furthermore, preferably, the method 100 further has a step of ascertaining S14 an object reference and a step of extending S15 the input dataset.
Claims
1. A computer-implemented method (100) for training an artificial intelligence module for determining an object in an environment of a vehicle (200), having the following steps: - providing (S1) a measurement dataset on a data carrier, wherein - a measurement data set having at least one data entry on a reflection of an ultrasound signal in an air acoustic domain and - at least one data entry on an object class, - generating (S2) a modified training data set on the basis of the measurement data set, wherein the generation of the modified training data set comprises the following steps: - forming (S3) an input data set on the basis of the data entry on the reflection of the ultrasound signal in the air acoustic domain, - forming (S4) an output data set on the basis of the data entry on the object class, - wherein the method (100) further has the following steps: - training (S5) the artificial intelligence module on the basis of the modified training data set.
2. The method (100) of claim 1, wherein The method (100) further has the following steps: - constructing (S6) a time course on the basis of the data entry on the reflection, - expanding (S7) the input data set by means of the time course.
3. The method (100) of claim 2, wherein The method (100) further has the following steps: - constructing (S8) a time-frequency proportion on the basis of the constructed time course and the data entry on the reflection, - expanding (S9) the input data set by means of the time-frequency proportion.
4. The method (100) according to any one of the preceding claims, wherein The measurement data set has data entries on a plurality of reflections of an ultrasound signal in the air acoustic domain, which is emitted by means of an ultrasound array, wherein the method (100) further has the following steps: - constructing (S10) a source map on the basis of the data entries on the plurality of reflections, - expanding (S11) the input data set by means of the source map.
5. The method (100) of claim 4, wherein The method (100) further has the following steps: - deriving (S12) at least one bounding volume and / or an object contour on the basis of the constructed source map, - expanding (S13) the input data set by means of the bounding volume and / or the object contour.
6. The method (100) according to claim 4 or 5, wherein The source map has at least one vector describing a reflection path and / or a reflection orientation, wherein the method (100) further has the following steps: - deriving (S14) an object reference, in particular an object center point, on the basis of the source map with the at least one vector, - expanding (S15) the input data set by means of the derived object reference. The ultrasound sensor unit (400) is designed to emit at least one ultrasound signal in an air acoustic domain, wherein the ultrasound sensor unit (400) is designed to receive the emitted ultrasound signal in the air acoustic domain, wherein the sensor system (300) can be connected to an artificial intelligence module (202) which has been trained by means of the method according to any one of claims 1 to 6, wherein the sensor system (300) is designed to determine a class of an object in an environment of the sensor system (300) by means of the artificial intelligence module (202) and the received ultrasound signal.
7. A sensor system (300) with an ultrasonic sensor unit (400), wherein 8. The sensor system (300) according to claim 7, wherein The ultrasound sensor unit (400) has an ultrasound array (402) with a plurality of ultrasound elements (404), wherein the ultrasound sensor unit (400) is configured to emit a set of ultrasound waves comprising the ultrasound signals by means of the ultrasound array (402), wherein the ultrasound sensor unit (400) is configured to adjust an orientation of the set of ultrasound waves.
9. The sensor system (300) according to claim 8, wherein The ultrasound sensor unit (400) is configured to create a source map by the adjustment of the orientation of the set of ultrasound waves.
10. The sensor system (300) according to any one of claims 7 to 9, wherein, The sensor system (300) is configured to determine a contour of the object based on the received ultrasound signals.
11. The sensor system (300) according to any one of claims 7 to 10, wherein The sensor system (300) is configured to determine a center point of the object based on the received ultrasound signals.
12. The sensor system (300) according to any one of claims 8 to 11, wherein The ultrasound sensor unit (400) is configured to emit and receive a second set of ultrasound waves, wherein the ultrasound sensor unit (400) is configured to adjust an orientation of the second set of ultrasound waves based on the determined contour and / or the determined center point of the object.
13. The sensor system (300) according to any one of claims 9 to 12, wherein The sensor system (300) is configured to adjust the source map based on the received second set of ultrasound waves.
14. The sensor system (300) according to claim 13, wherein The sensor system (300) is configured to determine a class of the object in an environment of the sensor system (300) by means of the artificial intelligence module (202), the adjusted source map, the determined contour of the object and / or the determined center point of the object.
15. A vehicle having a sensor system (300) according to any one of claims 7 to 14 and / or having an artificial intelligence module (202) trained by means of a method according to any one of claims 1 to 6.