Method and device for processing data associated with at least one ultrasonic sensor

Abstract

A method for processing data associated with at least one ultrasonic sensor, the method comprising: receiving at least one signal characterizing at least a portion of a transmitted ultrasonic signal; determining a two-dimensional data set based on the at least one signal, the two-dimensional data set having a plurality of cells characterizing a potential position of an object relative to an ultrasonic sensor associated with the at least one signal; the data set is processed using a neural network.

Description

Technical Field

[0001] The present disclosure relates to a method for processing data associated with at least one ultrasonic sensor.

[0002] The present disclosure also relates to an apparatus for processing data associated with at least one ultrasonic sensor. Background Art

[0003] None Summary of the Invention

[0004] Some example embodiments relate to a method, e.g., a computer-implemented method, for processing data associated with at least one ultrasonic sensor, the method comprising: receiving at least one signal characterizing at least a portion of an emitted ultrasonic signal; determining, based on the at least one signal, a two-dimensional data set representing a plurality of cells characterizing potential positions of an object relative to the ultrasonic sensor associated with the at least one signal; and processing the data set using a neural network, e.g., an artificial neural network.

[0005] In some examples, the ultrasonic sensor may comprise or represent at least one receiver for receiving ultrasonic signals.

[0006] In some examples, the ultrasonic sensor may comprise at least one transmitter for emitting ultrasonic signals.

[0007] In some examples, the ultrasonic sensor may comprise at least one transceiver or transducer for transmitting and receiving ultrasonic signals, e.g., in a time-division duplex scheme.

[0008] In some examples, at least one signal characterizing at least a portion of the emitted ultrasonic signal is, e.g., an electrical signal, e.g., a voltage, e.g., an output voltage provided by an ultrasonic receiver.

[0009] In some examples, at least one signal characterizing at least a portion of the emitted ultrasonic signal may be a digital signal, e.g., a time-discrete and / or value-discrete signal. In some examples, such a digital signal may be obtained by converting an analog output signal of an ultrasonic receiver using an analog-to-digital converter.

[0010] In some examples, the portion of the emitted ultrasonic signal may be the received portion of the ultrasonic signal that has been emitted by a transmitter and has been reflected and / or scattered by at least one object before being received by the receiver. Thus, in other words, in some examples, the portion of the emitted ultrasonic signal may include information related to at least one object (e.g., an obstacle), and in some examples, such information may be evaluated, e.g., using a neural network.

[0011] In some examples, the cells of the two-dimensional data set are associated with relative positions of the discretization, e.g., relative positions with respect to a receiver that receives the portion of the transmitted ultrasonic signal, wherein, in some examples, the spatial resolution can be determined based on a particular application. In some examples, for instance, if used for obstacle detection in a vehicle such as a car according to the principles of the present disclosure, a spatial resolution of several centimeters per cell (e.g., 10 cm) can be used.

[0012] In some examples, the data set represents a rectangular array of cells (e.g., organized in rows and columns), the cells corresponding to respective relative positions with respect to an ultrasonic sensor, wherein each cell can be assigned at least one cell value based on at least one signal, e.g., can include at least one cell value.

[0013] In some examples, the cell value can be a scalar. In some examples, the cell value can be a vector, e.g., including more than one element.

[0014] In some examples, the cell value can be determined based on at least one signal characterizing at least a portion of the transmitted ultrasonic signal.

[0015] In some examples, the neural network is a convolutional neural network, e.g., a convolutional neural network of the U-Net type, wherein, for example, the neural network is configured to receive image data as input data.

[0016] In some examples, the image data represents two-dimensional data, e.g., a matrix type or arrangement structure representing scalar values or vectors, e.g., a rectangular array.

[0017] In some examples, the two-dimensional data set representing a plurality of cells represents image data, wherein, for example, the respective cells characterize respective two-dimensional spatial coordinates, e.g., corresponding to image elements of the image data, e.g., pixels, and wherein the cell value of the respective cell characterizes at least one “intensity” or signal value of the corresponding pixel of the image data. In some examples, the cell value of the respective cell can be assigned or modified based on at least one signal characterizing at least a portion of the transmitted ultrasonic signal.

[0018] In some examples, the method includes: determining at least one echo based on the at least one signal, determining a curve associated with the at least one echo, e.g., an elliptical curve, and modifying the data set based on the curve.

[0019] In some examples, the echo is characterized by a maximum value of the at least one signal, e.g., a local maximum value. In some examples, the maximum value, e.g., the local maximum value, is a portion of the at least one signal where the signal amplitude exceeds, e.g., a predetermined threshold.

[0020] In some examples, an ultrasonic signal can be emitted by a transmitter arranged at a first position, and a reflected and / or scattered portion of the emitted ultrasonic signal can be received by a receiver arranged at a second position different from the first position. In some examples, a propagation time, such as a "round-trip" propagation time, can be determined based on the ultrasonic signal and the received portion of the ultrasonic signal, such as the reflected and / or scattered portion. In some examples, based on the propagation time, a relative distance between the transmitter and the receiver can be determined, and based on the relative distance, a curve, such as an elliptical curve, can be determined, wherein the first position and the second position correspond to respective foci of the elliptical curve.

[0021] In some examples, the method includes: mapping a curve to a plurality of cells and modifying, such as assigning, increasing, or decreasing, a cell value of a corresponding cell based on the mapping.

[0022] In some examples, mapping a curve to a plurality of cells can include at least one of the following aspects: a) determining whether the curve intersects a particular cell and, if so, assigning, such as a non-vanishing cell value, or modifying the cell value; or b) repeating aspect a) for at least one additional cell, such as for all cells.

[0023] In some examples, the method includes: determining a feature based on the at least one signal and modifying, such as assigning, increasing, or decreasing, a cell value of a corresponding cell based on the feature.

[0024] In some examples, the feature includes at least one of the following: a) an amplitude of at least one echo associated with the at least one signal; or b) an average background noise associated with the at least one signal; or c) a number of echoes associated with the at least one signal. In some other examples, combinations of the foregoing example features and / or other features are possible.

[0025] In some examples, the method includes: providing a data set to a neural network in the form of one or more layers, wherein, for example, each layer of the one or more layers represents the plurality of cells and the corresponding cell values associated with at least one feature determined based on the at least one signal, and optionally, processing the one or more layers using the neural network. In some examples, for instance, if each cell is assigned two values, such as a vector having two elements, the first layer of the data set can be formed by the first cell values and the second layer of the data set can be formed by the second cell values.

[0026] In some examples, the method includes: configuring, such as training, a neural network to provide as output data a two-dimensional data set representing a plurality of cells, where each cell is associated with at least one of the following: a) a first output value characterizing the probability of the presence of an object and / or at least one additional object; or b) a second output value characterizing the height of the object and / or at least one additional object.

[0027] In some examples, the method includes: providing, such as receiving, a plurality of signals characterizing at least a portion of the respective plurality of transmitted ultrasonic signals, and determining a two-dimensional data set based on the plurality of signals. In some examples, the plurality of signals can be obtained by repeatedly transmitting ultrasonic signals from a transmitter to a receiver. In some other examples, the plurality of signals can be obtained by transmitting ultrasonic signals from different transmitters to one or more receivers.

[0028] Some examples relate to a device configured to be capable of performing the method according to the present disclosure.

[0029] Some examples relate to a control unit, such as an electronic control unit, for example for a vehicle (such as a car or a truck).

[0030] Some examples relate to a vehicle (such as a car or a truck) including at least one device according to the present disclosure and / or at least one control unit according to the present disclosure.

[0031] Some examples relate to a computer program product including instructions that, when executed by a computer, cause the computer to perform the method according to the present disclosure.

[0032] Some examples relate to a computer-readable storage medium including instructions that, when executed by a computer, cause the computer to perform the method according to the present disclosure.

[0033] Some examples relate to a data carrier signal carrying and / or characterizing a computer program product according to the present disclosure.

[0034] Some examples relate to the application of the method according to the present disclosure and / or the device according to the present disclosure and / or the computer program product according to the present disclosure and / or the computer-readable storage medium according to the present disclosure and / or the data carrier signal according to the present disclosure for at least one of the following: a) processing at least one signal characterizing at least a portion of the transmitted ultrasonic signals; or b) using a neural network to determine the position of an object; or c) using a neural network to determine the height of an object; or d) assisting the parking process of a vehicle; or e) triggering the emergency braking of a vehicle. Description of the Drawings

[0035] Some example embodiments will now be described with reference to the drawings, where:

[0036] Figure 1 Schematically shows a simplified flowchart according to some examples,

[0037] Figure 2 Schematically shows a simplified diagram according to some examples,

[0038] Figure 3 Schematically shows a simplified diagram according to some examples,

[0039] Figure 4 Schematically shows a simplified diagram according to some examples,

[0040] Figure 5 Schematically shows a simplified flowchart according to some examples,

[0041] Figure 6 Schematically shows a simplified flowchart according to some examples,

[0042] Figure 7 Schematically shows a simplified flowchart according to some examples,

[0043] Figure 8 Schematically shows a simplified flowchart according to some examples,

[0044] Figure 9 Schematically shows a simplified flowchart according to some examples,

[0045] Figure 10A Schematically shows a simplified diagram according to some examples,

[0046] Figure 10B Schematically shows a simplified diagram according to some examples,

[0047] Figure 11 Schematically shows a simplified flowchart according to some examples,

[0048] Figure 12 Schematically shows a simplified block diagram according to some examples,

[0049] Figure 13 Schematically shows aspects of an application according to some examples,

[0050] Figure 14 Schematically shows a simplified flowchart according to some examples,

[0051] Figure 15 Schematically shows a simplified block diagram according to some examples. Detailed implementation

[0052] See, for example Figure 1 、 Figure 2 、 Figure 3Some embodiments relate to a method, e.g., a computer-implemented method, for processing data associated with at least one ultrasonic sensor, the method comprising: receiving 102( Figure 1 ) at least one signal US-1' that characterizes at least a portion of the transmitted ultrasonic signal US-1; determining 104 a two-dimensional data set DS-1 based on the at least one signal US-1', the two-dimensional data set DS-1 representing a plurality of cells C1, C2, C3( Figure 2 ) that characterize potential positions of an object OBJ( Figure 3 ) relative to the ultrasonic sensor associated with the at least one signal US-1'; processing 106 the data set DS-1 using a neural network NN, e.g., an artificial neural network.

[0053] In some examples, as Figure 2 shown, the ultrasonic sensor may comprise or represent at least one receiver RX-1 for receiving ultrasonic signals.

[0054] In some examples, as Figure 2 shown, the ultrasonic sensor may comprise at least one transmitter TX-1 for transmitting the ultrasonic signal US-1.

[0055] In some examples, as Figure 2 shown, the ultrasonic sensor may comprise at least one transceiver or transducer for transmitting and receiving ultrasonic signals, e.g., in a time-division duplex scheme. In some examples, the transceiver may comprise, e.g., a transmitter TX-1 and a receiver RX-1.

[0056] In some examples, as Figure 2 shown, at least one signal US-1' that characterizes at least a portion of the transmitted ultrasonic signal US-1 is, e.g., an electrical signal, e.g., a voltage, e.g., an output voltage provided by the ultrasonic receiver RX-1.

[0057] In some examples, at least one signal US-1' that characterizes at least a portion of the transmitted ultrasonic signal US-1 may be a digital signal, e.g., a time-discrete and / or value-discrete signal. In some examples, such a digital signal may be obtained by converting an analog output signal of the ultrasonic receiver RX-1 using an analog-to-digital converter (not shown).

[0058] In some examples, as Figure 2As shown, the portion of the ultrasonic signal US-1 that is transmitted can be the received portion of the ultrasonic signal US-1 that has been transmitted by the transmitter TX-1 and has been reflected and / or scattered by at least one object OBJ before it is received by the receiver RX-1. Thus, in other words, in some examples, the portion of the transmitted ultrasonic signal can include information related to at least one object OBJ (such as an obstacle), and in some examples, the information can be evaluated, for example, using a neural network.

[0059] Figure 2 Schematically shown is a transmitter TX-1 configured to transmit an ultrasonic signal US-1, also see the dashed arrow A2, wherein the transmitter TX-1 is arranged at a first position relative to the object OBJ, for example, mounted in a target system of a component such as a vehicle (see arrow A1). In some examples, a portion of the transmitted ultrasonic signal US-1 can be reflected and / or scattered by the object OBJ, and the reflected and / or scattered portion A3 can be received by the receiver RX-1, for example, to provide at least one signal US-1' based on the received signal portion A3. In some examples, based on the propagation times of signals A2 and A3, a portion of an elliptical curve EC1 can be determined, where the positions of the components TX-1 and RX-1 represent the two foci of the elliptical curve EC1.

[0060] In some examples, as Figure 1 shown, the method can further include transmitting 100 the ultrasonic signal US-1.

[0061] In some examples, as Figure 1 shown, the method can include evaluating 108 the output of a neural network NN, where the output can also represent, for example, a data set, for example, a two-dimensional (or higher-dimensional) data set DS-2.

[0062] In some examples, as Figure 3 shown, the cells C1, C2, C3,... of the two-dimensional data set DS-1 (an example diagram of which is shown by Figure 3 ) are associated with discretized relative positions, for example, relative positions with respect to the receiver RX-1 that receives the reflected and / or scattered portion A3 of the transmitted ultrasonic signal US-1 ( Figure 2 ), where, in some examples, the spatial resolution can be determined based on a specific application. In some examples, for example, if used for obstacle detection in a vehicle such as a car according to the principles of the present disclosure, a spatial resolution of several centimeters per cell (such as 10 centimeters) can be used.

[0063] In some examples, as Figure 3As shown, the data set DS-1 represents a rectangular array RA of units C1, C2, C3, ... (e.g., organized in rows and columns), where the units C1, C2, C3, ... correspond to respective relative positions with respect to an ultrasonic sensor (e.g., receiver RX-1 and / or transmitter TX-1), and where each unit can be assigned at least one unit value V1, V2, V3, ... based on at least one signal US-1', e.g., can include at least one unit value V1, V2, V3, ...

[0064] In some examples, the unit values V1, V2, V3, ... can be scalars. In some examples, the unit values V1, V2, V3, ... can be vectors, e.g., including more than one element.

[0065] In some examples, the unit values V1, V2, V3, ... can be determined based on at least one signal US-1' that characterizes at least a portion of the transmitted ultrasonic signal US-1.

[0066] In some examples, the neural network NN is a convolutional neural network CNN, e.g., a convolutional neural network CNN of the U-Net type, where, e.g., the neural network is configured to receive image data as input data.

[0067] In some examples, the U-Net type or architecture is associated with at least some of the following aspects. The U-Net architecture can include an encoder-decoder structure, where, e.g., the encoder is configured to capture context information of the input data (e.g., an input image or other two-dimensional data such as the data set DS-1) through, e.g., a series of convolutional layers and other types of layers such as pooling layers (e.g., for max pooling, etc.). In some examples, the decoder of the U-Net architecture can use the context information to generate output data, e.g., a segmentation or feature map, which can have the same or similar spatial dimensions as the input data.

[0068] In some examples, the U-Net architecture can include a contracting path, e.g., an encoder, which forms, e.g., the left side of a U shape. In some examples, the encoder can include multiple convolutional layers, e.g., followed by a max pooling layer, where these layers, e.g., gradually reduce the spatial dimensions of the input image, e.g., while increasing the number of feature channels, which can enable the neural network to learn high-level features and abstractions.

[0069] In some examples, the U-Net architecture may include an expansion path, such as a decoder, which forms the right side of the U shape. In some examples, the decoder consists of a series of upsampling and / or convolutional layers, e.g., to gradually increase the spatial dimension of the data, e.g., transforming it back to, e.g., the original input size. In some examples, skip connections may be provided, e.g., to connect elements or layers of the expansion path to corresponding elements or layers of the contraction path. In some examples, the skip connections may enable the decoder to access, e.g., low-level features and fine-grained details from the encoder, which may be beneficial, e.g., for accurate segmentation.

[0070] In some examples, other types or architectures of neural networks, i.e., other CNN types or other types besides CNNs, may be used for the neural network NN.

[0071] In some examples, the image data, which may be provided as an input to a U-Net type CNN, for example, represents two-dimensional data, such as a matrix type or arrangement structure representing scalar values or vectors, such as a rectangular array.

[0072] In some examples, a two-dimensional data set DS-1 representing a plurality of units C1, C2, C3,... Figure 1 , Figure 3 ) represents image data, where, for example, the corresponding unit Cn characterizes the corresponding two-dimensional spatial coordinates, such as picture elements corresponding to the image data, such as pixels, and where the unit value Vn of the corresponding unit Cn characterizes at least one "intensity" or signal value of the corresponding pixel of the image data. In some examples, the unit value Vn of the corresponding unit Cn may be assigned or modified based on at least one signal US-1' that characterizes at least a part of the emitted ultrasonic signal US-1.

[0073] In some examples, as Figure 4 , Figure 5 shown, the method includes: determining 110 at least one echo ECHO1 based on at least one signal US-1', determining 112 curves EC1, EC1' associated with the at least one echo ECHO1, such as elliptical curves EC1, EC1', and modifying 114 the data set DS-1 based on the curves EC1, EC1'.

[0074] In some examples, as Figure 4 shown, the echo ECHO1 is characterized by the maximum value of the at least one signal US-1', such as a local maximum. In some examples, the maximum value, such as a local maximum, is the part of the at least one signal US-1' where the signal amplitude exceeds, for example, a predetermined threshold TH. As an example, Figure 4Schematically shows the variation of the signal US-1' over time, the threshold TH, and four example echoes ECHO1, ECHO2, ECHO3, ECHO4, which can be caused, for example, by the reflection and / or scattering of the transmitted signal US-1( Figure 2 ) at one or more objects OBJ.

[0075] In some examples, as Figure 2 shown, as described above, the ultrasonic signal US-1 can be transmitted by a transmitter TX-1 arranged at a first position, and a part A3 after reflection and / or scattering of the transmitted ultrasonic signal US-1 can be received by a receiver RX-1 arranged at a second position different from the first position. In some examples, the propagation time, such as the "round-trip" propagation time, can be determined based on the ultrasonic signal A2 and the received part A3 of the ultrasonic signal, such as the part A3 after reflection and / or scattering. In some examples, based on the propagation time, the relative distance between the transmitter TX-1 and the receiver RX-1 can be determined, and based on the relative distance, a curve EC1, such as an elliptical curve EC1, can be determined, where the first position and the second position correspond to the respective foci of the elliptical curve EC1.

[0076] In some examples, as Figure 6 shown, the method includes: mapping 120 the curve EC1 to a plurality of cells C1, C2, C3,... and modifying 122, such as assigning 122a or increasing 122b or decreasing 122c, the cell value Vn of the corresponding cell Cn based on the mapping 120.

[0077] In this regard, Figure 3 the dashed curve EC1' of Figure 3 schematically shows aspects of the curve EC1 mapped to the cells C1, C2, C3,.... In some examples, as Figure 3 shown, mapping 120 the curve EC1 to a plurality of cells can include at least one of the following aspects: a) determining whether the curve EC1, EC1' intersects a specific cell Cx, and if so, assigning, for example, a non-zero cell value Vx or modifying the cell value Vx; or b) repeating aspect a) for at least one additional cell, such as for all cells. As an example, for Figure 3 the curve EC1' of Figure 3 when the curve EC1' intersects the cell Cn, the cell value Vn can be modified accordingly. For example, the cell value Vn can be set to a non-zero value (for example, assuming the initialization state is that all cell values are set to, for example, zero). As another example, for Figure 3 the curve EC1' of

[0078] In some embodiments, for example, as described above, for example, if curve EC1' intersects a particular cell, instead of assigning a value to the particular cell, the current cell value, which is potentially non - zero and already exists, can be modified, for example, by increasing the cell value based on mapping 120. Thus, in some examples, cells or their cell values can be used, for example, to integrate values associated with different curves (e.g., obtained from multiple received signal portions), and thus, for example, indicate whether a large number of ellipses have been mapped to the corresponding cell by the current cell value.

[0079] In some examples, as Figure 7 shown, the method includes: determining 130 a feature FEAT based on at least one signal US - 1' ( Figure 4 ), and modifying 132, for example, assigning 132a or increasing 132b or decreasing 132c the cell value Vn of the corresponding cell Cn based on the feature FEAT.

[0080] In some examples, the feature FEAT includes at least one of the following: a) the amplitude of at least one echo ECHO1 ( Figure 4 ) associated with the at least one signal US - 1'; or b) the average background noise associated with the at least one signal US - 1'; or c) the number of echoes ECHO1, ECHO2, ECHO3, ECHO4 associated with the at least one signal US - 1' (currently, for example Figure 4 four echoes in

[0081] ). In some other examples, combinations of the foregoing example features aspects and / or other features FEAT are possible. Figure 4 As an example, as Figure 6 shown, for each echo ECHO1,..., ECHO4 of the at least one signal US - 1', a corresponding (e.g., elliptical) curve can be determined, and the cell values of one or more (e.g., all) cells of the data set DS - 1 can be modified as described above, for example, with reference to Figure 7 or Figure 3 for each determined curve. For the sake of clarity,

[0082] the example of Figure 8 Figure 3 only shows a single curve EC1'. Thus, in some examples, information derived from at least one signal US - 1' using at least one or more features FEAT can be provided to the data set DS - 1, which characterizes the input data of the neural network.

[0082] In some examples, as Figure 8As shown, the method includes: providing a dataset DS-1 to a neural network NN in the form of one or more layers LAY, where, for example, each layer in the one or more layers LAY represents the multiple units C1, C2, C3,... and the corresponding unit values V1, V2, V3 associated with at least one feature FEAT determined based on the at least one signal US-1', and optionally, processing 142 the one or more layers LAY using the neural network NN. In some examples, for instance, if each unit is assigned two values (not shown), such as a vector with two elements, the first layer of the dataset can be formed by the first unit value, and the second layer of the dataset can be formed by the second unit value. Thus, in some examples, a first layer including unit values associated with a first feature can be provided, and where at least one additional layer including unit values associated with at least one additional feature can be provided.

[0083] In some examples, such as Figure 9 , Figure 10A As shown, the method includes: configuring 150, such as training 150a, a neural network NN to provide a two-dimensional dataset DS-2 ( Figure 10A shows a simplified illustration DS-2' of the two-dimensional dataset DS-2) as output data, where each unit is associated with at least one of the following: a) a first output value OV1-1, OV2-1, OV3-1,... characterizing the probability of the presence of an object OBJ ( Figure 2 ) and / or at least one additional object; or b) a second output value OV1-2, OV2-2, OV3-2,... characterizing the height of the object OBJ and / or at least one additional object ( Figure 10A ). Figure 9 The optional box 152 of

[0084] represents using the trained neural network NN to process the dataset DS-1.

[0085] In some examples, such as Figure 10B As shown, the output dataset DS-2 can include one layer, where each unit includes a unit value with two elements, for example, a vector unit value. Figure 10BThe simplified rectangular array includes a plurality of example cells, each of the plurality of example cells including two numbers or values, where, for example, a first value ranging between 0.0 and 1.0 characterizes the probability of the presence of an object OBJ at the position characterized by the corresponding cell, and where, for example, a second value including two possible values "0" and "1" indicates, for example, in the form of a current binary classification, the height of the object OBJ at the position characterized by the corresponding cell.

[0086] As an example, Figure 10B The dashed rounded rectangle RR1 of indicates a cell having a relatively large first value, for example, indicating a high probability of the presence of an object at the corresponding position, but having "0" as the second value, for example, indicating that the object has a relatively low height.

[0087] As another example, Figure 10B The dashed rounded rectangle RR2 of indicates a cell having a relatively large first value, for example, indicating a high probability of the presence of an object at the corresponding position, but having "1" as the second value, for example, indicating that the object is relatively high.

[0088] In some examples, the binary classification of height can be used, for example, to indicate applications in the vehicle field, whether an object is passable, i.e., whether the object is low enough for a vehicle to pass (e.g., a curb lowered relative to a wall, for example).

[0089] In some examples, as Figure 11 shown, the method includes: providing 160, for example receiving 160a a plurality of signals US' characterizing at least a portion of the corresponding plurality of transmitted ultrasonic signals US-1 ( Figure 2 ), and determining 162 a two-dimensional data set DS-1 based on the plurality of signals US'. In some examples, the plurality of signals US' can be obtained by repeatedly transmitting the ultrasonic signal US-1 from a transmitter TX-1 to a receiver RX-1. In some other examples, the plurality of signals US' can be obtained by transmitting ultrasonic signals from different transmitters (not shown) to one or more receivers RX-1. Figure 11 The optional block 164 of represents processing the data set DS-1 by a neural network NN, for example, to obtain an output data set DS-2.

[0090] Some examples, as Figure 12 shown, relate to a device 200 configured to be capable of performing the method according to the present disclosure.

[0091] In some examples, device 200 includes: at least one computing unit, such as processor 202, which includes, for example, at least one core 202a; and at least one storage unit 204 associated with (i.e., usable by) the at least one computing unit 202, for example for at least temporarily storing computer program product PRG and / or data DAT, wherein the computer program product PRG is configured, for example, to at least temporarily control the operation of device 200, for example for implementing at least some aspects of the method according to the present disclosure.

[0092] In some examples, data DAT may include, for example, at least one of the following: a) information associated with signal US-1; or b) information associated with signal US-1'; or c) information associated with neural network NN (e.g., parameters such as weights or hyperparameters); or d) information associated with at least one of datasets DS-1, DS-2.

[0093] In some examples, the at least one computing unit 202 may include at least one of the following elements: a microprocessor, a microcontroller, a digital signal processor (DSP), a programmable logic element (e.g., FPGA, field programmable gate array), an ASIC (application specific integrated circuit), a hardware circuit, a tensor processor, a graphics processing unit (GPU). According to further examples, any combination of two or more of these elements is also possible.

[0094] According to some examples, memory unit 204 includes at least one of the following elements: volatile memory 204a (e.g., random access memory (RAM)), non-volatile memory 204b (e.g., flash EEPROM).

[0095] In some examples, computer program product PRG is at least temporarily stored in non-volatile memory 204b. In some examples, data DAT may be at least temporarily stored in RAM 204a.

[0096] In some examples, an optional computer-readable storage medium SM includes instructions, for example in the form of computer program product PRG. As an example, storage medium SM may include or represent a digital storage medium, such as a semiconductor storage device (e.g., solid state drive, SSD) and / or a magnetic storage medium (e.g., disk or hard disk drive (HDD)) and / or an optical storage medium (e.g., compact disc (CD) or DVD (digital versatile disc)), etc.

[0097] In some examples, the device 200 may include an optional data interface 206, for example, for two-way data exchange with at least one other device (not shown). As an example, by means of the data interface 206, a data carrier signal DCS may be received, for example, from at least one other device via a wired or wireless data transmission medium, such as via a (virtual) private computer network and / or a public computer network (such as the Internet).

[0098] In some examples, the data carrier signal DCS may represent or carry a computer program product PRG or at least a part of the computer program product PRG according to the examples.

[0099] Some examples relate to a computer program product PRG including instructions that, when executed by a computer 202, cause the computer 202 to perform a method according to the present disclosure.

[0100] In some examples, as Figure 12 shown, the device 200 may include at least one ultrasonic sensor USS, for example, at least including a receiver RX-1 ( Figure 2 )(or, optionally, further including at least one transmitter TX-1). In some examples, the device 200 may not include at least one ultrasonic sensor USS, but may be connected to at least one ultrasonic sensor USS via, for example, the optional data interface 206.

[0101] In some examples, as Figure 12 shown, the device 200 may include at least one signal source SS, for example, providing at least one signal US-1', and the signal source SS includes, for example, a circuit for processing the output signal of the receiver RX-1 ( Figure 2 ). In some examples, the device 200 may not include at least one signal source SS, but may be connected to at least one signal source SS via, for example, the optional data interface 206.

[0102] Some examples, as Figure 13 shown, relate to an application 300 of a method according to the present disclosure and / or a device 200 according to the present disclosure and / or a computer program product PRG according to the present disclosure and / or a computer-readable storage medium SM according to the present disclosure and / or a data carrier signal DCS according to the present disclosure for at least one of the following: a) processing 301 at least one signal US-1' characterizing at least a part of the transmitted ultrasonic signal US-1; or b) using 302 a neural network NN to determine the position of an object OBJ; or c) using 303 a neural network NN to determine the height of an object OBJ; or d) assisting 304 the parking process of a vehicle; or e) triggering 305 an emergency brake of a vehicle.

[0103] Figure 14 Schematically shows a simplified flow chart according to some examples. Element E1 represents a part of a vehicle, such as the front part, where ultrasonic transmitters E2a, E2b, E2c and at least one ultrasonic receiver E3 are provided. Element E4 represents a plurality of received signals that can be obtained according to some examples, such as as part of the received ultrasonic signal A3( Figure 2 ), or for example in the form of a digital signal derived from the received ultrasonic signal. Element E5 represents an example data set DS-1 that can be obtained based on signal E4 in some embodiments, for example, by mapping elliptic curves E6a, E6b, E6c, E6d, E6e determined based on signal E4 to corresponding cells.

[0104] Element E7 represents a plurality of layers characterizing aspects of the example data set E5. Element E8 represents a neural network NN, such as a neural network NN of the CNN type, according to the U-Net architecture, which has an encoder part E7a, a decoder part E7b, and optionally skip connections E7c between the encoder part E7a and the decoder part E7b. In some examples, the neural network E8 is configured to receive a plurality of layers E7 as input and output a data set E8 including, for example, a plurality of cells corresponding to the cells of the data set E5. In some examples, the cells of the data set E8 may include a first cell value indicating the probability of the presence of an object, see arrows E8a, E8b. In some examples, the cells of the data set E8 may include a second cell value indicating the height of the object.

[0105] In some examples, the output data set E8 can be used to visualize the information obtained by the neural network E7, for example, using a graphical user interface, such as visualizing it to the driver of the vehicle 10.

[0106] In some examples, for example, related to the Figure 14 example aspects, a vehicle ultrasonic signal processing method can be provided as follows: The ultrasonic sensors of the vehicle first determine, for example, identify, for example, echoes from the reflection surface of at least one object OBJ( Figure 2 ). In some examples, possible positions, such as the source of the echo, can then be represented by an elliptic curve, which can be mapped to a cell array of the data structure DS-1, also see Figure 14 element E5. In some examples, the data falling into the cells (i.e., according to the mapping), thus forming cell values, can then be aggregated, for example, using one or more different methods, for example, cell by cell, for example, thereby generating a plurality of layers E7, such as a feature layer. In some examples, the neural network E7 is trained to detect and classify objects OBJ (e.g., the probability of presence and / or height) from the environment.

[0107] Some examples, such as Figure 15 shown, relate to a control unit, such as an electronic control unit 20, for example for a vehicle 10 (such as a car or a truck). In some examples, the control unit 20 may comprise or represent a device 200.

[0108] Some examples, such as Figure 15 shown, relate to a vehicle 10 (such as a car or a truck) comprising at least one device 200 according to the present disclosure and / or at least one control unit 20 according to the present disclosure.

[0109] In the following, further aspects and examples are disclosed, which in some examples may be combined with at least one of the aspects and / or examples disclosed above.

[0110] In some examples, the principles according to the present disclosure may be used for object detection, such as localization, and / or for height classification. In some examples, a framework may be provided, for example, for adapting information from ultrasonic sensors (such as at least one receiver RX-1) to a neural network NN, which neural network NN is, for example, configured to process two-dimensional data, such as image data.

[0111] In some examples, for instance, in each cycle, some ultrasonic sensors, such as transmitters, may, for example, emit pulses or signals accordingly according to a predefined pattern. In some examples, the scattered and / or reflected signal portions of these emitted signals may then be received by at least one receiver RX-1( Figure 2 ) which may, for example, identify one or more peaks (such as "echoes") in the received signal.

[0112] In some examples, one or more different quantities may be assigned to each echo, such as amplitude, frequency, background noise, round-trip time, and / or others. In some examples, for instance, an ellipse (such as an elliptical curve) may be determined based on the round-trip times belonging to different peaks, which ellipse is, for example, formed as a possible source of reflection / scattering. In some examples, the elliptical curve associated with the echo may then be rasterized, for example, assigned to corresponding cells, such as forming a grid or a rectangular array, which in some examples may, for example, form a fixed reference system. In some examples, in this way, each echo may be assigned to those grid cells through which the elliptical curve belonging to the corresponding echo passes.

[0113] In some examples, each cell is traversed one by one, and one or more different methods can be used, for example, to aggregate the echoes assigned to the cell, each of the methods being associated with each different feature, for example, including at least one of the following: a) the average amplitude of the echoes whose ellipses fall into the cell; or b) the sum of the amplitudes of the echoes in the cell; or c) the average background noise of the amplitude or the signal US-S'; or d) the total number of echoes in the grid cell.

[0114] In some examples, if the ellipse of echo ECHO1 passes through more cells than the ellipse of another echo ECHO2, then each cell belonging to the first echo ECHO1 may contain an object with a higher probability. In some examples, the fewer the number of cells assigned to an echo, the higher the probability that the reflection originated from a specific cell / position. In some examples, for this reason, at least some of the above example features can be calculated by dividing their sum or average by the number of cells through which the relevant ellipse passes.

[0115] In some examples, features can be calculated according to some specific ways in which echoes are aggregated within each cell. In some examples, different ways can be used to provide features associated with at least one signal US-1' and / or a different number of features. In some examples, each feature can be associated with a layer of the dataset DS-1, thus forming, for example, multiple feature layers.

[0116] In some examples, the feature layers forming the dataset DS-1 can be used as an input, which is fed into a neural network NN, such as a U-Net convolutional neural network architecture. In some examples, the neural network NN assigns a classification variable (e.g., cell value) to each grid cell. In some examples, the output of the neural network NN can be used for two purposes. First, the output can contain the probability of the presence of an object OBJ in each cell. In some examples, this can, for example, determine the position of the reflecting object OBJ relative to the present vehicle (e.g., the reference point of a target system such as vehicle 10), and optionally, how much free space there is around the present vehicle in all directions. In some examples, second, for example, if there is an object in the cell, the neural network NN can classify the object according to the following convention: a) if the object can be driven over (e.g., speed bump, lowered curb), the cell is classified as "low", b) if the object cannot be driven over (e.g., wall), the cell is classified as "high".

[0117] In some examples, the output of the neural network NN can thus be, for example, an object map in the form of a dataset DS-2, which, for example, precisely assigns a category: "empty", "low", or "high" to each cell.

[0118] In some examples, instead of the neural network NN, another type of machine learning model can be used.

[0119] In some examples, for example, to train a model such as the neural network NN, labeled measurements can be used, such as manually labeled measurements. In some examples, in addition to the echo itself, these labeled measurements can also record the position of the vehicle, such as the position in a specific reference system, such as the position attached to the ground. In some examples, the objects in the environment can then be manually located, for example, in the same reference system (e.g., together with the height level of the object). In some examples, these labels can later be used, for example, to label the cells containing objects with "none", "low", or "high".

[0120] In some examples, the output DS-2 of the neural network NN, such as the object map, which for example informs about the position where the object OBJ is located and whether the object can be driven over, can be used for various purposes, for example. In some examples, the cells where the objects are located can be clustered and assigned to the objects in the environment. In some examples, in this way, the output DS-2 can be used for object detection, for example, in the perception part of an ADAS (Advanced Driver Assistance System). In some examples, the empty cells (e.g., where no object is predicted) can be used for free space estimation, which can provide information for a parking system, for example. In some examples, for example, if a cell containing a high object is predicted at close range, the ADAS system can utilize this information to decide, for example, whether emergency braking should be triggered.

Claims

1. A method, such as a computer-implemented method, for processing data associated with at least one ultrasonic sensor, the method comprising: receiving (102) at least one signal (US-1') representing at least a portion of the transmitted ultrasonic signal (US-1); Based on the at least one signal (US-1'), a two-dimensional data set (DS-1) is determined (104), the two-dimensional data set (DS-1) representing a plurality of cells (C1, C2, C3, ...) representing potential positions of an object (OBJ) relative to an ultrasonic sensor associated with the at least one signal (US-1'); the data set (DS-1) is processed (106) using a neural network (NN), such as an artificial neural network.

2. The method according to claim 1, wherein: The data set (DS-1) represents a rectangular array (RA) of cells (C1, C2, C3, ...), the cells (C1, C2, C3, ...) corresponding to respective relative positions relative to the ultrasonic sensor, each cell (C1, C2, C3, ...) being assignable with at least one cell value (V1, V2, V3, ...) based on the at least one signal (US-1').

3. A method according to any one of the preceding claims, wherein: The neural network (NN) is a convolutional neural network, for example a convolutional neural network of the U-Net type, for example, the neural network (NN) is configured to receive image data as input data.

4. A method according to any one of the preceding claims, wherein: The method comprises determining (110) at least one echo (ECHO1) based on the at least one signal (US-1′), determining (112) a curve (EC1), for example an elliptic curve (EC1), associated with the at least one echo (ECHO1), and modifying (114) a data set (DS-1) based on the curve (EC1).

5. The method according to claim 4, wherein: The method comprises mapping (120) the curve (EC1) to the plurality of cells (C1, C2, C3, . . . ), modifying (122), for example assigning (122a) or increasing (122b) or decreasing (122c) a cell value (Vn) of a corresponding cell (Cn) based on the mapping (120).

6. A method according to any one of the preceding claims, wherein: The method comprises determining (130) a characteristic (FEAT) based on the at least one signal (US-1'), modifying (132), for example assigning (132a) or increasing (132b) or decreasing (132c) a cell value (Vn) of a corresponding cell (Cn) based on the characteristic (FEAT).

7. The method according to claim 6, wherein: The feature (FEAT) comprises at least one of: a) the amplitude of at least one echo (ECHO1) associated with the at least one signal (US-1'); or b) the average background noise associated with the at least one signal (US-1'); or c) the number of echoes associated with the at least one signal (US-1').

8. A method according to any one of the preceding claims, comprising: A data set (DS-1) is provided (140) to a neural network (NN) in the form of one or more layers (LAY), wherein, for example, each of the one or more layers (LAY) represents the plurality of cells (C1, C2, C3, ...) and corresponding cell values ​​(V1, V2, V3, ...) associated with at least one feature (FEAT) determined based on the at least one signal (US-1'), and optionally, the one or more layers (LAY) are processed (142) using the neural network (NN).

9. A method according to any one of the preceding claims, comprising: The neural network (NN) is configured (150), for example trained (150a), to provide a two-dimensional data set (DS-2) representing the plurality of cells (C1, C2, C3, ...) as output data, wherein each cell (C1, C2, C3, ...) is associated with at least one of: a) a first output value (OV1-1, OV2-1, OV3-1, ...) characterizing a probability of the existence of the object (OBJ) and / or at least one other object; or b) a second output value (OV1-2, OV2-2, OV3-2, ...) characterizing a height of the object (OBJ) and / or at least one other object.

10. A method according to any one of the preceding claims, wherein: The method comprises providing (160), for example receiving (160a), a plurality of signals (US') representing at least a portion of a respective plurality of transmitted ultrasound signals, and determining (162) a two-dimensional data set (DS-1) based on the plurality of signals (US').

11. A device (200) configured to perform the method according to any one of the preceding claims.

12. A computer program product (PRG; PRG') comprising instructions which, when the program (PRG; PRG') is executed by a computer (202), cause the computer (202) to perform the method according to at least one of claims 1 to 10.

13. A computer-readable storage medium (SM) comprising instructions (PRG') which, when executed by a computer (202), cause the computer (202) to perform the method according to at least one of claims 1-10.

14. A data carrier signal (DCS) carrying and / or characterizing a computer program product (PRG; PRG') according to claim 12.

15. An application of a method according to any one of claims 1 to 10 and / or an apparatus (200) according to claim 11 and / or a computer program product (PRG; PRG') according to claim 12 and / or a computer-readable storage medium (SM) according to claim 13 and / or a data carrier signal (DCS) according to claim 14, for at least one of the following: a) processing (301) at least one signal (US-1') representing at least a part of the emitted ultrasonic signal (US-1); or b) using (302) a neural network to determine the position of an object (OBJ); or c) using (303) a neural network to determine the height of an object (OBJ); or d) assisting (304) a vehicle parking process, e) triggering (305) emergency braking of the vehicle.

Images