Method and device for operating a radar-based vehicle interior monitoring system

Abstract

A method for operating a radar-based vehicle interior monitoring system is described, wherein the vehicle interior monitoring system comprises, as radar sensors, a first UWB sensor (10) with at least one antenna (11) and a second UWB sensor (20) with at least one antenna (21), as well as an evaluation unit (30). The first UWB sensor (10), the second UWB sensor (20), and the evaluation unit are interconnected for data exchange via a data bus (40). The first and second UWB sensors (10, 20) are operated as a bistable radar, wherein radar pulses emitted by the first UWB sensor (10) are received by the second UWB sensor (20) as radar echoes.The evaluation unit (30) synchronizes the radar pulses emitted by the first UWB sensor (10) with the radar echoes received by the second UWB sensor (20) before determining the channel impulse response from the emitted radar pulses and the radar echoes.

Description

[0001] The invention relates to a method and a device for operating a radar-based vehicle interior monitoring system using two UWB (ultra-wideband) sensors. Such a UWB sensor is used for occupant detection in the motor vehicle.

[0002] The principle of a UWB sensor, which is a radar sensor, is to emit radar pulses and then receive radar echoes reflected from objects. Typically, a large number of radar pulses are emitted within a specified period of time, which are then received as radar echoes. The duration of a single radar pulse is usually in the range of a few nanoseconds to several hundred picoseconds. By appropriately evaluating the radar echoes, signs of life from vehicle occupants and thus their presence can be detected. Detection can be determined, for example, by the rise and fall of the vehicle occupant's chest as a result of breathing. Depending on the vehicle, one or more UWB sensors can be provided for complete monitoring of the vehicle interior.

[0003] The raw data to be processed comprises a large number of radar echoes from a radar pulse repeatedly transmitted within a specified period of time as a function of a number of so-called taps. A tap represents a characteristic value for the range of the UWB sensor. The higher the so-called tap number, the greater the distance between the UWB sensor and the object reflecting the signal. After being filtered by a sensor processing unit, the large number of radar echoes is transmitted to a higher-level processing unit to be stored together as a radar image for further processing. Storing the data as a radar image enables the higher-level processing unit to further process the data, e.g., with the help of machine learning. For example, image analysis approaches from machine learning can be used.However, this requires that the higher-level processing unit has sufficient powerful resources available for processing.

[0004] It is an object of the invention to provide a method and a device which provide improved performance of the vehicle interior monitoring system.

[0005] These objects are achieved by a method according to the features of claim 1, a computer program product according to the features of claim 11 and a device according to the features of claim 12. Advantageous embodiments emerge from the dependent claims.

[0006] A method for operating a radar-based vehicle interior monitoring system is proposed. The vehicle interior monitoring system comprises, as radar sensors, a first UWB sensor with at least one antenna and at least one second UWB sensor with at least one antenna, as well as an evaluation unit. The first UWB sensor, the second UWB sensor, and the evaluation unit are connected to each other via a data bus for data exchange. The first and second UWB sensors are each ultra-wideband radar sensors. They are preferably arranged in a vehicle interior under the roof of the motor vehicle for monitoring the interior.

[0007] Raw data from UWB sensors is processed in a conventional manner. The raw data comprises a multitude of radar echoes from a radar pulse repeatedly transmitted within a specified period of time, as a function of a number of taps. The taps represent a characteristic value for the range of a particular UWB sensor. After filtering, the multitude of radar echoes is transmitted to a processing unit, where they are stored together as a radar image for further processing.

[0008] To improve the detection properties of objects in the vehicle interior, the first and second UWB sensors are operated as a bi-static radar, with radar pulses emitted by the first UWB sensor being received as radar echoes by the second UWB sensor. The evaluation unit synchronizes the radar pulses emitted by the first UWB sensor with the radar echoes received by the second UWB sensor before determining a channel impulse response from the emitted radar pulses and the radar echoes.

[0009] The channel impulse response includes the processed radar echoes, which are related to the radar pulses to determine the radar image mentioned above.

[0010] According to a practical embodiment, for synchronization purposes, a received signal is extracted from an uncorrelated radar image generated from the radar pulses emitted by the first UWB sensor and the radar echoes received by the second UWB sensor. This received signal is compared with a reference transmitted signal. The received signal and the reference transmitted signal represent a relationship between a number of radar echoes as a function of a number of taps, which represent a characteristic value for the range of the vehicle interior monitoring system. The uncorrelated radar image initially only represents the relationship between a radar pulse and the associated radar echo. The lack of correlation results from the fact that the transmitter and receiver are not synchronized with each other, particularly in terms of time.However, this synchronization is necessary to process the two signals, radar pulse and associated radar echo, in the usual way for the radar image to be generated. This synchronization is achieved by combining the uncorrelated radar image and extracting a received signal, which is compared with a reference transmitted signal. The reference transmitted signal is derived from the radar pulse.

[0011] According to a practical embodiment, the received signal and the reference transmitted signal are adjusted with respect to the number of taps. In other words, a range adjustment of the received signal and the reference transmitted signal is performed. In this context, it is expedient to process a data point in the reference transmitted signal or the received signal that has the highest tap number as the reference for the adjustment.

[0012] It is also expedient if the received signal and the reference transmitted signal are also temporally aligned. In other words, this step involves a range alignment of the received signal and the reference transmitted signal. Preferably, the first local maximum in the reference transmitted signal or the received signal is processed as the reference for the temporal alignment. This step corresponds to a shift of one of the two signals along the axis specifying the tap number.

[0013] In a further preferred processing step, the extracted received signal is interpolated to a larger number of data points before being compared with the reference transmitted signal. This improves the accuracy of the synchronization of the reference transmitted signal and the received signal.

[0014] A further preferred embodiment provides that the first and / or the second UWB sensor are additionally operated as monostatic radar. According to this embodiment, either only the first UWB sensor or only the second UWB sensor, or both UWB sensors, can be operated as monostatic radar.In this embodiment, for synchronization purposes, a further received signal is extracted from at least one further uncorrelated radar image, which is generated from the radar pulses emitted by the first UWB sensor and the radar echoes received by the first UWB sensor and / or from the radar pulses emitted by the second UWB sensor and the radar echoes received by the second UWB sensor. This further received signal is compared with an associated reference transmitted signal (in particular the respective transmitted radar pulse), wherein the respective further received signal and the associated reference transmitted signal represent a relationship between a number of radar echoes as a function of a number of taps, which represent a characteristic value for the range of the vehicle interior monitoring system. In other words, the synchronization procedure described above is initially repeated for all uncorrelated radar images.

[0015] It is advisable to subject the determined channel impulse responses to a temporal alignment. The temporal alignment can be performed separately for each radar under consideration, as described above.

[0016] According to a second aspect, a computer program product is proposed which comprises instructions which, when the program is executed by a computing unit, cause the computing unit to carry out the method according to one or more embodiment variants.

[0017] According to a third aspect, a device for operating a radar-based vehicle interior monitoring system is proposed, wherein the vehicle interior monitoring system comprises, as radar sensors, a first UWB sensor with at least one antenna and a second UWB sensor with at least one antenna, as well as an evaluation unit. The first UWB sensor, the second UWB sensor, and the evaluation unit are connected to one another for exchanging data via a data bus. The first and second UWB sensors are configured to be operated as bi-static radar, wherein radar pulses emitted by the first UWB sensor are received as radar echoes by the second UWB sensor.The evaluation unit is configured to synchronize the radar pulses emitted by the first UWB sensor with the radar echoes received by the second UWB sensor before the channel impulse response is determined from the emitted radar pulses and the radar echoes.

[0018] In further embodiments, the device is configured to carry out one or more embodiments of the method according to the invention.

[0019] The invention is explained in more detail below using an exemplary embodiment in the drawing. They show: Fig. 1 a schematic representation of a first embodiment of a device according to the invention for operating a radar-based vehicle interior monitoring system; Fig. 2 a flow chart showing the essential process steps for operating the Fig. 1 illustrated device; Fig. 3 shows an uncorrelated exemplary radar image based on a plurality of radar echoes of a radar pulse repeatedly transmitted by a UWB sensor within a predetermined time period, the radar echoes of which were received by a second UWB sensor, as a function of a number of taps; Fig. 4a and Fig. 4b a schematic representation of a received signal extracted from the uncorrelated radar image, once not interpolated and once interpolated; Fig. 5a and Fig. 5b a comparison of a received signal and associated reference transmitted signal, once before a range adjustment and once after; Fig. 6a and Fig. 6b a comparison of a received signal and associated reference transmitted signal, once before a temporal alignment and once after; Fig. 7 a schematic representation of a second embodiment of a device according to the invention for operating a radar-based vehicle interior monitoring system; Fig. 8 a flow chart showing the essential process steps for operating the Fig. 7 illustrated device.

[0020] Fig. Figure 1 shows a schematic representation of a first embodiment of an apparatus according to the invention for operating a radar-based vehicle interior monitoring system. The vehicle interior monitoring system comprises a first UWB sensor 10, a second UWB sensor 20, and an evaluation unit 30. The first UWB sensor 10, the second UWB sensor 20, and the evaluation unit 30 are connected to one another via a data bus 40 for data exchange.

[0021] Contrary to the graphic representation, the evaluation unit 30 does not need to be a separate evaluation unit from the UWB sensors 10 or 20. Rather, the evaluation unit 30 could also be an internal evaluation unit of one of the two UWB sensors 10, 20. Regardless of this, however, the first and second UWB sensors 10, 20 are connected to each other via the data bus 40 for data exchange.

[0022] The first and second UWB sensors 10, 20 are essentially identical in design. Thus, the first UWB sensor 10 comprises, in a known manner, two antennas 11, 12 coupled to a transmit / receive unit 13. For example, antenna 11 serves as the transmitting antenna (Tx) and antenna 12 serves as the receiving antenna (Rx).

[0023] Similarly, the second UWB sensor 20 comprises two antennas 21, 22 coupled to a transmit / receive unit 23. For example, antenna 21 serves as a receive antenna (Rx) and antenna 22 serves as a transmit antenna (Tx).

[0024] The respective transmitting / receiving units 13, 23 are designed to repeatedly transmit radar pulses RI via the respective transmitting antenna 11 and / or 22 within a predetermined time period ZS and to transmit the radar echoes RE received by the antenna 12 and / or 21 to a respective sensor computing unit (not shown) for processing.

[0025] The radar echoes RE are created by reflection of the radar pulses RI from static or moving objects, which are Fig. 1 are schematically marked with the reference numeral 50.

[0026] The first and second UWB sensors 10, 20 are arranged in a vehicle interior (not shown in detail) for monitoring the interior, preferably under the roof lining (not shown) of a motor vehicle. For example, the first UWB sensor 10 is arranged near the interior mirror under the roof lining and oriented such that the radar pulses it emits are directed towards the front and rear seats. By processing the radar echoes RE reflected by the objects (static objects of the motor vehicle and / or vehicle occupants), the presence of occupants can then be deduced. For processing, a large number of radar echoes RE are processed within the predetermined time period as a function of a number of taps TP.

[0027] For example, the second UWB sensor 10 is arranged in the area of ​​the front seat backrests under the headliner and oriented such that the radar pulses RI it emits are directed toward the rear seats. By processing the radar echoes RE reflected by the objects (static objects of the motor vehicle and / or vehicle occupants), the presence of occupants in the rear seats can then be determined. For processing, a plurality of radar echoes RE are processed within the specified time period ZS and radar pulses RI repeatedly emitted as a function of a number of taps TP.

[0028] The processing of the radar pulses RI and associated radar echoes RE results in a radar image that can be Fig. 3. A respective radar image is generated by the evaluation unit 30, wherein the radar pulses RI and the associated radar echoes RE are transmitted to it from the UWB sensors 10, 20. The evaluation unit 30 typically comprises a computing unit, an image processing unit, and a data memory. The aforementioned components (not shown) are used to receive the signals as a data stream and process them with the image processing unit to create the radar image. The image processing unit comprises, for example, a machine learning algorithm, in particular a neural network (convolutional neural network, CNN), which is suitable for evaluating images. As part of the evaluation, it is then determined whether or not a vehicle occupant is present in the vehicle interior.The type of image evaluation is of secondary importance for the method described here, so that reference is made to the analysis options known from the state of the art.

[0029] In addition to the procedure described above, which is known in principle from the prior art, the invention provides for the first and second UWB sensors 10, 20 to be operated as a bi-static radar. Fig. 1 shows radar pulses RI emitted by the first UWB sensor 10 being received as radar echoes RE by the second UWB sensor 20. Corresponding information representing the radar pulses RI and the radar echoes RE is transmitted to the evaluation unit 30, which synchronizes the radar pulses RI emitted by the UWB sensor 10 with the radar echoes RE received by the second UWB sensor 20 before a channel impulse response (CIR) is determined from the emitted radar pulses RI and the radar echoes RE.

[0030] The starting point for the synchronization of the radar pulses RI emitted by the first UWB sensor 10 with the radar echoes RE received by the second UWB sensor 20 is an uncorrelated radar image UCIR 11-21 , as exemplified in Fig. 3. The index "11-21" means that radar pulses RI were transmitted by the antenna 11 of the first UWB sensor 10 and received by the antenna 21 of the second UWB sensor 20.

[0031] The Fig. 3 shown uncorrelated radar image UCIR 11-21is generated by the evaluation unit 30. It comprises a plurality of radar echoes RE1, ..., REn, which were received consecutively within a predetermined time period ZS from t = 0 to t ≈ 1200 by the second UWB sensor 20. The suffix "n" stands for the nth received radar echo. The amplitude curve per time slot or radar echo REi (i = 1, ..., n) is a function of the number of taps TP, where the taps are designated, for example, by a number from z = 0 to z = 65. The larger the tap number TPz, the greater the distance of an object 50 from which the radar pulse RI was reflected. The tap number thus represents a characteristic value for the range of the radar system and indicates the distance up to which data has been or can be recorded. The representation of the uncorrelated radar image UCIR 11-21is merely exemplary, taking into account that there is no correlation between the radar pulses RI emitted by the first UWB sensor 10 and the radar echoes RE received by the second UWB sensor 20.

[0032] In order to perform this correlation or synchronization, a reception signal ES is derived from the uncorrelated radar image UCIR 11-21 extracted. This represents the flow chart of the Fig. 2 represents the process step S10. The received signal ES represents a relationship between a number of radar echoes RE as a function of a number of taps TP (see Fig. 4a). Fig. 4a, in the course of the received signal ES each circle represents a data point from the uncorrelated radar image UCIR 11-21from which the received signal ES is then generated. To improve the accuracy of the method, an optional interpolation, e.g. by a factor of 10, can be performed, so that ten times the number of data points is available for further processing. This is exemplified in Fig. 4b, where the received signal is labeled ESi (where the index “i” stands for interpolated).

[0033] In a next step S14, a range comparison of the extracted (and optionally interpolated) received signal ES or ESi is carried out with a reference transmitted signal RS, which is determined from one or more radar pulses RI emitted by the first UWB sensor 10. This procedure is described in the Fig. 5a and Fig. 5b visualized, where the figures represent the reference transmission signal RS and the reception signal ES as a relationship between the number of radar echoes RE as a function of the number of taps TP.

[0034] As from Fig. 5a, the reference transmission signal RS and the reception signal ES have a similar course, but are shifted with respect to their tap number TPz and their number of radar echoes RE ( Fig. 5a). In step S14, an adjustment is performed for a data point in the reference transmission signal RS or the reception signal ES that has the highest tap count TPz. After the temporal adjustment, the tap count TPz for this data point is identical ( Fig. 5b). The tap count TPz for the other data points is adjusted relatively by the same factor.

[0035] In step S16, a time alignment of the reference transmission signal RS11 and the reception signal ES21 is carried out. This procedure is described in the Fig. 6a and Fig. 6b, where the figures show the reference transmission signal RS and the reception signal ES as a relationship between the number of radar echoes RE as a function of the number of taps TP. The temporal alignment is achieved by shifting the first local maximum in the reference transmission signal RS or the reception signal ES along the x-axis RE until the local maxima of the first local maximum overlap (see Fig. 6b). This is followed by optional downsampling of the correlated signals and generation of the channel impulse response determined from the transmitted radar pulses and the radar echoes.

[0036] Fig. Figure 7 shows a second embodiment in which the first and / or second UWB sensors 10, 20 are each operated monostatically in addition to the bistatic operation. This means that radar pulses RI10 emitted by the first UWB sensor 10 are received by its antenna 12 as radar echoes RE10. An additional uncorrelated radar image can be generated from this information. This radar image can be generated by a processing unit of the first UWB sensor 10 or the evaluation unit 30.

[0037] The same applies to radar pulses RI22 emitted by the second UWB sensor 20, which are received as radar echoes RI20 by the antenna 21 of the second UWB sensor 20. Another uncorrelated radar image can be generated from the radar pulses RI20 and the associated radar echoes RE20. This radar image can be generated by a processing unit of the second UWB sensor 10 or the evaluation unit 30.

[0038] The uncorrelated radar images, which are not shown in the figures, have a similar shape as shown in Fig. 3 for the bi-static operation of the device.

[0039] In addition to the measures taken in connection with Fig. 1 and Fig. 2 described steps are used to operate the device from Fig.7 In a step S20, respective received signals ES (as above) and additionally ES10 and / or ES20 are extracted from the two or three (generally m) uncorrelated radar images UCIR. In step S22, a temporal alignment of the uncorrelated radar images is carried out as a preprocessing step, whereby the previously described method of orientation towards the first local maximum with the largest tap number TPz can be used for this purpose. Steps S24, S26, and S28 are carried out for each uncorrelated radar image in a manner similar to steps S14, S16, and S18. In step S30, an output signal is then additionally generated from m correlated channel impulse responses CIR.

[0040] Interpolation and the final downsampling of averaged channel impulse responses can optionally be performed additionally.

[0041] The ability to operate the vehicle interior monitoring system, consisting of two UWB radar systems, in a bi-static mode improves performance compared to purely monostatic operation of each UWB sensor. In particular, it facilitates the differentiation between people inside the vehicle and people outside the vehicle at or near a vehicle window. List of reference symbols 10 first UWB sensor 11 first antenna 12 second antenna 13 Transmitting / receiving unit 20 second UWB sensor 21 first antenna 22 second antenna 23 Transceiver unit 30 evaluation unit 40 data bus 50 objects UCIR 11-21 uncorrelated radar image in bi-static operation RI radar echo RE radar echo ZS time span TP Tap TPz tap number with z=1...x ES extracted received signal RS reference transmit signal

Claims

[1] Method for operating a radar-based vehicle interior monitoring system, wherein the vehicle interior monitoring system comprises, as radar sensors, a first UWB sensor (10) with at least one antenna (11) and a second UWB sensor (20) with at least one antenna (21) as well as an evaluation unit (30), wherein the first UWB sensor (10), the second UWB sensor (20) and the evaluation unit are connected to one another for exchanging data via a data bus (40), in which - the first and second UWB sensors (10, 20) are operated as bi-static radar, wherein radar pulses emitted by the first UWB sensor (10) are received by the second UWB sensor (20) as radar echoes; and - the evaluation unit (30) synchronizes the radar pulses emitted by the first UWB sensor (10) with the radar echoes received by the second UWB sensor (20) before a channel impulse response is determined from the emitted radar pulses (RI) and the radar echoes (RE). [2] Method according to claim 1, characterized by in that for synchronization, a reception signal (ES) is extracted from an uncorrelated radar image (UCIR) generated from the radar pulses emitted by the first UWB sensor (10) and the radar echoes received by the second UWB sensor (20), which reception signal is compared with a reference transmission signal (RS), wherein the reception signal (ES) and the reference transmission signal (RS) represent a relationship between a number of radar echoes (RE) as a function of a number of taps (TP), which represent a characteristic value for the range of the vehicle interior monitoring system. [3] Method according to claim 2, characterized by that the received signal (ES) and the reference transmitted signal (RS) are adjusted in terms of the number of taps (TP). [4] Method according to claim 3, characterized by that a data point in the reference transmit signal (RS) or the receive signal (ES) which has the highest tap number (TP) is processed as a reference for the adjustment. [5] Method according to one of claims 2 to 4, characterized by that the received signal (ES) and the reference transmitted signal (RS) are aligned in time. [6] Method according to claim 5, characterized by that the first local maximum in the reference transmission signal (RS) or the reception signal (ES) is processed as a reference for the temporal alignment. [7] Method according to one of claims 2 to 6, characterized by that the extracted received signal (ES) is interpolated to a larger number of data points before being compared with the reference transmitted signal (RS). [8] Method according to one of the preceding claims, characterized by that the first and / or the second UWB sensor (10, 20) are additionally operated as a mono-static radar. [9] Method according to claim 8, characterized byin that for synchronization, a further reception signal (ES10, ES20) is extracted from at least one further uncorrelated radar image, which is generated from the radar pulses emitted by the first UWB sensor (10) and the radar echoes received by the first UWB sensor (10) and / or from the radar pulses emitted by the second UWB sensor (20) and the radar echoes received by the second UWB sensor (20), which is compared with an associated reference transmission signal (RS10, RS20), wherein the respective further reception signal (ES10, ES20) and the associated reference transmission signal (RS10, RS20) represent a relationship between a number of radar echoes (RE) as a function of a number of taps (TP), which represent a characteristic value for the range of the vehicle interior monitoring system. [10] Method according to claim 8 or 9, characterized by that the determined channel impulse responses are subjected to a temporal comparison. [11] Computer program product comprising instructions which, when the program is executed by a computing unit, cause the computing unit to carry out the method according to one of claims 1 to 10. [12] Device for operating a radar-based vehicle interior monitoring system, wherein the vehicle interior monitoring system comprises, as radar sensors, a first UWB sensor (10) with at least one antenna (11) and a second UWB sensor (20) with at least one antenna (21) as well as an evaluation unit (30), wherein the first UWB sensor (10), the second UWB sensor (20) and the evaluation unit are connected to one another for exchanging data via a data bus (40), in which - the first and second UWB sensors (10, 20) are configured to operate as a bi-static radar, wherein radar pulses emitted by the first UWB sensor (10) are received by the second UWB sensor (20) as radar echoes; and - the evaluation unit (30) is configured to synchronize the radar pulses (RI) emitted by the first UWB sensor (10) with the radar echoes (RE) received by the second UWB sensor (20) before the channel impulse response is determined from the emitted radar pulses and the radar echoes. [13] Device according to claim 12, characterized by that it is adapted to carry out the method according to one of claims 2 to 10.

Images