Method and device for providing data for monitoring the interior of a vehicle and system for monitoring the interior of a vehicle
By reducing the dimensions of the radar sensor data, low-dimensional second data is generated, which solves the problem of high vehicle bus load and realizes efficient vehicle internal space monitoring.
Patent Information
- Application Number
- CN202380090575.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2023-01-20
- Filing Date
- 2023-10-13
- Publication Date
- 2025-08-05
AI Technical Summary
When using radar sensors to monitor internal space in vehicles, the high data scale of sensor data results in high bus load on the vehicle bus, affecting data processing efficiency.
By dimensionality reduction of the original data of the radar sensor, a second data with a lower dimension is generated and transmitted over the vehicle bus to reduce data scale and bus load.
It realizes reducing data scale before transmission, reducing bus load, while retaining important information, suitable as input signals for classifiers and machine learning algorithms, and improving monitoring accuracy.
Smart Images
Figure CN120435667A_ABST
Abstract
Description
Technical Field
[0001] Various embodiments of the present invention relate to providing data for monitoring a vehicle interior and to monitoring a vehicle interior. In particular, these embodiments relate to a method and an apparatus for providing data for monitoring a vehicle interior, a method and a system for monitoring a vehicle interior, a vehicle, and a computer program. Background Art
[0002] Radar sensors can be used in vehicles to monitor the vehicle interior. For example, they can detect people or vehicle occupants, events, and the like within the vehicle interior. However, high levels of motion and frequency can occur during vehicle interior monitoring, resulting in a correspondingly high volume of sensor data from radar sensors. For further processing of this sensor data, it is conceivable to provide the sensor data to a vehicle control device, such as a controller. For this purpose, the sensor data can be transmitted to the control device, for example, via a vehicle bus. However, the high volume of sensor data results in a high bus load on the vehicle bus. Summary of the Invention
[0003] Therefore, there is a need to provide a possible approach for monitoring the interior of a vehicle with a radar sensor. This need is taken into account by the subject matter of the independent claims.
[0004] Embodiments relate to a method for providing data for monitoring a vehicle interior. The method includes obtaining first data, the first data including raw data related to the vehicle interior from a radar sensor, and the first data having a first dimension. Furthermore, the method includes performing a dimensionality reduction process on the first data to generate second data, the second data having a second dimension lower than the first data. Furthermore, the method includes providing the second data to a vehicle bus.
[0005] The proposed method allows for processing the raw data from radar sensors to reduce the data size before the corresponding data is transmitted. In particular, the second data, which has a compressed data size compared to the first data, generates a lower bus load on the vehicle bus than the first data, or the raw data, from the radar sensor. Furthermore, the second data is suitable as an input signal for classifiers, machine learning algorithms, neural networks, etc., because the dimensionality reduction process makes the first data clearer, for example, by efficiently summarizing the information contained in the first data. Furthermore, the dimensionality reduction process can also automatically identify and / or select features that contribute to the information content of the first data or are important for monitoring the vehicle interior, or focus the second data on helpful or important features, making the second data well-suited as an input signal for classifiers, machine learning algorithms, neural networks, etc.
[0006] As used herein, a radar sensor may be any type of radio-based sensor that transmits a signal and receives an echo reflected from an object as a corresponding signal, thereby, for example, detecting or locating the object. For example, the radar sensor may be an ultra-wideband (UWB) sensor configured for radar operation. A radar sensor may also be configured to detect vital signs of vehicle occupants, such as breathing. The radar sensor may be positioned within the vehicle interior. Multiple radar sensors may also be provided to monitor the vehicle interior.
[0007] A "dimensionality reduction process" can be any type of mathematical, statistical, or other dimensionality reduction process, method, or technique, and / or selection of important features, suitable for reducing the size of first data without losing too much information, maintaining sufficient information content, or preserving important information. Dimensionality reduction thus reduces the dimensionality and, therefore, the size of the (first) data, while simultaneously preserving important information. Suitable for this purpose are any process, method, or technique that reduces a certain number of features in the corresponding data to a smaller set of original features or new features, where the new features can be linear or nonlinear combinations and / or functions of the original features. For example, the dimensionality reduction process can include and / or utilize principal component analysis (PCA), which can alternatively or additionally include and / or utilize factor analysis, canonical correlation analysis, correspondence analysis, multidimensional scaling, random forests, or combinations thereof. The first dimension and the second dimension can also be understood as the first data size and the second data size, where the second data size is smaller than the first data size. Accordingly, the smaller second data size results in a lower bus load than the first data.
[0008] The vehicle bus may be any type of bus system, for example CAN, LIN, FlexRay, MOST, etc. The second data may be provided for the vehicle bus via a corresponding bus interface.
[0009] The method may be executed by any processor (eg, digital signal processor, etc.), computer, controller, etc. of the vehicle. In addition, the method may be implemented in hardware, software, or a combination thereof.
[0010] In some embodiments, the dimensionality reduction process can reduce the set of first features of the first data and / or combine one or more first features into a set of second features, and generate second data using the reduced set of first features and / or the reduced set of second features. In other words, the dimensionality reduction process can automatically combine important features and / or reduce the first data to such important features, that is, in other words, generate a feature set that is meaningful for further processing of the sensor data.
[0011] In some embodiments, the dimensionality reduction process can identify and modify a certain number of amplitude-related and / or phase-related signal features in the first data, and generate the second data using these amplitude-related and / or phase-related signal features. These signal features of the radar sensor's raw data, i.e., the first data, allow for precise monitoring of the vehicle interior while maintaining a relatively small data size.
[0012] In some embodiments, the dimensionality reduction process can include principal component analysis (PCA). Although other aforementioned processes, methods, or techniques for dimensionality reduction can also be used, principal component analysis has proven to have particularly good properties with respect to the data size of the second data and, therefore, the information content available for further processing of the sensor data.
[0013] In some embodiments, the second data may include one or more information for or derived from the first data, the information selected from the group consisting of: mean, variance, curvature, skewness, maximum value, minimum value, coefficient of variation, span, value of the arg max operator, standard deviation, peak intercept, peak magnitude, peak phase, peak characteristic, peak width, zero-crossing rate, FFT maximum amplitude, FFT maximum frequency, entropy of power spectral density, spectral flatness, autocorrelation width, autocorrelation peak, sine frequency, sine amplitude, sine R², main speed, main speed magnitude, speed center of gravity, speed curvature, speed skewness, arrival time difference, arrival phase difference, and arrival angle. This information enables good monitoring of the vehicle interior with a small data size or low bus load.
[0014] It should be noted that the dimensionality reduction process or the generation of secondary data can not only reduce the raw data of the radar sensor to important features, but the method can also include an automatic selection of the most important information, features, etc., wherein this selection can be specifically carried out for the corresponding interior scene to be detected.
[0015] In some embodiments, the second data may be designed or configured to be processed by a monitoring process utilizing a classifier, a machine learning algorithm, or the like, which is trained using training data having the second dimension. For example, the classifier, machine learning algorithm, or the like may receive the second data via a vehicle bus and use the second data as an input signal, and evaluate or further process the second data based on its training.
[0016] Furthermore, embodiments relate to a method for monitoring a vehicle interior. The method includes providing second data according to the aforementioned method for providing data for monitoring a vehicle interior. The method also includes receiving the second data via a vehicle bus. The method also includes executing a vehicle interior monitoring process using the second data.
[0017] As described above, the proposed method enables high-precision detection and / or monitoring of the vehicle interior, persons or vehicle occupants located therein, their vital signs, etc., while resulting in the lowest possible bus load on the vehicle bus.
[0018] In some embodiments, the monitoring process can utilize a classifier, machine learning algorithm, or the like, trained using training data having a second dimension. For example, the algorithm used in the monitoring process can be trained using real or simulated training data (which is similar to the second data, e.g., having a similar or identical data structure, similar or identical features or feature sets, etc.). This allows for precise detection or monitoring of the vehicle interior.
[0019] In some embodiments, a classifier, machine learning algorithm, or the like can be trained to detect objects, people, or vehicle occupants in the vehicle interior and / or properties related to objects, people, or the vehicle interior using the second data. For example, the algorithm can detect or assess the presence of people or vehicle occupants and / or their vital signs.
[0020] Embodiments further relate to a device for providing data for monitoring a vehicle interior. The device includes at least one data interface configured to obtain first data, wherein the first data includes, for example, raw data from a radar sensor related to the vehicle interior and has a first dimension. The device also includes processing logic configured to perform a dimensionality reduction process on the first data to generate second data, the second data having a second, lower dimension than the first data. The at least one data interface is further configured to provide the second data to a vehicle bus.
[0021] The proposed device is basically designed to carry out the method described above, so that the device can be further developed accordingly according to the method described here. The device enables precise detection or monitoring of the vehicle interior while simultaneously having a low bus load on the vehicle bus.
[0022] The device can be constructed separately or form a common structural unit with the radar sensor and / or be integrated therein. The processing logic can be any type of processor or data processor, etc. The data interface can be any type of input circuit, etc., through which sensor data, i.e., first data, can be obtained as raw data. The same data interface or another data interface can also include a vehicle bus module, such as a CAN module or a LIN module, via which the device can be coupled to the vehicle bus.
[0023] Embodiments further relate to a system for monitoring a vehicle interior. The system includes the aforementioned device and a control device configured to receive second data via a vehicle bus and to perform a monitoring process for the vehicle interior using the second data. The control device can be configured, for example, to detect objects, persons, or vehicle occupants, their vital signs, and the like, i.e., to monitor the vehicle interior.
[0024] The proposed system enables evaluation of sensor data from radar sensors and precise detection or monitoring of the vehicle interior with low bus load on the vehicle bus.
[0025] The control device can be, for example, a controller of the vehicle and accordingly have a vehicle bus module, such as a CAN module, a LIN module, etc., in order to obtain or receive the second data therefrom. Furthermore, the control device can have processing logic, a processor, etc. and be designed to receive the second data and evaluate it for monitoring the vehicle interior.
[0026] In some embodiments, the control device may include a classifier or machine learning algorithm selected from the group consisting of support vector machines, SVMs, perceptrons, multilayer perceptrons, MLPs, and convolutional neural networks to implement the monitoring process. For example, the classifier may be trained using real or simulated training data (which is similar to the second data, e.g., having a similar or identical data structure, features, or feature sets). This allows for accurate detection or monitoring of the vehicle interior.
[0027] In some embodiments, the control device can also be designed to control at least one vehicle function based on the monitoring process. The vehicle function can be any type of function for which the vehicle interior is monitored. For example, the vehicle function can include vehicle entry, door opening and / or door closing, air conditioning of the vehicle interior, a vehicle alarm, etc. Furthermore, the monitoring process or the at least one vehicle function can advantageously include driver monitoring, detection of children, dogs, etc. left in the vehicle, wherein detection of left-behind children, dogs, etc. can be configured to avoid emergency situations in a heated vehicle interior and can include corresponding warnings to the vehicle operator or vehicle owner, activation of the air conditioning, opening of the windows, etc.
[0028] These embodiments also relate to a computer program comprising instructions that, when executed by a processor, a computer, processing logic, or the like, cause the program to perform the method for providing data for monitoring a vehicle interior and / or the method for monitoring a vehicle interior, and / or implement the apparatus for providing data for monitoring a vehicle interior. BRIEF DESCRIPTION OF THE DRAWINGS
[0029] Next, various embodiments are described in detail with reference to the accompanying drawings.
[0030] Figure 1 A schematic block diagram shows an apparatus for providing data for monitoring a vehicle interior according to one embodiment;
[0031] Figure 2 A schematic side view shows a vehicle having a device for providing data for monitoring a vehicle interior or having a system for monitoring a vehicle interior according to one embodiment; and
[0032] Figure 3 A method for providing data for monitoring a vehicle interior according to one exemplary embodiment is shown in a flow chart.
[0033] Figure 4 A method for monitoring a vehicle interior according to one exemplary embodiment is shown in a flow chart. DETAILED DESCRIPTION
[0034] Various embodiments will now be described in more detail with reference to the accompanying drawings, in which some embodiments are shown. In the accompanying drawings, the thickness and size of lines, layers and / or regions may be exaggerated for clarity.
[0035] Figure 1 A schematic block diagram shows a device 100 for providing data for monitoring a vehicle interior. The device 100 is installed in a vehicle 1 (see FIG. Figure 2) and is coupled to at least one radar sensor 200 and a vehicle bus 300 , wherein a control device 400 is also coupled to vehicle bus 300 .
[0036] Device 100 includes at least one data interface 110 and processing logic 120. The at least one data interface 110 is designed to obtain first data 111, wherein the first data 111 describes, includes, etc., raw data from the at least one radar sensor 200 related to the vehicle interior and has a first dimension. The raw data from the wheel sensor 200 is high-dimensional due to the information content contained therein and therefore has a very large data size. Processing logic 120 is designed to perform a dimensionality reduction process on the first data 111 to generate second data 112, wherein the second data 112 has a second dimension lower than that of the first data 111. The dimensionality reduction process is designed to reduce the dimensionality of the first data 111 while retaining important information. In addition, the at least one data interface 110 is also designed to provide the second data 112 to the vehicle bus 300 or transmit the second data via the vehicle bus 300.
[0037] The device 100 can be constructed separately or form a common structural unit with the radar sensor 200 and / or be integrated into the radar sensor. The processing logic 120 can be any type of processor or data processor, etc. The data interface 110 can be any type of input circuit, communication module, etc., through which sensor data, that is, the first data 111, can be obtained or received as raw data. When the device 100 and the radar sensor 200 form a common structural unit, the first data 111 can be received, for example, via a conductor track, a circuit, etc. In contrast, when the device 100 and the radar sensor 200 are constructed separately, the device and the radar sensor can be connected to each other by cable. The same or another data interface 110 can also include a vehicle bus module, such as a CAN module, a LIN module, etc., via which the device 100 can be coupled to the vehicle bus 300.
[0038] In some embodiments, the dimensionality reduction process implemented by processing logic 120 can reduce the set of first features of first data 111 and / or combine one or more first features into a set of second features, and generate second data 112 using the reduced set of first features and / or the reduced set of second features. Thus, the dimensionality of first data 111 can be reduced while retaining important information. In this embodiment, the dimensionality reduction process is, by way of example, principal component analysis or the use of principal component analysis (PCA). Alternatively or additionally, factor analysis, canonical correlation analysis, correspondence analysis, multidimensional scaling, random forests, or combinations thereof may also be used.
[0039] In some embodiments, the dimensionality reduction process may identify and modify a number of amplitude-related signal features and / or phase-related signal features in the first data 111, and generate the second data 112 using the signal features. For example, the second data 112 may include information about the first data, the information selected from the group consisting of: mean, variance, curvature, skewness, maximum value, minimum value, coefficient of variation, span, value of the arg max operator, standard deviation, peak intercept, peak magnitude, peak phase, peak feature, peak width, zero-crossing rate, FFT maximum magnitude, FFT maximum frequency, entropy of power spectral density, spectral flatness, autocorrelation width, autocorrelation peak, sine frequency, sine amplitude, sine R², main velocity, main velocity magnitude, velocity centroid, velocity curvature, velocity skewness, arrival time difference, arrival phase difference, and arrival angle.
[0040] The at least one radar sensor 200 may be any type of radio-based sensor that transmits a signal and receives an echo reflected from an object as a corresponding signal, thereby detecting, locating, or otherwise detecting the object. For example, the at least one radar sensor 200 may be an ultra-wideband (UWB) sensor configured for radar operation. The at least one radar sensor 200 may also be configured to detect vital signs of vehicle occupants, such as breathing. The radar sensor may be positioned within the vehicle interior. Multiple radar sensors may also be provided to monitor the vehicle interior.
[0041] Figure 2 A vehicle having the above-described device 100 for providing data for monitoring the vehicle interior or having a system for monitoring the vehicle interior, which system has the device 100 and the above-mentioned control device 400 , is shown in a schematic side view.
[0042] As mentioned above, the device 100 is coupled to the at least one radar sensor 200, which is designed and arranged to detect or monitor the vehicle interior of the vehicle 1. As mentioned above, a plurality of radar sensors 200 may also be provided, although Figure 2 A single radar sensor is shown for exemplary purposes only. Device 100 receives first data 100 via at least one data interface 110 , for example, raw data from at least one radar sensor 200 . For exemplary purposes only, at least one radar sensor 200 is an ultra-wideband (UWB) sensor that operates in radar mode. Furthermore, device 100 , having at least one data interface 110 , is coupled to the aforementioned vehicle bus 300 . A control device 400 is in turn coupled to vehicle bus 300 .
[0043] Device 100 described herein is designed to process first data 111 obtained from the at least one radar sensor 200 such that second data 112, which is then transmitted via vehicle bus 300, contains information relevant for monitoring the vehicle interior, but the data size of second data 112 is reduced compared to first data 111, i.e., the raw data of the at least one radar sensor 200. This allows the bus load on vehicle bus 300 to be kept low while simultaneously enabling precise detection and / or monitoring of the vehicle interior.
[0044] Control device 400 is designed to receive second data 112 via vehicle bus 300 and to perform a monitoring process for the vehicle interior using second data 112. Control device 400 may, for example, be designed to detect objects, persons, or vehicle occupants, their vital signs, etc., i.e., monitor the vehicle interior and trigger or control corresponding vehicle functions based on evaluation of second data 112. These vehicle functions may include, for example, alarms, air conditioning, vehicle entry, door operation, etc., of vehicle 1. Control device 400 may, for example, be a vehicle controller and may accordingly include a vehicle bus module, such as a CAN module or a LIN module, to thereby obtain or receive second data 112. Control device 400 may also include processing logic, a processor, etc., and may be designed to receive second data 112 and evaluate it for monitoring the vehicle interior. To perform the monitoring process, control device 400 may include a classifier, a machine learning algorithm, etc., selected from the group consisting of a support vector machine, an SVM, a perceptron, a multilayer perceptron, an MLP, and a convolutional neural network. For example, the classifier may be trained with the aid of real or simulated training data that is at least similar to the second data 112 (eg, has a similar or identical data structure, similar or identical features or feature sets, etc.).
[0045] Figure 3 A flow chart illustrates a method for providing data for monitoring a vehicle interior. The method can be implemented, for example, using the aforementioned apparatus 100. The method includes obtaining (S110) first data 111, which represents or includes raw data related to the vehicle interior from a radar sensor 200 and has a first dimension. The method also includes performing (S120) a dimensionality reduction process on first data 111 to generate second data 112, which has a lower second dimension than first data 111. Furthermore, the method includes providing (S130) second data 112 to a vehicle bus 300.
[0046] Figure 4 The method for monitoring the interior of a vehicle is shown in a flow chart. The method can be implemented, for example, with the above-mentioned system having the device 100 and the control device 400. The method includes the following steps: Figure 3) provides ( S210 ) second data ( S112 ). Furthermore, the method includes receiving ( S220 ) the second data 112 via the vehicle bus 300 . Furthermore, the method includes implementing ( S230 ) a monitoring process for the vehicle interior using the second data 112 . Optionally, the method also includes providing third data based on the monitoring process to control at least one vehicle function.
[0047] In addition, a computer program may be provided, which includes instructions that, when executed by a processor, such as the processing logic 120 of the device and / or the processing logic of the processor or the processing logic of the control device 400, causes it to perform one or more of the above methods and / or implement the device 100 or the above system.
[0048] Reference Signs List
[0049] 1 vehicle
[0050] 100 devices
[0051] 110 data interface
[0052] 120 processing logic (eg, processor)
[0053] 200 radar sensors
[0054] 300 vehicle bus
[0055] 400 control device
Claims
1. A method for providing data for monitoring a vehicle interior, the method comprising: Obtaining (S110) first data (111), the first data providing raw data of the radar sensor (200) related to the vehicle interior space and the first data having a first dimension, performing (S120) a dimensionality reduction process on the first data (111) to generate second data (112), the second data having a lower second dimension than the first data (111), and The second data (112) is provided (S130) to the vehicle bus (300).
2. The method according to claim 1, wherein The dimensionality reduction process reduces the group of first features of the first data (111) and / or combines one or more first features into a group of second features, and generates the second data (112) with the help of the reduced group of first features and / or the reduced group of second features.
3. The method according to claim 1 or 2, wherein: The dimensionality reduction process identifies and modifies a certain number of amplitude-related signal features and / or phase-related signal features in the first data (111), and generates the second data (112) with the aid of the amplitude-related signal features and / or phase-related signal features.
4. A method according to any one of the preceding claims, wherein The dimensionality reduction process includes principal component analysis PCA.
5. A method according to any one of the preceding claims, wherein The second data (112) includes information about the first data (111), the information being selected from the group consisting of: mean, variance, curvature, skewness, maximum value, minimum value, coefficient of variation, span, value of an arg max operator, standard deviation, peak intercept, peak magnitude, peak phase, peak characteristic, peak width, zero crossing rate, FFT maximum amplitude, FFT maximum frequency, entropy of power spectral density, spectral flatness, autocorrelation width, autocorrelation peak, sine frequency, sine amplitude, sine R², main velocity, main velocity magnitude, velocity center of gravity, velocity curvature, velocity skewness, arrival time difference, arrival phase difference, and arrival angle.
6. A method for monitoring a vehicle interior, the method comprising: providing (S210) second data (S112) according to the method of any one of the preceding claims, receiving (S220) the second data (112) via the vehicle bus (300), and A monitoring process of the vehicle interior is carried out ( S230 ) using the second data ( 112 ).
7. The method according to claim 6, wherein: The monitoring process uses a classifier that is trained using training data having the second dimension.
8. The method according to claim 7, wherein: The classifier is trained to detect objects, persons, and / or object-related or person-related properties in the vehicle interior using the second data.
9. The method according to any one of claims 6 to 8, further comprising: Based on the monitoring process, third data are provided for controlling at least one vehicle function.
10. A device (100) for providing data for monitoring a vehicle interior, the device comprising: At least one data interface (110) is designed to obtain first data (111), wherein: The first data (111) provides raw data of the radar sensor (200) related to the vehicle interior space and has a first dimension; and processing logic (120) configured to perform a dimensionality reduction process on the first data (111) to generate second data (112), the second data having a second dimension lower than that of the first data (111), The at least one data interface (110) is further designed to provide the second data (112) to the vehicle bus (300).
11. A system for monitoring a vehicle interior, the system comprising: The device (100) according to claim 10, and A control device (400) is designed to receive the second data (112) via the vehicle bus (300) and to carry out a monitoring process for the vehicle interior using the second data (112).
12. The system according to claim 11, wherein The control device includes a classifier for carrying out the monitoring process, the classifier being selected from the group consisting of a support vector machine, an SVM, a perceptron, a multilayer perceptron, an MLP, and a convolutional neural network.
13. The system according to claim 11 or 12, wherein: The control device (400) is further designed to control at least one vehicle function based on the monitoring process.
14. A vehicle comprising a device according to claim 10 or a system according to any one of claims 11 to 13.
15. A computer program comprising instructions which, when executed by a processor, cause the processor to carry out the method according to any one of claims 1 to 5 and / or the method according to any one of claims 6 to 9 and / or to implement the apparatus according to claim 10.