METHOD FOR DETECTING A FOOTSPACE OCCUPANCY STATE

Abstract

In a method for detecting the footwell occupancy state of a seating arrangement with at least one seat, a data set for the footwell occupancy state is acquired by means of a radar scan of a spatial area surrounding the seating arrangement, including at least one footwell, at least partially. The radar scan is performed using a radar sensor located above the seating arrangement. A footwell occupancy state is then determined using an evaluation model. The measurement data are acquired for at least two consecutive time points, including the current time point. For the current time point and at least one further consecutive time point, a confidence value for the footwell occupancy state of the at least one footwell is determined, indicating the probability that the actual footwell occupancy state is "occupied" or "unoccupied."The measurement data from the point in time at which a footwell occupancy state determined by the evaluation model is more likely to correspond to the actual footwell occupancy state are used.

Description

[0001] The present invention relates to a method, a system configured to carry out the method and a computer program, each for detecting a footwell occupancy state for a seating arrangement with at least one seat, in particular in a vehicle.

[0002] In various situations, it may be necessary to automatically determine the current occupancy status of a seating arrangement with at least one seat. Such a situation can occur particularly in vehicles, for example, in motor vehicles, where vehicle configuration or the activation, deactivation, and / or control of one or more vehicle functionalities is to be carried out depending on the current occupancy status. For example, it is known in motor vehicles to issue an acoustic or visual warning to vehicle occupants to fasten their seat belts or to control the activation or deactivation of airbags, depending on a detected occupancy status.

[0003] Methods using radar technology are known for the automated detection of the current occupancy status of one or more seats, particularly the arrangement of seats in a vehicle. Radar sensors scan the vehicle interior, generating measurement data in the form of radar point clouds. The current seat occupancy can then be determined from the measured radar point cloud.

[0004] In addition to monitoring seat occupancy, it may also be necessary to monitor the footwell, or even the individual footwells assigned to each seat in the seating arrangement. This could be used to check whether objects or people are in the rear footwell before a front seat is automatically adjusted. This prevents objects or people's feet / legs from being trapped during automatic seat adjustment. It can also simply help detect the presence of an object, such as a bag, left in a footwell.

[0005] Due to the distance to a radar sensor, which is usually mounted on the ceiling of the vehicle interior above the seating, and the obstruction caused by the passengers themselves, this part of the vehicle interior is more difficult to monitor and is easily affected by noise. This is why existing solutions use separate radar sensors in the footwell. However, this increases the cost of such a system. Therefore, it is often only possible to monitor the footwells of individual seats as a whole. This, however, results in the loss of information about which footwell is occupied or unoccupied.

[0006] It is an object of the present invention to provide an improved solution for detecting the occupancy status of the footwell in a seating arrangement with at least one seat, particularly in a vehicle. In particular, the improved solution is intended to achieve reliable monitoring of the footwells without additional radar sensors in the footwells.

[0007] The solution to this problem is achieved according to the teaching of the independent claims. Various embodiments and further developments of the invention are the subject of the dependent claims.

[0008] A first aspect of the solution presented here concerns a method, particularly a computer-implemented one, for detecting the footwell occupancy status of a seating arrangement with at least one seat, especially in or for a vehicle, such as an automobile (e.g., truck, car, or bus). The method involves acquiring a data set for the footwell occupancy status, where the data set represents measurement data for at least a current point in time. This measurement data represents an associated radar point cloud, which was or is obtained based on a radar scan of a spatial area surrounding the seating arrangement, including at least a section of the footwell. The radar scan is performed using a radar sensor located above the seating arrangement.A footwell occupancy status is determined using an evaluation model. The data set serves as input data for the evaluation model, which, based on the radar point cloud and the acquired data set, outputs a footwell occupancy status for at least one footwell at the current time. Footwell occupancy status information, defined based on the evaluation result, is then displayed.

[0009] The measurement data are recorded for at least two consecutive time points, including the current time point, whereby a confidence value for a footwell occupancy state of the at least one footwell is determined for the current time point and at least one further of the consecutive time points, which indicates the probability with which the actual footwell occupancy state is "occupied" or "unoccupied", wherein the measurement data of the time point for which a footwell occupancy state determined by means of the evaluation model corresponds with a higher probability to an actual footwell occupancy state are used as input data for the evaluation model for determining the footwell occupancy state.

[0010] The term "footwell occupancy status" used here refers specifically to information indicating whether or to what extent a footwell or multiple footwells are occupied by an object, particularly an item such as a bag, or by a person or their feet or legs. Specifically, each seat in the seating arrangement is assigned a footwell, and the "footwell occupancy status" can indicate individually for each footwell whether it is occupied or free. Since a footwell is typically assigned to a seat, the footwell occupancy status can be described as "for a seating arrangement with at least one seat."

[0011] The term "seat occupancy status" used here refers specifically to information indicating whether, or to what extent, the seating arrangement, or at least one of its seats, is occupied by an object, particularly a thing or a person. In a simple example, the seat occupancy status can simply indicate the presence or absence of an object. In a more advanced example, if at least one object is present on the seating arrangement, or on one or more of its seats, it can provide information about the type or other property of the object, such as its spatial extent.

[0012] The term "radar point cloud" as used here refers specifically to a set of points in a vector space obtained by radar scanning of at least one object's surface, exhibiting a typically unorganized spatial structure ("cloud"). In the case of a radar point cloud, the points within the radar point cloud can be called "radar points." A (radar) point cloud can be described, in particular, by the (radar) points it contains. The radar points, in turn, can each be described, in particular, by their spatial coordinates, which specify, for each radar point, the location of the reflection of an emitted radar signal from an object's surface, as measured during the radar scan. Additional attributes, such as the measured Doppler velocity or the signal-to-noise ratio (SNR), can be recorded for each radar point.

[0013] The term "evaluation model" used here refers to a model, particularly a mathematical one, that uses a radar point cloud or one or more characteristics or parameters characterizing it as input(s) to deliver an evaluation result based on these parameters, in this case, one of several predefined possible seat occupancy states of the seating arrangement. The evaluation model can, in particular, be a mathematical estimator, where the radar point cloud represents empirical data as a sample and the evaluation result is an estimated value determined based on this sample. The evaluation model can, in particular, be a "machine learning model" (or...The term "machine learning model" (ML model) is synonymous with "machine learning model," which, in this context, refers specifically to a mathematical, particularly statistical, model created using at least one machine learning algorithm based on example data referred to as training data. This model is used to make predictions or decisions without the algorithm(s) being explicitly programmed to make such predictions or decisions. Decision tree-based machine learning models and artificial neural networks are particularly relevant examples of machine learning models.

[0014] The terms “comprises”, “includes”, “includes”, “has”, “with”, or any other variant thereof, as used, are intended to cover non-exclusive inclusion. For example, a method or apparatus that includes or has a list of elements is not necessarily limited to those elements, but may include other elements not expressly listed or inherent in such method or apparatus.

[0015] Furthermore, unless explicitly stated otherwise, "or" refers to an inclusive or and not an exclusive "or". For example, a condition A or B is satisfied by one of the following: A is true (or present) and B is false (or not present), A is false (or not present) and B is true (or present), and both A and B are true (or present).

[0016] The terms "ein" or "eine," as used here, are defined as "one or more." The terms "ein anderer" and "ein Weitere," as well as any other variant thereof, are to be understood as "at least one more."

[0017] The term “plural” or “several”, as used here, is to be understood in the sense of “two or more”.

[0018] The terms "configured" or "set up" (and any variations thereof) used in connection with the invention mean that the device is already in a configuration or setting in which it can perform the function, or at least is adjustable—i.e., configurable—so that it can perform the function after appropriate adjustment. Configuration can be achieved, for example, by adjusting parameters of a process sequence or by using switches or similar devices to activate or deactivate functionalities or settings. In particular, the device can have several predetermined configurations or operating modes, allowing configuration by selecting one of these configurations or operating modes.

[0019] The method according to the first aspect is therefore based in particular on the fact that a footwell occupancy state for a seating arrangement with at least one seat can be detected without requiring one or more additional radar sensors in the footwell. In other words, the footwell occupancy state can be detected even though only one or more radar sensors, preferably a single radar sensor above the seating arrangement, are used.

[0020] Although the radar signal, as described earlier, can be affected by the distance to the radar sensor and potential obstructions, for example by a person, this is mitigated by recording the measurement data for at least two consecutive time points. The measurement data from the time point for which the result of the evaluation model most likely corresponds to the actual footwell occupancy is then used. In this way, the risk of information being lost, for example through an averaging filter, even with a weak or noisy signal, can be reduced. The reliability of detecting footwell occupancy can thus be increased, even though only one radar sensor (or several radar sensors) above the seating arrangement is used. In particular, no additional radar sensor in the footwell area is necessary.

[0021] The following describes various exemplary embodiments of the method, which, unless expressly excluded or technically impossible, can be combined with each other and with the other described aspects of the present solution.

[0022] In some embodiments, the confidence value is a value in the range of 0 to 1. This normalization of the confidence values ​​simplifies the indication of the probability that the actual footwell occupancy state is "occupied" or "unoccupied".

[0023] In some related embodiments, if the confidence values ​​are greater than or equal to 0.5, the measurement data from the time point at which the confidence value is at its maximum is used as input data for the evaluation model. Specifically, confidence values ​​greater than 0.5 indicate that a footwell is likely occupied. Therefore, if the confidence values ​​are greater than or equal to 0.5, the measurement data from the time point at which the confidence value is highest—that is, with the greatest probability that a footwell is occupied—is used. This increases the reliability in detecting that the footwell is occupied.

[0024] In some related embodiments, if the confidence values ​​are less than 0.5, the measurement data from the time point for which the confidence value is at its minimum is used as input data for the evaluation model. Conversely to the situation just described, confidence values ​​less than 0.5 indicate that a footwell is probably unoccupied, i.e., free. Therefore, by using the measurement data from the time point for which the confidence value is lowest, the reliability in detecting that a footwell is unoccupied can be increased.

[0025] In some related embodiments, if the confidence values ​​are both greater than and less than 0.5, the confidence values ​​of at least three consecutive time points, including the current time, are determined. If the majority of the confidence values ​​are greater than 0.5, the measurement data from the time point for which the confidence value is a maximum of the confidence values ​​is used as input for the evaluation model. If the majority of the confidence values ​​are less than 0.5, the measurement data from the time point for which the confidence value is a minimum of the confidence values ​​is used as input for the evaluation model. There may also be cases where confidence values ​​are both less than and greater than 0.5.Then, measurements from at least three consecutive time points are determined, and finally, the highest or lowest confidence level is selected, depending on whether more confidence levels are less than or greater than 0.5. This can increase the reliability in detecting whether a footwell is free or occupied.

[0026] In some embodiments, the seating arrangement comprises multiple seats, each with an associated footwell, and an individual footwell occupancy status is determined for each footwell. By using a radar sensor positioned above the seating arrangement, the footwells of several, and in particular all, seats—for example, the seats in the rear row of a vehicle—can be monitored. By selectively assigning radar points to a specific footwell, an occupancy status can be determined for each footwell individually.

[0027] In some embodiments, the method further includes detecting the occupancy status of the seating arrangement. A data set for a seat occupancy status is acquired, wherein the data set represents measurement data for at least a current point in time, and the measurement data represents the associated radar point cloud, which was or is acquired based on radar scanning. The occupancy status of the seating arrangement is determined using the evaluation model, where the data set forms input data for the evaluation model. Depending on the radar point cloud and the acquired data set, the evaluation model provides an output indicating the occupancy status of the seating arrangement at the current point in time. Seat occupancy status information, defined based on the evaluation result, is then output.Since the radar sensor is located above the seating arrangement, its field of view includes not only the footwells but also the seats. Therefore, a single measurement can determine not only the occupancy of the footwells but also the occupancy of the seats.

[0028] In some related embodiments, i.e., in the detection of seat occupancy, the measurement data are acquired for at least two consecutive time points, including the current time point. An average value is determined for the current time point and at least one further of the consecutive time points. The input data for the evaluation model for determining the seat occupancy includes this determined average value. In this way, outliers in the measurement data can be eliminated.

[0029] It should be noted that a different filter is used for the measurement data from different time points when detecting seat occupancy than when detecting footwell occupancy. While, as described above, it is advantageous for detecting footwell occupancy not to eliminate outliers, but rather to use them as input data for the evaluation model, the comparatively strong signal for detecting seat occupancy (especially due to the shorter distance to the radar sensor and the greater mass of a person's body compared to their feet or legs) makes it advantageous to eliminate outliers to avoid, for example, flickering.

[0030] Overall, the method described in the first aspect, based on the radar point cloud obtained by radar scanning of a spatial area surrounding the seating arrangement, allows for the generation of stable evaluation results (particularly in terms of prediction or classification) that characterize not only the footwell occupancy status but also the seat occupancy status of the seating arrangement. This enables the implementation of radar-based solutions, especially in the vehicle context (particularly for automobiles), that can reliably detect footwell and seat occupancy status (particularly exclusively) using radar.

[0031] In some embodiments, the determination of the footwell occupancy status also takes the seat occupancy status into account. For example, the probability that a seat's footwell is occupied may be increased if the corresponding seat is occupied by a person. However, it is also possible that a person has placed their legs / feet in the footwell of an adjacent seat. Particularly in the case of a middle seat in the rear row, due to the limited legroom of this seat, it is often the case that a person places their legs / feet in one or both of the adjacent footwells. It is also possible that a footwell is occupied by an object, while the corresponding seat is then unoccupied. For this reason, considering the seat occupancy status can help in detecting the footwell occupancy status, but it cannot be used as the sole criterion.

[0032] In some embodiments, a measure of motion activity at the current time is determined from Doppler shift values ​​in the measurement data, with the data set containing this determined measure of motion activity as input data for the evaluation model. Since a change in the footwell occupancy state or seat occupancy state is generally only possible if a minimum level of motion activity is detected at the current time, the probability of a change in the respective occupancy state can be reduced if no motion is detected. Motion activity can refer in particular to the movement activity within a vehicle, especially of one or more people. For example, a high level of motion activity occurs when getting in and out of a vehicle, whereas passengers in a vehicle typically move very little while it is in motion.

[0033] In some embodiments, the radar point cloud is segmented into several clusters by assigning a subset of the radar points from the respective radar point cloud to each seat and each footwell as a cluster, depending on their respective position, such that the radar points of the cluster lie within a defined closed spatial area, in particular a cuboid, in the vicinity of the footwell or seat. This enables particularly simple and computationally efficient cluster formation and thus seat-specific detection of the occupancy of the footwells or seats, whereby the location (position and orientation) and the shape of the spatial area are or can be defined such that they strongly overlap the spatial area typically occupied by a typical object to be detected, in particular a person, on a seat of the seating arrangement.In addition to the seating arrangement, a separate area for the footwells is also planned to be defined as a cluster, for example also in the form of a cuboid.

[0034] Clustering thus enables seat-specific, i.e., individual, seat occupancy detection or individual footwell occupancy detection in the case of a multi-seat seating arrangement. This is particularly advantageous or even necessary when a seat-specific response to the detected occupancy status is required, for example, by preventing automatic seat adjustment when the footwell is occupied, and / or by activating, deactivating, or otherwise controlling a specific functionality or system for a particular seat, such as a seat-specific airbag system, a seat-specific seatbelt warning, or a seat-specific seat heating, depending on its detected occupancy. In some embodiments, clustering can be implemented in such a way that the clusters are disjoint, so that no radar point is assigned to two different clusters.

[0035] In some embodiments, features (characteristic parameters) of the radar point cloud are used for evaluation and provided in the form of a feature vector. The evaluation model is thus defined in such a way that an evaluation result is determined depending on the respective values ​​of the features for the radar point clouds. In this way, the evaluation model can be defined and applied in a simplified manner, since only the values ​​of the features need to be considered as input variables instead of entire radar point clouds. In particular, the temporal progression of the values ​​can reflect a movement or a specific movement pattern of one or more objects in a footwell or on the seating arrangement, so that the evaluation model can determine the footwell occupancy status or the seat occupancy status of the seating arrangement in a particularly reliable manner.

[0036] In some embodiments, the individual radar points of the radar point cloud are each represented by a position of the respective radar point in three-dimensional space and by at least one of the following parameters: (i) a Doppler shift value of the radar signal relative to the respective radar point; (ii) a signal-to-noise ratio value of the radar signal relative to the respective radar point. These parameters can be used, in particular, for pre-filtering the radar point cloud as part of a pre-processing step preceding feature determination.

[0037] The information to be output can, in particular, represent the evaluation result itself. It can also be a detectable signal, especially one perceptible to a human sense, such as a warning, a control signal for activating a signal source, or a data signal carrying the information.

[0038] In some embodiments, outputting the information includes controlling a signal source based on the information, causing the signal source to output a defined signal in response to the control. The signal source can be, in particular, an audio source, an optical signal source (especially a display device for images or text), and / or a haptic actuator, or a combination of at least two of the aforementioned signal sources. Thus, the detected footwell occupancy or seat occupancy status can be communicated to a user or used to control another technical system.

[0039] In particular, the signal source can be controlled depending on the information so that it outputs a signal, defined in particular by the control, when the information results from an evaluation result according to which at least one footwell or seat of the seating arrangement is occupied and / or a selected predetermined footwell occupancy state or seat occupancy state exists.

[0040] In some embodiments, the at least one footwell includes at least one footwell assigned to a rear seat of the seating arrangement. A signal source is controlled based on the information from the evaluation result such that a signal output by the signal source prevents automatic rearward adjustment of a seat in the seating arrangement located directly in front of the footwell of the rear seat. This allows information about the footwell occupancy status to be taken into account during automatic (i.e., motorized) seat adjustment. In particular, this prevents an object or a person's feet from becoming trapped. Simultaneously, a warning, such as an acoustic or visual signal, can be issued indicating that the footwell behind the seat to be adjusted is occupied.

[0041] It goes without saying that the evaluation model, which is primarily a machine learning model, can be trained before its actual use. Datasets representing at least a radar point cloud or values ​​of one or more specific features at different times are provided to the machine learning model as input data. This allows for a particularly flexible and adaptable implementation of the evaluation model, whereby machine learning can be used to continuously improve the evaluation model and thus the quality and reliability of the process according to the first aspect. The machine learning model can, in particular, be based on an artificial neural network.

[0042] A second aspect of the present solution relates to a system for detecting a footwell occupancy state for a seating arrangement with at least one seat, wherein the system comprises a data processing device which is configured, in particular by means of a corresponding computer program, to detect a footwell occupancy state for a seating arrangement and, if applicable, to perform the method according to one of the preceding claims.

[0043] A third aspect of the present solution concerns a computer program or computer program product comprising instructions which, when executed on the data processing device of the system according to the second aspect, cause the system to execute the procedure according to the first aspect.

[0044] The computer program can be stored, in particular, on a non-volatile data carrier. Preferably, this is a data carrier in the form of an optical data carrier or a flash memory module. This can be advantageous if the computer program itself is to be handled independently of a processor platform on which the one or more programs are to be executed. In another implementation, the computer program can exist as a file on a data processing unit, in particular on a server, and be downloadable via a data connection, for example, the Internet or a dedicated data connection, such as a proprietary or local network. Furthermore, the computer program can comprise a plurality of interacting individual program modules. The modules can, in particular, be configured, or at least be usable, in such a way that they function in the sense of distributed computing (i.e., distributed computing)."Distributed computing" is performed on different devices (computers or processor units) that are geographically separated and connected via a data network.

[0045] The system described in the second aspect may accordingly have a program memory in which the computer program is stored. Alternatively, the system may also be configured to access an external computer program, for example on one or more servers or other data processing units, via a communication link, in particular to exchange data with it that is used during the execution of the procedure or computer program or represents outputs of the computer program.

[0046] A fourth aspect of the present solution relates to a vehicle comprising: (i) a seating arrangement with at least one seat; (ii) a radar sensor for at least sectional radar scanning of the seating arrangement including at least one footwell of the seating arrangement, wherein the radar sensor is arranged above the seating arrangement; and (iii) a system according to the second aspect.

[0047] The features and advantages explained in relation to the first aspect of the present solution also apply accordingly to the other aspects of the solution.

[0048] Further advantages, features and application possibilities of the present solution will become apparent from the following detailed description in conjunction with the drawings.

[0049] This shows: Fig. 1 schematically an exemplary embodiment of a vehicle which is equipped with a system for the automated detection of a footwell occupancy state and a seat occupancy state of a seating arrangement in the vehicle; Fig. 2 schematically the vehicle from Fig. 1, where the passenger seat is occupied; Fig. 3A an exemplary two-dimensional representation of a radar sensor of the vehicle Fig. 2 recorded radar point clouds; Fig. 3B an exemplary representation of a clustering of the radar point cloud from Fig. 3A according to the positions of the individual seats in the seating arrangement; Fig. 4 schematically a view of the rear seat from inside the vehicle Fig. 1; Fig. 5A an exemplary three-dimensional representation of a radar sensor of the vehicle Fig. 1 recorded radar point cloud of the rear seat in a front view, where the rear left seat is occupied, including a clustering; Fig. 5B an exemplary three-dimensional representation of the radar point cloud from Fig. 5A in a view from the left; Fig. 6 a flowchart to illustrate an exemplary embodiment of a method for the automated detection of a footwell occupancy state and a seat occupancy state of a seating arrangement.

[0050] In the figures, identical reference symbols denote identical, similar, or corresponding elements. Elements depicted in the figures are not necessarily shown to scale. Rather, the various elements depicted in the figures are represented in such a way that their function and general purpose are understandable to a person skilled in the art. Connections and couplings between functional units and elements shown in the figures can, unless expressly stated otherwise, also be implemented as indirect connections or couplings. Functional units can, in particular, be implemented as hardware, software, or a combination of hardware and software.

[0051] The in Fig. Figure 1, a schematically illustrated exemplary embodiment of a vehicle 100, has a seating arrangement 105 with five individual seats or seating positions 105a to 105e. Each of the seats 105a to 105e is suitable for accommodating one person as a passenger of the vehicle 100. The vehicle 100 also has a radar sensor 110, which is mounted inside the vehicle cabin on its ceiling, i.e., above the seating arrangement 105, and is configured such that it can scan the seating arrangement 105, at least substantially, using radar beams. Accordingly, the seats 105a to 105e, in particular their seating surfaces, are located, at least predominantly, within a field of view 110a that can be scanned by the radar sensor 110. Furthermore, the corresponding footwells, in particular the footwells 405c to 405e of the seats 105c to 105e of the rear bench, are located in the field of view 110a, as shown in Figure 1. Fig. 4 shown.

[0052] Furthermore, the vehicle 100 has a system 115 for the automated detection of a footwell occupancy state and a seat occupancy state of the seating arrangement 105 depending on a radar scan of the seating arrangement 105, including the footwells, carried out by the radar sensor 110, at least sectionally with respect to the observation field 110a.

[0053] System 115 includes, in particular, a data processing unit 115a with at least one microprocessor and a signal-connected memory 115b, in which a memory for carrying out the process described below with reference to Fig. The computer program configured for the automated detection of the footwell and seat occupancy status of the seat arrangement 105, as described in section 6, is stored in memory 115b. Furthermore, the sensor data generated by the radar sensor 110 during radar scanning, or information already obtained from it through further processing, may be stored or will be stored in memory 115b.

[0054] The in Fig. 2. Vehicle 100 shown corresponds to the vehicle from Fig. 1, however, the passenger seat 105b is occupied by person P. In the further discussion of the Fig. 3A and Fig. 3B is based on the constellation of Fig. 2. Referenced.

[0055] The following section will discuss the Fig. 3A and Fig. Reference is made to Figure 3B, each representing a radar point cloud, whereby, for the purpose of representation, the respective, inherently three-dimensional radar point cloud was reduced to two dimensions by projecting the positions of the radar points of the radar point cloud onto a plane spanned by two of its dimensions. Corresponding radar point clouds of the rear seat, i.e., seats 105c to 105e, including the associated footwells 405c to 405e, are shown in the Fig. 5A and Fig. 5B is shown, where seat 105c (rear left) is occupied. The following explanations regarding Fig. 3A and Fig. 3B therefore also applies accordingly to Fig. 5A and Fig. 5B.

[0056] In Fig. Figure 3A illustrates an exemplary radar point cloud 305, as it was acquired as a result of a radar scan of the seating arrangement 105 by the radar sensor 110 during a defined time interval (measurement period). The position of the individual radar points within the radar point cloud 305 can be represented by spatial coordinates; for example, Cartesian coordinates X and Y can be assigned to the plane of the drawing and to each individual point. In reality, if the dimensional reduction due to the drawing is disregarded, a third coordinate Z for the third spatial dimension is also required.

[0057] If, during radar scanning, not only the spatial positions of the points where the radar beam is reflected by the scanned objects are recorded as coordinates, but also a corresponding Doppler shift is measured, then the individual radar points can be classified according to the magnitude of this Doppler shift, in particular divided into two different classes. The latter can be done, for example, by comparing the Doppler shift with a predefined shift threshold that corresponds to a specific displacement velocity. Depending on the result of the comparison, those radar points 310 that, according to the value of their assigned Doppler shift, exhibit no velocity or a surface velocity at the reflection point that is below the displacement wave can be classified as "static" radar points (in the Fig. 3A and Fig. 3B each represented by a filled black circle). Conversely, those radar points 315 that exhibit a Doppler shift above the shift threshold can be classified as “dynamic” radar points 315 (in the Fig. 3A and Fig. 3B each represented by a black ring).

[0058] Classifying radar points 310 and 315 according to their Doppler shift is not strictly necessary; however, it can be used to process radar point cloud 305, particularly as part of preprocessing prior to evaluation, and especially to filter based on the classification. For example, such filtering could be performed to consider only dynamic radar points 315 for evaluation, in order to detect only moving objects.

[0059] In Fig. 3B is the same radar point cloud 305 as in Fig. Figure 3A is shown. In addition, cuboid (3D case) or rectangular (in the present 2D representation) selected spatial areas 325a to 325e are shown, which are spatially assigned to the respective location of the individual seats 105a to 105e. The definition of these spatial areas 325a to 325e can now be used to cluster the radar point cloud 305, whereby each radar point 310 or 315 is assigned, as far as possible, to the spatial area 325a to 325e in which it lies. All radar points not located in one of the spatial areas 325a to 325e can be disregarded in the following. It can be seen in particular that the areas 320 with a particularly high radar point density are located in the area of ​​the front passenger seat 105b, on which, according to Fig. 2. Person P is located.

[0060] Accordingly, (as a 3D representation) in the Fig. 5A and Fig. 5B shows additional room areas 525c to 525e of the associated footwells 405c to 405e. Areas with a particularly high radar point density are visible here in the area of ​​seat 105c.

[0061] Fig. Figure 6 shows a flowchart illustrating an exemplary embodiment 600 of a method for automatically detecting the footwell and seat occupancy status of a seating arrangement. The method can, in particular, be designed as a computer-implemented method. For this purpose, it can, in particular, be stored as a computer program in memory 115b of system 115 and be executable on the data processing unit 115a.

[0062] In method 600, a radar point cloud 305 is acquired by receiving and processing radar measurement data, in this example from the radar sensor 110 of the vehicle 100, in a process 610 to form one or more radar point clouds.

[0063] The radar point cloud 305 now available can then be clustered in a further process 620 by checking for each of its radar points whether it lies within one of the defined spatial areas 325a to 325e (see Fig. 3B) or 525c to 525e (see Fig. 5A and Fig.5B) and, if applicable, in which area. Thus, each point can be assigned either to one of the spatial areas 325a to 325e (or 525c to 525e) or to the other observation field. All radar points that lie within the same spatial area 325a to 325e (or 525c to 525e) are grouped into a respective cluster. As a result, each of the seats 105a to 105e and each of the footwells 405c to 405e of seats 105c to 105e in the rear row of seats is assigned a corresponding cluster of radar point cloud 305. This forms the basis for the subsequent individual evaluation of each seat 105a to 105e and each footwell 405c to 405e to determine whether the respective seat 105a to 105e or footwell 405c to 405e is or was occupied or not while radar point cloud 305 was formed.

[0064] In process 630, the clusters of room areas 325c to 325e and 525c to 525e are classified according to which of these room areas belongs to a seat 105c to 105e or to a footwell 405c to 405e. Depending on the type, i.e., "seat" or "footwell," the measurement data is evaluated differently, as described below.

[0065] In process 640a, the clusters of room areas 325c to 325e, which were classified as belonging to seats 105c to 105e, are evaluated, as is the cluster for room areas 325a and 325b of the front seats 105a and 105b. In particular, features can be determined for each of the clusters for the subsequent evaluation of the clustered radar point cloud 305. These features can include, in particular, the number of radar points in the cluster. If no filtering according to the Doppler shift value has taken place, this can be a combined count of both the static and the dynamic radar points 310 and 315, respectively. However, if the static radar points 310 were filtered out beforehand, it is only a count of the dynamic radar points 315.Other features can include the position of a cluster's centroid, the point density within a cluster, or the mean Doppler value of a cluster. These features are each summarized in a feature vector for each cluster, as will be explained in more detail below.

[0066] The features for the cluster corresponding to seat 105b can now be evaluated (the same can be done analogously for the respective clusters for the other seats). For this purpose, the feature vector (or a temporal progression of the features in a single vector, su) is provided as input to a trained evaluation model. This can be, in particular, a machine learning-based model, such as an artificial neural network.

[0067] To eliminate outliers, the aforementioned processes are not only performed for a single point in time (frame), but an average is calculated over multiple points in time (frames). This represents the key difference in the evaluation of the clusters assigned to seats 105a to 105e on the one hand, and those assigned to footwells 405c to 405e on the other. Otherwise, the evaluation can be carried out in the same way.

[0068] In process 640b, the clusters of room areas 525c to 525e, which have been classified as belonging to footwells 405c to 405e, are therefore evaluated. Reference is made to the explanations for process 640a. However, no averaging over multiple time points (frames) takes place here. Instead, the measurement data from the time point (or the corresponding feature vectors) for which a footwell occupancy state determined by the evaluation model is more likely to correspond to the actual footwell occupancy state are used as input data for the evaluation model. Corresponding confidence values ​​are then calculated.

[0069] Specifically, a confidence level between 0 and 1 is determined for each footwell (405c to 405e), indicating the probability that the footwell is occupied or unoccupied. A confidence level less than 0.5 means that the corresponding footwell is unoccupied. Conversely, a confidence level of 0.5 or higher means that the respective footwell is occupied. For confidence levels greater than or equal to 0.5, the measurements from the time point with the highest confidence level are used. Conversely, for confidence levels less than 0.5, the measurements from the time point with the lowest confidence level are used. In other words, the measurements that indicate with a higher probability that a footwell is occupied or unoccupied are used, instead of balancing these values ​​by averaging.This can increase the reliability of detecting footwell occupancy, especially given the potentially weaker and noisier radar signal in the footwell (compared to the radar signal for the seats). While monitoring has been described here for footwells 405c to 405e of seats 105c to 105e in the rear row, it should be noted that this also applies equally to the footwells of front seats 105a and 105b. With a suitable radar sensor in an appropriate position above the seat assembly 105, preferably a single radar sensor, it may also be possible to monitor both rows of seats, including all footwells.

[0070] In both process 640a and process 640b, information is ultimately output as to whether a seat or footwell is occupied or not.

[0071] In the present example, in a process 650a, this information about the seat occupancy status is to be used in particular to check whether, depending on the seat occupancy status of a respective seat 105a to 105e and the result of a check as to whether a corresponding seat belt has been fitted for this seat or not, a seat belt warning signal should be issued or not.

[0072] This can involve checking whether the seat belt is fastened for the relevant seat (here, for example, seat 105b) and controlling a function of vehicle 100 depending on the information about the seat occupancy status and the specific status of the seat belt output in process 640a. In particular, this can be done by activating a signal source to output a seat belt status signal, especially a visual and / or acoustic one, in order to signal to one or more other occupants of the vehicle that a seat is occupied but the seat belt is not fastened.

[0073] In a 650b process, information about the footwell occupancy status is used to prevent automatic seat adjustment. Specifically, this can be achieved by activating a signal source to prevent a seat adjustment, particularly a rearward adjustment, if the footwell directly behind the seat is occupied. This prevents an object or a person's foot from being trapped. Simultaneously, a signal source can be activated to output a warning signal, particularly visual and / or audible, indicating that the footwell is occupied. Alternatively, a corresponding message can simply appear on a display when the footwell is occupied, which can be useful, for example, if an object like a bag would otherwise be left behind.

[0074] Finally, it should be noted that the training and, if applicable, validation data used for prior training of the evaluation model can be structured in such a way that each data entry contains a corresponding, correct class for a classification of possible footwell or seat occupancy states. This allows the model to be trained and validated using supervised learning. In the simplest case, footwell or seat occupancy states indicate whether the footwell or seat is occupied or not. However, more sophisticated classifications are also conceivable, in which, if an object is present, the respective class additionally indicates the type of object, for example, whether it is moving or stationary, and, in the case of a moving object, specifically whether it is a person (generally recognizable based on a breathing pattern over the course of the features).

[0075] While at least one exemplary embodiment has been described above, it should be noted that a large number of variations exist. It should also be noted that the described exemplary embodiments are merely non-limiting examples, and it is not intended to restrict the scope, applicability, or configuration of the devices and methods described herein. Rather, the preceding description will provide the person skilled in the art with guidance for implementing at least one exemplary embodiment. It is understood that various modifications to the function and arrangement of the elements described in an exemplary embodiment can be made without derogating from the subject matter defined in the appended claims and their legal equivalents. REFERENCE MARK LIST P Person in the front passenger seat 100 vehicles 105 Seating arrangement 105a-e seats or seating places 110 radar sensor 110a Observation field of the radar sensor 110 115 System for detecting footwell and seat occupancy status 115a Data processing unit 115b memory 305 radar point cloud 310 static radar points 315 dynamic radar points 320 areas of radar point cloud 305 with high radar point density 325a-e Room areas for cluster definition (seating capacity) 405c-e Footwells 525c-e Room areas for cluster definition (foot spaces) 600 methods for detecting footwell and seat occupancy status 610-650b individual processes or procedural steps within the framework of procedure 600

Claims

[1] Method (600) for detecting a footwell occupancy state for a seating arrangement (105) with at least one seat (105a-e), the method comprising: - Acquiring a data set for a footwell occupancy state, wherein the data set represents measurement data at least for a current time, wherein the measurement data represent an associated radar point cloud (305) which was or is obtained on the basis of a radar scan of a spatial area surrounding the seating arrangement (105) including at least a footwell (405c-e) at least sectionally, wherein the radar scan is carried out by means of a radar sensor (110) which is arranged above the seating arrangement (105); - Determining a footwell occupancy state using an evaluation model, wherein the data set forms input data for the evaluation model and the evaluation model, depending on the radar point cloud (305), provides a footwell occupancy state of at least one footwell (405c-e) at the current time as output based on the acquired data set; and - Outputting footwell occupancy status information defined as a function of the evaluation result, wherein the measurement data are recorded for at least two consecutive time points, including the current time point, wherein a confidence value for a footwell occupancy status of the at least one footwell (405c-e) is determined for the current time point and at least one further of the consecutive time points, indicating the probability that the actual footwell occupancy status is "occupied" or "unoccupied", wherein the measurement data of the time point for which a footwell occupancy status determined by means of the evaluation model is more likely to correspond to the actual footwell occupancy status are used as input data for the evaluation model for determining the footwell occupancy status. [2] Method according to claim 1, wherein the confidence value is a value in the range of 0 to 1. [3] Method according to claim 2, wherein, if the confidence values ​​are greater than or equal to 0.5, the measurement data of the time point for which the confidence value is a maximum of the confidence values ​​are used as input data for the evaluation model. [4] Method according to claim 2 or 3, wherein, if the confidence values ​​are less than 0.5, the measurement data of the time point for which the confidence value is a minimum of the confidence values ​​are used as input data for the evaluation model. [5] Method according to any one of claims 2 to 4, wherein, if the confidence values ​​are both greater and less than 0.5, the confidence values ​​are determined from at least three consecutive time points, including the current time point, wherein If the majority of the confidence values ​​are greater than 0.5, the measurement data from the time point in time for which the confidence value is a maximum of the confidence values ​​are used as input data for the evaluation model, or If the majority of the confidence values ​​are less than 0.5, the measurement data from the time point in time for which the confidence value is a minimum of the confidence values ​​are used as input data for the evaluation model. [6] Method according to one of the preceding claims, wherein the seating arrangement (105) comprises several seats (105a-e), wherein each of the seats is assigned a foot space (405c-e), and wherein an individual foot space occupancy state is determined for each of the foot spaces (405c-e). [7] Method according to any of the preceding claims, further comprising a detection of a seat occupancy state of the seat arrangement (105), wherein the detection of the seat occupancy state comprises: - Acquiring a data set for a seat occupancy status, wherein the data set represents measurement data at least for a current time, wherein the measurement data represent the associated radar point cloud (305) which was or is obtained on the basis of radar scanning; - Determining a seat occupancy state of the seating arrangement (105) using the evaluation model, wherein the data set forms input data for the evaluation model and the evaluation model, depending on the radar point cloud (305), provides a seat occupancy state of the seating arrangement (105) at the current time as output based on the acquired data set; and - Outputting seat occupancy status information defined based on the evaluation result. [8] Method according to claim 7, wherein the measurement data are recorded for at least two successive time points, including the current time point, wherein an average value is determined for the current time point and at least one further of the successive time points, wherein the input data for the evaluation model for determining the seat occupancy state includes the determined average value. [9] Method according to claim 7 or 8, wherein the determination of the footwell occupancy state takes into account the determined seat occupancy state. [10] Method according to one of the preceding claims, wherein a measure of motion activity at the current time is determined from Doppler shift values ​​from the measurement data, wherein the data set contains the determined measure of motion activity as input data for the evaluation model. [11] Method according to one of the preceding claims, wherein the radar point cloud (305) is segmented into several clusters by assigning to each of the seats (105a-e) and each of the footwells (405c-e) as a cluster a subset of the radar points of the respective radar point cloud (305) depending on their respective position such that the radar points of the cluster are located in a defined closed spatial area (325a-e, 525c-e) in the vicinity of the seat (105a-e) or the footwell (405c-e). [12] Method according to one of the preceding claims, wherein the at least one footwell (405c-e) comprises at least one footwell which is assigned to a rear seat (105c-e) of the seating arrangement (105), wherein a signal source is controlled in such a way as to depend on the information from the evaluation result, such that a signal output by the signal source prevents an automatic seat adjustment to the rear of a seat (105a-b) of the seating arrangement which is located directly in front of the footwell (405c-e) of the rear seat (105c-e). [13] System (115) for detecting a footwell occupancy state for a seating arrangement (105) with at least one seat (105a-e), wherein the system (115) comprises a data processing device configured to perform the method according to one of the preceding claims for detecting a footwell occupancy state for a seating arrangement (105). [14] Computer program or computer program product comprising instructions which, when executed on the data processing device of the system (115) according to claim 13, cause the system (115) to execute the method according to any one of claims 1 to 12. [15] Vehicle (100), comprising: a seating arrangement (105) with at least one seat (105a-e); a radar sensor (110) for at least partial radar scanning of the seating arrangement (105) including at least one footwell (405c-e) of the seating arrangement (105), wherein the radar sensor (110) is arranged above the seating arrangement (105); and a system (115) according to claim 13.

Images