Method for detecting a seat occupancy state of a seat arrangement

Abstract

In a method for the automated detection of a seat occupancy state of a seat arrangement having at least one seat, a data set is captured for a seat occupancy state, in particular in the form of a radar point cloud which is obtained on the basis of a radar scan. Static and dynamic features (531,532) are extracted therefrom. A seat occupancy state of the seat arrangement is determined on the basis of an evaluation model, which is constructed as follows. In a first stage, the static features (531) are processed in a first classifier (532) and, in parallel, the dynamic features (532) are processed in a second classifier (535) in order to obtain a static partial evaluation result or a dynamic partial evaluation result. In a second stage, by means of an optimal state estimator (536) to which the obtained static partial evaluation result and the obtained dynamic partial evaluation result are supplied, one of multiple predefined possible seat occupancy states of the seat arrangement is estimated and output as the evaluation result As a result of the method, information defined according to the evaluation result is output in order to output, for example, a seat belt warning on the basis thereof.

Description

[0001] 2023PF02572 METHOD FOR DETECTING THE OCCUPANCY STATE OF A SEATING ARRANGEMENT The present invention relates to a method, a computer program, and a system configured to execute the method, each for automatically detecting the occupancy state of a seating arrangement with at least one seat using a trained evaluation model, in particular a machine learning model. In various situations, it may be necessary to automatically determine the current occupancy state of a seating arrangement with at least one seat. Such a situation can occur particularly in vehicles, for example, in motor vehicles, where vehicle configuration or the activation, deactivation, and / or control of one or more vehicle functionalities is to be carried out depending on the current occupancy state.For example, in motor vehicles, it is known to issue an acoustic or visual warning to vehicle occupants to fasten their seat belts, or to control the activation or deactivation of airbags, depending on the detected seat occupancy status. For safety reasons, it may also be necessary to detect the presence of a child in a seat ("life presence detection" or "child presence detection"). Methods using radar technology are known for the automated detection of the current occupancy status of one or more seats, particularly the arrangement of seats in a vehicle. Radar sensors scan the vehicle interior, generating measurement data in the form of radar point clouds. The current seat occupancy can then be determined based on the measured radar point cloud.For this purpose, a suitable evaluation model can be used, which can be a machine learning model. This model can be trained with relevant data before its actual use. The known methods, using the machine learning model, can also recognize the type of seat occupancy, for example, whether a seat is occupied by an adult or a child. The evaluation model returns either "occupied" or "unoccupied" as the result for a specific seat. So far, the classification is based solely on the assumption of a static scenario (seat "occupied" or "unoccupied"). A temporal filter can also be performed in a post-processing step. However, this also results in a static representation of the seat occupancy status.To improve the evaluation result, an evaluation model can, in principle, use input data from more than one measurement cycle or utilize additional features. For this purpose, all input data, e.g., the features from the current and the previous N cycles, can be fed into the evaluation model, or the features can utilize previously collected data. The disadvantage, however, is the significant computational effort, as the model must process large amounts of data. In particular, the requirement for training data is also high. It is an object of the present invention to provide an improved solution for the automated detection of the occupancy status of a seating arrangement with at least one seat. In particular, the improved solution is intended to further increase the reliability of the seat occupancy status detection. This object is achieved according to the teaching of the independent claims.Various embodiments and further developments of the invention are the subject of the dependent claims. A first aspect of the solution presented here relates to a method, particularly a computer-implemented method, for the automated detection of the occupancy status of a seating arrangement with at least one seat, in particular a seat in or for a vehicle, such as an automobile (e.g., truck, car, or bus). A data set for a seat occupancy status (hereinafter also referred to as "status") is acquired, wherein the data set represents features of a radar point cloud. The features comprise at least one static feature and at least one dynamic feature. The features are derived from measurement data, in particular extracted or...The occupancy status of the seating arrangement is calculated, with each measurement data representing an associated radar point cloud at a specific time, obtained from a radar scan of a spatial area at least partially surrounding the seating arrangement. An evaluation model determines the occupancy status of the seating arrangement. The data set serves as input data for the evaluation model, which, based on the acquired data set and the radar point cloud, delivers one of several predefined possible occupancy states as an evaluation result. The evaluation model is structured as follows: In a first stage, at least one static feature is processed in at least one first ("static") classifier, and at least one dynamic feature is processed in at least one second ("dynamic") classifier to generate a static partial evaluation result.The process involves obtaining a dynamic partial evaluation result by executing the first and second classifiers in parallel. In a second stage, a state estimator, fed with both the obtained static and dynamic partial evaluation results, estimates (predicts) one of several predefined possible seat occupancy states for the seating arrangement and outputs this as the evaluation result. The result of the procedure is information defined based on the evaluation result. The procedure described in the first aspect is therefore based, in particular, on providing an evaluation model for detecting a seat occupancy state, especially for locating and classifying occupants in a vehicle cabin, which, due to its architecture, offers particularly reliable and efficient seat occupancy state detection.The architecture of the evaluation model is characterized by a two-stage approach. The parallel processing of static and dynamic features in the first stage reduces the size and complexity of the model architecture, especially compared to combined processing. Furthermore, the advantage of using a dynamic classifier alongside a static classifier lies in the ability to utilize additional information contained in the dynamic features. This dynamic information would otherwise be lost, as it is not represented in the static features. The state estimator then uses static and dynamic information—more precisely, a respective (partial) evaluation result from the two (or potentially more) classifiers—to perform a state estimation.One advantage of combining dynamic and static classifiers with the state estimator is the reduced need for training data for the classifiers, especially compared to a recurrent neural network (RNN), which requires all seat occupancy combinations, transitions, and the times between transitions. The classifiers are used particularly for the part of the problem that is difficult to model (namely, the correlation between the seat occupancy state and measurement data). The second part of the problem (the consistency of the occupancy state over time, i.e., for example, whether the state remains the same unless the movement of occupants in and out of the vehicle changes it) can then be explicitly modeled in a state model (also called a "state transition model"), as will be explained in more detail later. Here, for example,One scenario could be modeled where, if a seat is empty and a person boards, the seat will likely become occupied, and so on. In summary, the advantage lies in reduced training effort and the ability to explicitly model the system's dynamic behavior and apply and modify this model without additional training. Finally, the estimator's accuracy benefits from exploiting the known relationship between dynamic and static state measurements. Generally, using the method described in the first aspect, based on a radar point cloud obtained by radar scanning a spatial area surrounding the seating arrangement, a stable evaluation result characterizing the seating arrangement's occupancy state (particularly in terms of prediction or classification) can be obtained by simultaneously using static and dynamic features.This allows for the implementation of radar-based solutions, particularly in the vehicle context (especially for automobiles), that can reliably detect seat occupancy (especially exclusively) using radar and, based on this, activate, deactivate, or control / regulate certain functionalities or systems, such as a seatbelt warning system or an airbag system, either completely or selectively. The term "seat occupancy status" as used here refers specifically to information indicating whether or to what extent the seating arrangement, or at least one of its seats, is occupied by an object, particularly a thing or a person. In a simple example, the seat occupancy status can simply indicate the presence or absence of an object, or, in a more advanced example, it can indicate the presence of at least one object on the seating arrangement.one or more of its locations, a statement about the type or other property, such as a spatial extent, of the object. 2023PF02572 The term "radar point cloud" used here refers in particular to a set of points in a vector space obtained by radar scanning of at least one object surface, which exhibits a typically unorganized spatial structure ("cloud"). In the case of a radar point cloud, the points of the radar point cloud can be referred to as "radar points". A (radar) point cloud can be described in particular by the (radar) points it contains. The radar points, in turn, can each be described in particular by their spatial coordinates, which indicate, for each radar point, a location measured during radar scanning where an emitted radar signal is reflected from an object surface. Additional attributes, such as...The measured Doppler velocity or signal-to-noise ratio (SNR) may be recorded. These can be extracted from the measurement data as features, particularly static or dynamic features. Each individual radar point in the radar point cloud can be described by its position in three-dimensional space, as well as by a Doppler shift value of the radar signal relative to the respective radar point and / or a signal-to-noise ratio value of the radar signal relative to the respective radar point. Radar points whose Doppler shift value is at or above a predetermined shift threshold (not zero) can be referred to as "dynamic radar points," i.e., those radar points that indicate movement of the scanned object and whose Doppler shift value is at or above the shift threshold.Points exhibiting dynamic movement are therefore highly likely to be associated with a living being, particularly a person or animal. In parallel, static points on object surfaces, such as points on a seat, are also considered. The term "evaluation model" used here refers to a model, particularly a mathematical one, that uses a radar point cloud or one or more of its characterizing features or parameters as input to deliver an evaluation result, in this case, one of several predefined possible seat occupancy states of the seating arrangement. The evaluation model can, in particular, be a mathematical estimator, where the radar point cloud represents empirical data as a sample and the evaluation result is a value determined based on this sample.The evaluation model can, in particular, be a "machine learning model" (or, equivalently, a "machine learning model" (ML model)), which here refers specifically to a mathematical, especially statistical, model for making predictions or decisions, created using at least one machine learning algorithm based on example data referred to as training data, without the algorithm(s) being explicitly programmed to make such predictions or decisions. Decision tree-based machine learning models (or "decision trees") and artificial neural networks are specifically examples of machine learning models. The term "state estimator" or "optimal state estimator" refers specifically to a method used to determine the (discrete) seat occupancy states, i.e., for a seat, for example...Whether features are "occupied" or "not occupied" can only be estimated because they cannot be measured directly, but can only be determined, estimated, or predicted indirectly from the (continuous) measurement data, i.e., the radar point cloud, and especially from features extracted from it. To reduce measurement noise and obtain more accurate results, the so-called Bayesian filter can be used. This filter is specifically designed to minimize the variance of the state estimation (i.e., the error). Any terms used, such as "includes," "contains," "includes," "exhibits," "has," "with," or any other variant thereof, are intended to cover non-exclusive inclusion.For example, a method or apparatus that includes or comprises a list of elements is not necessarily limited to those elements, but may include other elements not expressly listed or inherent in such method or apparatus. Furthermore, unless expressly stated otherwise, "or" refers to an inclusive or, not an exclusive "or." For example, a condition A or B is satisfied by any of the following: A is true (or present) and B is false (or not present), A is false (or not present) and B is true (or present), and both A and B are true (or present). The terms "a" or "an" as used herein are defined as "one or more." The terms "another" and "another," and any other variant thereof, are to be understood as "at least one other."2023PF02572 The term "plural" or "several," as used here, is to be understood as "two or more." The terms "configured" or "set up" to perform a specific function (and their respective variations) are to be understood, within the meaning of the invention, as meaning that the corresponding device already exists in a configuration or setting in which it can perform the function, or at least it is adjustable—i.e., configurable—so that it can perform the function after appropriate adjustment. The configuration can be achieved, for example, by appropriately setting parameters of a process sequence or by using switches or similar devices to activate or deactivate functionalities or settings.In particular, the device can have several predetermined configurations or operating modes, so that configuration can be performed by selecting one of these configurations or operating modes. Various exemplary embodiments of the method are described below, which, unless expressly excluded or technically impossible, can be combined with each other and with the other described aspects of the present solution. In some embodiments, the state estimator is an optimal state estimator that minimizes an error in an estimated seat occupancy state in an iterative process. The state estimator can be a Bayesian filter. These filters are well suited for radar signal processing.The Bayesian filter predicts the seat occupancy state, which can be modeled by discrete state variables, for the next time step and updates the state estimate based on the measurement data. The uncertainties of the state estimate and the measurement data can be modeled by Gaussian random variables. By using both the static and dynamic partial evaluation results, a reliable evaluation result can be obtained using the optimal state estimator. In some embodiments, a state model represents possible seat occupancy states for a seat in the seating arrangement, with the state estimator (a posteriori) considering state transition probabilities that represent the probabilities of a transition from a seat occupancy state at a first time point to a seat occupancy state at a subsequent second time point.For example, it is likely that movement will be detected while a person is boarding a vehicle. If a seat was previously occupied, its state may change to "unoccupied." If it does not change, it remains "occupied." In some embodiments, the seating arrangement has a plurality of seats, with a state model that aggregates possible seat occupancy states for at least a subset of the plurality of seats containing more than one seat in the seating arrangement. The state estimator considers state transition probabilities, which represent the likelihood of a transition from a seat occupancy state at a first time point to a seat occupancy state at a subsequent second time point. For example, for a left seat L and a right seat R, the seat occupancy state at the earlier time point might be "L: occupied, R: occupied."The seat occupancy state can then change to "L: unoccupied, R: occupied," "L: occupied, R: unoccupied," or "L: unoccupied, R: unoccupied," or remain unchanged. The subset of the plurality of seats can, in particular, include seats in a row of adjacent seats in the seating arrangement. The subset can also include seats in several rows or, alternatively, all seats in the seating arrangement. This allows for a broader definition of state transitions compared to considering only one seat. For two adjacent seats, such as the driver's and passenger's seats of a vehicle (left seat L and right seat R, respectively), this results in four seat occupancy states ("L: occupied, R: occupied"; "L: occupied, R: unoccupied"; "L: unoccupied, R: occupied"; "L: unoccupied, R: unoccupied"). In some embodiments, the state transition probabilities are adaptively set for each point in time.This is advantageously achieved through the iterative design of the (optimal) state estimator. In some embodiments, probabilities for different seat occupancy states are determined using the state transition probabilities, with the state estimator outputting the seat occupancy state with the highest probability as the evaluation result. In some embodiments, the static partial evaluation result includes at least one from the following list of static states for a seat: occupied, unoccupied, type of seat occupancy (e.g., distinction between a living object (human or animal) and a non-living object (i.e., an item such as a bag, a water bottle, etc.)). For further refinement, the THUMS human model can be used (su).In some embodiments, the dynamic partial evaluation result includes at least one of the following dynamic states for a seat: no movement, movement, movement towards the seat, movement away from the seat. The type and direction of movement can further improve the result compared to a model that can only distinguish between "movement" and "no movement." In some embodiments, in the first stage, the first classifier, the second classifier, and / or at least one further classifier process at least one mixed feature consisting of static and dynamic features in addition to the static or dynamic feature. Generally, more than two classifiers can be used in parallel to obtain partial evaluation results that are fed to the state estimator. This can further improve the recognition accuracy.In some embodiments, the seating arrangement is a seating arrangement in a vehicle, wherein the evaluation model additionally considers vehicle status data, the vehicle status data comprising at least one of the following: door status, engine status, vehicle speed, and seatbelt fastening status. The door status indicates, in particular, whether a door is open or closed, and specifically which door. The engine status indicates whether the engine is running or not. For speed, a value can be considered, or only the fact of whether a vehicle is stationary or moving. The seatbelt fastening status indicates whether a seatbelt is fastened for a seat or not. Due to vibrations generated when the engine is running and the vehicle is moving, the engine status and vehicle speed can influence the measurement results of the radar sensors.Door status and seatbelt fastening status can provide an indication of whether a seat is occupied or unoccupied, or whether the occupancy status might change, or is likely to change. Vehicle status data can be incorporated into the evaluation as boundary conditions. For example, state transitions that are logically possible can be favored, while others can be classified as highly improbable. For instance, it is highly unlikely, if not impossible, that the occupancy status of a seat, especially a driver's or passenger's seat, has changed between two points in time while the vehicle is moving. Similarly, if no door has been opened, a change is less likely than if one or more doors have been opened, particularly for a seat associated with a specific door (e.g., driver's seat and driver's door).Similarly, this can also apply to the seatbelt status. For example, it can be assumed that a seat previously identified as "occupied" and where the seatbelt remains fastened will very likely continue to be "occupied." In some embodiments, each radar point cloud is segmented into several clusters by assigning each seat a subset of the radar points from the respective radar point cloud, depending on their position, such that the radar points of the cluster lie within a defined closed spatial area, in particular a cuboid, in the vicinity of the seat.This enables particularly simple and computationally efficient clustering and thus seat-related seat occupancy detection, whereby the location (position and orientation) and shape of the spatial area is or can be defined in such a way that it significantly overlaps the spatial area typically occupied by a recognizable object, especially a person, on a seat in the seating arrangement. In addition to the seats in the seating arrangement, it may also be possible to define a separate spatial area as a cluster for other spatial areas, such as footwells, for example, also in the form of a cuboid. Through clustering, seat-related, i.e.,Individual seat occupancy detection is enabled for each seat, which is particularly advantageous or even necessary when a seat-specific response to the detected occupancy is required, for example, by activating, deactivating, or otherwise controlling a specific functionality or system for a particular seat, such as a seat-specific airbag system, a seat-specific seatbelt reminder, or a seat-specific seat heater, depending on its detected occupancy. In some embodiments, cluster formation can be carried out in such a way that the clusters are disjoint, so that no radar point is assigned to two different clusters. For each cluster, a fixed set of features can be calculated, and a feature vector can be created that contains a corresponding numerical value for each feature.The calculated features describe various properties of the cluster, such as the number of points, the position of the center of gravity, the point density, or the mean Doppler value of the points belonging to the cluster. In some embodiments, the seating arrangement has multiple seats, for which a seat occupancy state must be determined individually or cumulatively within the framework of the method. For each (individual) radar point cloud, the set of its radar points is subdivided into several clusters, each containing a subset of the radar points, by means of cluster formation, depending on the respective spatial location of the radar points relative to the seats, in order to individually assign each seat to the cluster closest to it.The occupancy status of each seat is determined based on the radar point cloud assigned to the corresponding cluster. This process yields an evaluation result, and potentially a classification result, that identifies the occupancy status of each seat. The information to be output is then defined based on the individual evaluation results for each seat. After the two-stage process, as mentioned above, information defined by the evaluation result is output. This information can represent the evaluation result itself. It can also be a detectable signal, particularly one perceptible to humans, such as a warning, a control signal for activating a signal source, or a data signal carrying the information.In some embodiments, outputting the information includes controlling a signal source based on the information, causing the signal source to output a defined signal depending on the control. The signal source can be, in particular, an audio source, an optical signal source, especially a display device for images or text, and / or a haptic actuator, or a combination of at least two of the aforementioned signal sources. Thus, the detected seat occupancy status can be communicated to a user based on the signaling or used to control another technical system, such as an airbag system.2023PF02572 In some of these embodiments, the signal source is controlled depending on the information so that it outputs a signal, in particular defined by the control, when the information results from an evaluation result according to which at least one seat of the seating arrangement is occupied and / or a selected predetermined seat occupancy state exists.In some embodiments, the method further comprises: (i) detecting a seatbelt fastening state of at least one seat in the seating arrangement or receiving seatbelt information characterizing this seatbelt fastening state; (ii) wherein the signal source is controlled depending on the seatbelt information and the information from the evaluation result such that it outputs a seatbelt fastening warning signal when, according to the information, at least one seat in the seating arrangement is occupied and / or a selected predetermined seat occupancy state exists and seatbelt information indicates that the corresponding seatbelt of the seat is not fastened. This allows for a seatbelt fastening check and warning system that is exclusively radar-based, particularly with regard to detection. In some embodiments, the seat occupancy state includes at least one type of seat occupancy for at least one seat in the seating arrangement.In particular, determining the seat occupancy status can include determining the type of seat occupancy. The type of seat occupancy can include at least one of the following: an unoccupied (i.e., free) seat, a seat occupied by an adult, a seat occupied by a child, a seat occupied by a child seat with a toddler, and a seat occupied by an infant carrier with a baby. Additionally, it can be provided to detect whether a seat is occupied by, for example, an object or a doll. A person can be simulated, for instance, using the established THUMS human model. In this model, values ​​such as AM95, AM50, and AF05 represent different adult persons (tall / heavy man, medium / medium-weight man, and small / light woman, respectively). Children can be identified by their age, e.g., 4YO or 6YO for a four-year-old and a six-year-old child, respectively.By determining not only whether a seat is occupied, but also how, and outputting the corresponding information, more nuanced control of, for example, a seatbelt warning system or airbag system can be achieved. For instance, certain airbags can only be activated if an adult occupies a seat, while an airbag can (or must) be deactivated if a seat is occupied, but not by an adult, but rather, for example, by a rear-facing infant car seat. Furthermore, when using a child car seat or infant carrier, which are typically not secured with a seatbelt but rather with a special attachment system (e.g., Isofix), it is advantageous if a seatbelt warning is not issued, thus avoiding unnecessary warning messages that are unhelpful and may be perceived as annoying by the user.A second aspect of the present solution relates to a system, in particular a data processing device, for automatically detecting the occupancy status of a seating arrangement with at least one seat, in particular with at least one vehicle seat in or for a vehicle. The system includes a data processing device configured, in particular by means of a corresponding computer program, to detect the occupancy status and execute the method according to the first aspect. A third aspect of the present solution relates to a computer program or computer program product comprising instructions which, when executed on the data processing device of the system according to the second aspect, cause the system to execute the method according to the first aspect. The computer program may, in particular, be stored on a non-volatile data carrier.Preferably, this is a data carrier in the form of an optical data carrier or a flash memory module. This can be advantageous if the computer program itself is to be handled independently of a processor platform on which the one or more programs are to be executed. In another implementation, the computer program can exist as a file on a data processing unit, in particular on a server, and be downloadable via a data connection, for example the Internet or a dedicated data connection, such as a proprietary or local network. Furthermore, the computer program can comprise a plurality of interacting individual program modules. The modules can be configured, in particular, or at least be usable in such a way that they can be run on different devices (computers or other devices) in the sense of distributed computing.Processor units are executed that are geographically separated and interconnected via a data network. 2023PF02572 The system according to the second aspect can accordingly have a program memory in which the computer program is stored. Alternatively, the system can also be configured to access an external computer program, for example on one or more servers or other data processing units, via a communication link, in particular to exchange data with it that is used during the execution of the procedure or computer program or represents outputs of the computer program.A fourth aspect of the present solution relates to a vehicle comprising: (i) a seating arrangement with at least one seat; (ii) a radar sensor for at least partial radar scanning of the seating arrangement; and (iii) a system according to the second aspect for automatically detecting, in particular, the occupancy status of the seating arrangement based on at least partial radar scanning of the seating arrangement performed by the radar sensor, in particular according to a method according to the first aspect. The features and advantages explained with regard to the first aspect of the present solution also apply accordingly to the further aspects of the solution. Further advantages, features, and application possibilities of the present solution will become apparent from the following detailed description in conjunction with the drawings. Figure 1 shows:1. A schematic representation of an exemplary embodiment of a vehicle equipped with a system for the automated detection of the occupancy status of a seating arrangement in the vehicle; Fig. 2. A schematic representation of the vehicle from Fig. 1, where the passenger seat is occupied; Fig. 3A. An exemplary two-dimensional representation of a radar point cloud recorded by a radar sensor of the vehicle from Fig. 2; Fig. 3B. An exemplary representation of a clustering of the radar point cloud from Fig. 3A according to the positions of the individual seats of the seating arrangement; 2023PF02572 Fig. 4. A flowchart illustrating an exemplary embodiment of a method for the automated detection of the occupancy status of a seating arrangement; and Fig. 5. A schematic representation of an exemplary embodiment of a system for the automated detection of the occupancy status of a seating arrangement; Fig. 6. A schematic first example of a state model; and Fig.Figure 7 schematically shows a second example of a state model. In the figures, identical reference numerals denote identical, similar, or corresponding elements. Elements depicted in the figures are not necessarily shown to scale. Rather, the various elements shown in the figures are represented in such a way that their function and general purpose are understandable to a person skilled in the art. Connections and couplings between functional units and elements shown in the figures can, unless expressly stated otherwise, also be implemented as indirect connections or couplings. Functional units can, in particular, be implemented as hardware, software, or a combination of hardware and software. First, with reference to Figures 1 to 4, the detection of a vehicle's seat occupancy using a radar system is described, since the method according to the invention is advantageously applicable in this context.Further details of the invention will then be explained, in particular with reference to Figures 5 to 7. The exemplary embodiment of a vehicle 100 shown schematically in Figure 1 has a seating arrangement 105 with five individual seats or seating positions 105a to 105e. Each of the seating positions 105a to 105e is suitable for accommodating one person as a passenger of the vehicle 100. The vehicle 100 also has a radar sensor 110, which is mounted on the ceiling inside the vehicle cabin and configured so that it can scan the seating arrangement 105, at least substantially, using radar beams. Accordingly, the seating positions 105a to 105e, in particular their seating surfaces, are located, at least predominantly, within a field of view 110a that can be scanned by the radar sensor 110.Furthermore, the vehicle 100 has a system 115 for the automated detection of a 2023PF02572 seat occupancy status of the seating arrangement 105 as a function of a radar scan of the seating arrangement 105 performed by the radar sensor 110, at least partially with respect to the observation field 110a. The system 115 includes, in particular, a data processing unit 115a with at least one microprocessor and a signal-connected memory 115b in which a computer program configured to carry out the method for the automated detection of a seat occupancy status of the seating arrangement 105, described below with reference to Fig. 4, is stored. Furthermore, the sensor data generated by the radar sensor 110 during the radar scan, or information already obtained from it through further processing, can be stored or will be stored in the memory 115b. The vehicle 100 shown in Fig. 2 corresponds to the vehicle from Fig. 4.1, where, however, the passenger seat 105b is occupied by a person P. The subsequent discussion of Figures 3A and 3B refers to the configuration shown in Figure 2. Figures 3A and 3B, which each represent a radar point cloud, are now referred to. For the purpose of visualization, the respective, inherently three-dimensional radar point cloud has been reduced to two dimensions by projecting the positions of the radar points onto a plane spanned by two of its dimensions. Figure 3A illustrates an exemplary radar point cloud 305, as it was acquired as a result of a radar scan of the seating arrangement 105 by the radar sensor 110 during a defined time interval (measurement period).The position of individual radar points within radar point cloud 305 can be represented by spatial coordinates. For example, Cartesian coordinates X and Y can be assigned to the plane of the drawing and to each individual point. In reality, if the dimensional reduction due to the drawing is disregarded, a third coordinate Z for the third spatial dimension is added. If, during radar scanning, not only the spatial positions of the points where the radar beam is reflected by the scanned objects are recorded as coordinates, but also a corresponding Doppler shift is measured, then the individual radar points can be classified according to the magnitude of this Doppler shift, in particular divided into two different classes.The latter (2023PF02572) can be achieved, for example, by comparing the Doppler shift with a predefined shift threshold corresponding to a specific shift velocity. Depending on the result of the comparison, those radar points 310 that, according to the value of their associated Doppler shift, exhibit no velocity or a surface velocity at the reflection point that is below the shift wave, can be classified as "static" radar points (represented in Figures 3A and 3B with a filled black circle). Conversely, those radar points 315 that exhibit a Doppler shift above the shift threshold can be classified as "dynamic" radar points 315 (represented in Figures 3A and 3B with a black ring).The classification of radar points 310 and 315 according to their Doppler shift can be used to process radar point cloud 305, particularly as part of a preprocessing step prior to its evaluation, in order to derive "static features" and "dynamic features," the processing of which will be explained in detail below. Specifically, the various features are processed in parallel in at least two classifiers, and their respective partial evaluation results are then jointly fed to an optimal state estimator (e.g., a Bayesian filter) to obtain a (discrete) evaluation result regarding the current seat occupancy status. Figure 3B shows the same radar point cloud 305 as Figure 3A. However, in addition, cuboid (3D case) or, in the present 2D representation, rectangular, selected spatial regions 325a to 325e are shown, which are spatially assigned to the respective location of the individual seats 105a to 105e.The definition of these spatial regions 325a to 325e can now be used to cluster the radar point cloud 305, whereby each radar point 310 or 315 is assigned, as far as possible, to the spatial region 325a to 325e in which it lies. All radar points not located in one of the spatial regions 325a to 325e can be disregarded in the following. It can be seen in particular that the regions 320 with a particularly high radar point density are located in the area of ​​the passenger seat 105b, on which, according to Fig. 2, person P is located. Fig. 4 shows a flowchart to illustrate an exemplary embodiment of a method 400 for the automated detection of a seat occupancy status of a seating arrangement. The method can, in particular, be designed as a computer-implemented method 2023PF02572. In particular, it can be stored as a computer program in memory 115b of system 115 and be executable on the data processing unit 115a. Fig.Figure 5 shows a schematic illustration of an exemplary embodiment of a corresponding system 500 for the automated detection of the occupancy status of a seating arrangement. In the method 400, a radar point cloud 305 is acquired by receiving and further processing radar measurement data in a process 410. In this example, radar measurement data is received from the radar sensor 110 of the vehicle 100, or, with reference to the system 500, by means of corresponding device(s) 510 in the vehicle for acquiring measurement data, in particular from radar sensors 511. The radar point cloud 305 now available can then be clustered in a further process 420 by checking for each of its radar points whether it lies within one of the defined spatial regions 325a to 325e (see Fig. 3B) and, if so, in which one. Thus, each of the points can be assigned either to one of the spatial regions 325a to 325e or to the other observation field.All radar points located within the same spatial area 325a to 325e are grouped into a single cluster. As a result, each of the seats 105a to 105e is assigned a corresponding cluster of radar point cloud 305. This forms the basis for subsequently evaluating, individually for each seat 105a to 105e, whether the respective seat 105a to 105e was occupied or unoccupied while radar point cloud 305 was being generated. For the clustered radar point cloud 305, characteristics can be determined for each cluster. These characteristics can include, in particular, the number of radar points in the cluster. Further characteristics can include the position of a cluster's centroid, the point density in a cluster, or the average Doppler value of a cluster. These characteristics can each be summarized in a feature vector for each cluster.In a process, 430 features are extracted from the measurement data, in particular from the radar point cloud 305 or, if applicable, the clusters. This can be done in a preprocessing step 520 by suitable processing 521 of the radar point cloud 305. If necessary, further (optional) processing procedures 522 can also be carried out here. In a postprocessing step 530, the features, which include static features 531, dynamic features 532, and, if applicable, correspondingly mixed features 533, are processed. Vehicle status data 550 can also be included in the features 531, 532, and 533. These can also be used for the condition estimator 536 and for controlling the functions 540.In process 440, which forms the first stage of the evaluation model, the static features 531 are processed using a first ("static") classifier 534, and the dynamic features 532 are processed separately and in parallel using a second ("dynamic") classifier 535. The static classifier 534 classifies the seat occupancy state as "occupied" and "unoccupied" (other classifications, e.g., "adult present," "child present," and "other," are conceivable; see also the explanations above). The dynamic classifier 535, on the other hand, classifies as "no movement," "movement into the seat," and "movement out of the seat" (alternatively, a simplified classification of "movement" and "no movement" could also be used).The training and, if applicable, validation data used for the prior training of classifiers 534 and 535 can be structured such that each data contains an assigned, correct class of a classification. In this way, the model, or rather the classifiers 534 and 535, can be trained and validated using supervised learning. In a second stage of process 450, the evaluation model uses an optimal state estimator 536 to determine a seat occupancy state for each seat 105a to 105e (or collectively for a row), which constitutes the evaluation result of the evaluation model. The respective partial evaluation results from the static classifier 534 and the dynamic classifier 535 are fed to the state estimator 536 for joint processing in order to estimate, i.e., predict, the evaluation result, as will be described in more detail below.In the second stage, the results of both classifiers 534 and 535 are processed to optimally estimate the current seat occupancy status. When estimating the seat occupancy state, the current and (optionally) previous classification results are used to obtain an estimate of the current seat occupancy status. If the state estimation algorithm includes temporal filtering, the past classification results 2023PF02572 can be used and implicitly stored in the algorithm's internal state information. For seat occupancy detection and classification, the system is modeled by discrete state variables (seat occupied, seat empty). State transitions can be modeled by probabilities for a transition from one state to another from time step k to k+1. Corresponding state models are shown in Fig. 6 and Fig. 7.Figure 7 illustrates this and is described in more detail below. The (discrete) classification results (the partial evaluation results) of classifiers 534 and 535 can also be modeled probabilistically. In particular, discrete state variables sk ∈ {adult on seat, seat empty, ...} are available at each time step k. The probabilities of being in one of the possible states can be given by a vector P whose elements sum to 1. Two examples are worked through below. The first example has a binary state variable with two states. In the second example, a state variable with n states is assumed. In both examples, it is assumed that the measurement is received at the output of a classifier that has two or n classes directly corresponding to the system states, and conditional probabilities can be given for each state sk that lead to a correct measurement M. klead to (and corresponding probabilities for incorrect classifications). As a first example, two discrete possible states s ∈ {Sadult, Sempty} are considered. The probabilities for the states at time step k are: To estimate the state probabilities from the previous cycle P k-1 up to the current cycle P k In order to perform these actions, the probabilities for the state transitions must be known: 2023PF02572 The updated probability for each state element is then given by: For each cycle, measurements are taken and used to classify the system state with a classifier. In this case, the range of possible values ​​for the classifier's output class yk corresponds to the range of system states: yk ∈ {Madult, Mempty}. The classifier's output is uncorrelated over time. The probabilities for a correct and incorrect classification of each system state are assumed to be known: Starting from certain probabilities for the system states, the probabilities for the classifier's output can be derived by: Bayes' theorem can be used to calculate the updated (a posteriori, i.e., conditional) probability Pk when the measurement Yk is available:

[0002] 2023PF02572 The state with the highest probability is selected as the optimal state estimate for the current cycle. Example 2: The system state variable has n possible states s ∈ {S1, S2, ..., Sn}. The corresponding measured values ​​are defined as: y ∈ {M1, M2, ..., M n} 2023PF02572 The state transition probabilities are part of the state model and are assumed to be known. They can also be adaptively set for each time step, i.e., adjusted to the vehicle speed, engine state, etc. The probabilities can also be adjusted if movement in the cabin is detected by a dynamic classifier. If more than one classifier is used (i.e., sA ∈ {sA1, sA2}, sB ∈ {sB1, sB2, SB3}), the results of these multiple classifiers can be interpreted as a single result derived from the set of all combinations of all possible individual results of the classifier (s ∈ {sA1SB1, sA1sB2, sA1sB3, sA2sB1, sA2sB2, sA2sB3}). In this case, Hk will not be a square matrix.The conditional probabilities for the measurements are also modeled, and they can be adaptively set for each time step according to the further conditions or the confidence output of a classifier. Referring to Figures 6 and 7, two examples of corresponding state models are now described. When monitoring a row of seats in a vehicle, m seats per row are assumed (in particular, m = 2 or m = 3). In this case, the n possible seat occupancy states can be specified separately for each seat occupancy state sl ∈ {S1, S2, ..., Sn} | l = 1…m. Instead of considering the seats of a row separately, the seat occupancy state for all seats of a row can be specified as a combination of the substates for each seat, resulting in a total of n. m results in the following states. The overall state (for three seats) s row can be a combination of partial states s row ^ { (S1S1S1), (S1S1S2), ..., (S1S1Sn ), (S1S2S1), ..., (S n S n S n )} = { S row 1 , S row 2 , .... , S row n , S row n+1 , ..., s row n*m} can be specified. This has the advantage that, if the overall system state is determined as a combination of the states of all seats, the conditional probabilities for state transitions can be modeled as the transitions between adjacent seats (e.g., the probability of a state transition from [empty, empty, adult] to [empty, adult, empty] can be set differently than the probability from [empty, empty, empty] to [empty, adult, empty]). This improves the accuracy of the seat occupancy state estimation. With reference to Fig. 6, a first example of a state model 600 for a single seat is described, which contains four state variables S1, …, S4. These include static states (“unoccupied” / “empty” 601 and “occupied” 602) and dynamic states, which represent corresponding movements of a person onto the seat (603) and out of the seat (604).Accordingly, some of the corresponding state transition probabilities P(Sa->Sb), with a,b ∈ {1, …, 4}, are indicated as examples. The second example of a state model 700 shown in Fig. 7, on the other hand, considers a row of two seats. The combined state for two seats (left seat SL and right seat SR) with three states per seat ("empty" 701, "occupied" 702 and "movement" 703) yields 3. 2= 9 states for the row of seats. Accordingly, some of the corresponding state transition probabilities P(Srow a->Srow b), with a,b ∈ {1, …, 9}, are indicated as examples. It goes without saying that four or more states can also be used here, for example, the four states as in the first example. A row of seats can also have more than two seats, in the context of a vehicle in particular three or possibly even more. Now referring again to Fig.4. If, in process 450, a seat occupancy status for the seating arrangement 105 has been determined using the evaluation model, in particular for one or more of its seats 105a to 105e individually, this evaluation result can be output as corresponding information in process 460, for example, at a user interface of the vehicle or in the form of data for further processing by one or more other systems, in particular systems of the vehicle for controlling correspondingly intended functions 540. In the present example, this information is to be used in particular to check, within the framework of a seat belt warning 541, whether or not to issue a seat belt warning signal, depending on the seat occupancy status of a respective seat 105a to 105e and the result of a check as to whether a corresponding seat belt has been fastened for this seat or not.Other functions can be provided alternatively or additionally, such as an airbag control unit 542. For this purpose, process 470 can check whether the seat belt is fastened for the relevant seat (here, for example, seat 105b), and process 480 can control a functionality of the vehicle 100 depending on the seat occupancy information output in process 460 and the seat belt status determined in process 470 2023PF02572. In particular, this can be done by activating a signal source in process 480 to output a seat belt status signal, especially an optical and / or acoustic signal, in order to signal to one or more other vehicle occupants that a seat is occupied but the seat belt is not fastened. The procedure then branches back to process 410 to start another loop iteration.While at least one exemplary embodiment has been described above, it should be noted that a large number of variations exist. It should also be noted that the described exemplary embodiments are merely non-limiting examples, and it is not intended to restrict the scope, applicability, or configuration of the devices and methods described herein. Rather, the preceding description will provide the person skilled in the art with guidance for implementing at least one exemplary embodiment. It is understood that various modifications to the function and arrangement of the elements described in an exemplary embodiment can be made without derogating from the subject matter defined in the appended claims and their legal equivalents.

[0003] 2023PF02572 REFERENCE SYMBOL LIST P Person in the front passenger seat 100 Vehicle 105 Seating arrangement 105a–e Seats or seating positions 110 Radar sensor 110a Radar sensor observation field 110 115 System for automated detection of a seat occupancy state 115a Data processing unit 115b Memory 305 Radar point cloud 310 Static radar points 315 Dynamic radar points 320 Areas of the radar point cloud 305 with high radar point density 325a–e Spatial areas for cluster definition 400 Method for automated detection of a seat occupancy state 410–480 Individual processes or process steps within the procedure 400 500 System for automated detection of a seat occupancy state 510 Device(s) for recording measurement data 511 Radar sensors 520 Preprocessing 521 Processing of measurement data (radar point cloud) 522 Further methods 530 Post-processing 531 Static features 532 Dynamic features 533 Mixed features 534 Static classifier 535 Dynamic classifier536 Condition Estimator 540 Functions 541 Seat Belt Warning 542 Airbag Control 550 Vehicle Status Data 600 Condition Model 601 Seat Empty 2023PF02572 602 Seat Occupied 603 Movement In 604 Movement Out 700 Condition Model 701 Seat Empty 702 Seat Occupied 703 Movement

Claims

2023PF02572 CLAIMS 1. Method for the automated detection of a seat occupancy state of a seating arrangement (105) with at least one seat (105a–e), wherein the method comprises: – Acquiring a data set for a seat occupancy state, wherein the data set represents features of a radar point cloud (305) derived from measurement data, wherein the measurement data each represent an associated radar point cloud (305) at a time point obtained on the basis of a radar scan of a spatial area surrounding the seating arrangement (105) at least partially.is, wherein the features comprise at least one static feature (531) and at least one dynamic feature (532); – Determining a seat occupancy state of the seat arrangement (105) using an evaluation model, wherein the data set forms input data for the evaluation model and the evaluation model, depending on the radar point cloud (305), provides one of several predefined possible seat occupancy states of the seat arrangement (105) as an evaluation result based on the acquired data set; wherein the evaluation model is structured such that: (i) in a first stage, the at least one static feature (531) is processed in at least one first classifier (534) and the at least one dynamic feature (532) is processed in at least one second classifier (535) to obtain a static partial evaluation result orto obtain a dynamic partial evaluation result, wherein the first and second classifiers are executed in parallel; and (ii) in a second stage, using a state estimator (536), to which the obtained static partial evaluation result and the obtained dynamic partial evaluation result are fed, one of several predefined possible seat occupancy states of the seat arrangement (105) is estimated and output as the evaluation result; and – outputting information defined as a function of the evaluation result. 2023PF02572 2. A method according to claim 1, wherein the state estimator (536) is an optimal state estimator which minimizes an error of an estimated seat occupancy state in an iterative process.

3. A method according to claim 1 or 2, wherein the state estimator (536) is a Bayesian filter.

4. A method according to any one of the preceding claims, wherein a state model represents possible seat occupancy states for a seat (105a–e) of the seating arrangement (105), wherein the state estimator considers state transition probabilities that represent probabilities for a transition from a seat occupancy state at a first time point to a seat occupancy state at a subsequent second time point. 5.A method according to any one of claims 1 to 3, wherein the seating arrangement (105) comprises a plurality of seats (105a–e) and, in a state model, possible seat occupancy states are represented collectively for at least a subset of the plurality of seats (105a–e) with more than one of the seats of the seating arrangement, wherein the state estimator considers state transition probabilities that represent probabilities for a transition from a seat occupancy state at a first time point to a seat occupancy state at a subsequent second time point.

6. A method according to claim 5, wherein the subset of the plurality of seats (105a–e) comprises seats (105a–b, 105c–d) of a row of adjacent seats of the seating arrangement (105).

7. A method according to any one of claims 4 to 6, wherein the state transition probabilities are adaptively set for each time point. 8.Method according to one of claims 5 to 7, wherein probabilities for different seat occupancy states are determined using the state transition probabilities, wherein the state estimator outputs as the evaluation result the seat occupancy state with the highest probability. 2023PF02572 9. A method according to any one of the preceding claims, wherein the static partial evaluation result comprises at least one from the following list of static states for a seat (105a–e): occupied, unoccupied, type of seat occupancy.

10. A method according to any one of the preceding claims, wherein the dynamic partial evaluation result comprises at least one from the following list of dynamic states for a seat (105a–e): no movement, movement, movement towards the seat, movement away from the seat.

11. A method according to any one of the preceding claims, wherein in the first stage, in the first classifier (534), the second classifier (535), and / or at least one further classifier, at least one mixed feature (533) of static and dynamic features is processed in addition to the static feature (531) or dynamic feature (532). 12.A method according to any one of the preceding claims, wherein the seating arrangement (105) is a seating arrangement in a vehicle, the evaluation model additionally taking into account vehicle status data (550), the vehicle status data comprising at least one door status, engine status, vehicle speed, and seatbelt fastening status.

13. A method according to any one of the preceding claims, wherein the radar point cloud (305) is segmented into several clusters by assigning to each of the seats (105a–e) as a cluster a subset of the radar points (310, 315) of the respective radar point cloud (305) depending on their respective position such that the radar points (310, 315) of the cluster lie in a defined closed spatial area in the vicinity of the seat (105a–e). 14.Method according to one of the preceding claims, wherein the output of the information comprises controlling a signal source depending on the information in order to cause the signal source to output a defined signal depending on the control, wherein the signal source is controlled depending on the information such that it outputs a signal when the information results from an evaluation result accordingly. 2023PF02572 at least one seat of the seating arrangement (105) is occupied and / or a selected predetermined seat occupancy state exists.

15. Method according to claim 14, further comprising: detecting a seat belt fastening state of at least one seat of the seating arrangement (105) or receiving seat belt information characterizing this seat belt fastening state; wherein the signal source is controlled depending on the seat belt information and the information from the evaluation result such that it outputs a seat belt fastening indicator signal when, according to the information, at least one seat of the seating arrangement (105) is occupied and / or a selected predetermined seat occupancy state exists and seat belt information indicates that the corresponding seat belt of the seat is not fastened. 16.System (115) for automatically detecting the occupancy status of a seating arrangement (105) with at least one seat (105a–e), wherein the system (115) comprises a data processing device configured to perform the method according to any one of the preceding claims for detecting the occupancy status.

17. Computer program or computer program product comprising instructions which, when executed on the data processing device of the system (115) according to claim 16, cause the system (115) to perform the method according to any one of claims 1 to 15. 18.Vehicle (100) comprising: a seating arrangement (105) with at least one seat (105a–e); a radar sensor (110) for at least section-wise radar scanning of the seating arrangement (105); and a system (115) according to claim 16 for automatically detecting a seat occupancy state of the seating arrangement (105) depending on at least section-wise radar scanning of the seating arrangement (105) performed by the radar sensor (110).

Images