Method for producing an evaluation model for detecting a seat occupancy state of a seat arrangement

Abstract

In a method for producing an evaluation model, in particular a machine learning model, for automated detection of a seat occupancy state of a seat arrangement (105) having at least one seat (105a-e), parameters that are assigned to at least one of a plurality of predefined possible seat occupancy states of the seat arrangement (105), and also hyperparameters for the evaluation model, are provided, the hyperparameters being configured to be adjusted on the basis of a metric. A measure (F) is determined which indicates a non-linear relationship between an accuracy (P) and a hit ratio (R), the accuracy (P) indicating the proportion of the evaluation results correctly output by the evaluation model with the provided hyperparameters in the entirety of the corresponding evaluation results, and the hit ratio (R) indicating the probability with which the evaluation model with the provided hyperparameters correctly outputs a corresponding evaluation result. A metric is provided and evaluated, the metric taking into account at least one value from the determined measure in order to output a value for the determined measure. The hyperparameters of the evaluation model are then adjusted accordingly, with the metric being optimized by means of an optimization method in order to obtain an optimum for the output value and to adjust in this way the hyperparameters for which the metric reaches the optimum. The evaluation model is produced with the adjusted hyperparameters for further use for training using training data for a method for automated detection of a seat occupancy state of a seat arrangement (105) having at least one seat (105a-e).

Description

[0001] 2023PF03001

[0002] METHOD FOR CREATING AN EVALUATION MODEL FOR DETECTING THE SEAT OCCUPANCY STATE OF A SEATING ARRANGEMENT

[0003] The present invention relates to a method for creating an evaluation model, in particular a machine learning model, for use in a method for detecting the occupancy status of a seating arrangement with at least one seat. The invention also relates to a method for training the evaluation model, as well as a method, a computer program, and a system configured to execute the method, each for detecting the occupancy status using such a trained evaluation model.

[0004] In various situations, it may be necessary to automatically determine the current occupancy status of a seating arrangement with at least one seat. Such a situation can occur particularly in vehicles, for example, in motor vehicles, where vehicle configuration or the activation, deactivation, and / or control of one or more vehicle functionalities is to be carried out depending on the current occupancy status. For example, it is known in motor vehicles to issue an acoustic or visual warning to the vehicle occupants to fasten their seat belts or to control the activation or deactivation of airbags, depending on a detected occupancy status.

[0005] Methods are known for the automated detection of current seat occupancy, particularly the arrangement of vehicle seats, by monitoring the vehicle's interior. Methods are especially known that utilize radar technology for the automated detection of current seat occupancy. Radar sensors scan the vehicle interior, generating measurement data in the form of radar point clouds. The current seat occupancy can then be determined from the measured radar point cloud.

[0006] To analyze the radar point clouds, a suitable evaluation model is used, which can be a machine learning (ML) model. Known methods using this machine learning model can also identify the type of seat occupancy, for example, whether a seat is occupied by an adult or a child. Methods are also known that can perform a more differentiated analysis of seat occupancy, particularly with regard to the height and weight of an adult (2023PF03001) or the age of a child. For example, it may be necessary to distinguish between a six-year-old and a four-year-old child to account for their different heights.

[0007] When creating the machine learning model, hyperparameters are set using optimization methods. A metric for the recognition accuracy of an evaluation result for known parameters can be used, based on the difference between the correct result (e.g., the actual seat occupancy status) and a result of the evaluation model. With unbalanced datasets, i.e., datasets in which, for example, one class is overrepresented and another underrepresented, the choice of a metric can lead to further improvements in the evaluation results for the overrepresented class, which may already be of very good quality. This is because these results compensate for the, on average, poorer evaluation results (of the underrepresented class), which, conversely, are then not further improved.In the context of a seatbelt warning system, it can therefore happen that the already good detection of average adults is continuously improved, whereas a potentially inadequate detection of, for example, children is not improved.

[0008] It is an object of the present invention to provide an improved solution for the automated detection of the occupancy status of a seating arrangement with at least one seat. In particular, an improved solution for generating an evaluation model and for the automated detection of the occupancy status of a seating arrangement is to be provided, which offers more balanced detection accuracy across different classes.

[0009] The solution to this problem is achieved according to the teaching of the independent claims. Various embodiments and further developments of the invention are the subject of the dependent claims.

[0010] A first aspect of the solution presented here concerns a method, particularly a computer-implemented one, for creating an evaluation model for the automated detection of the occupancy status of a seating arrangement with at least one seat. The method provides parameters that are assigned to at least one of several predefined possible occupancy states of the seating arrangement, as well as hyperparameters for the evaluation model, 2023PF03001, where the hyperparameters are configured to be set based on a metric.

[0011] A measure is defined that indicates a non-linear relationship between accuracy and hit rate. Accuracy represents the proportion of correctly generated results produced by the evaluation model with the provided hyperparameters out of all corresponding evaluation results (i.e., the proportion of correctly classified seat occupancy states out of all seat occupancy states). Hit rate represents the probability that the evaluation model with the provided hyperparameters correctly generates a corresponding evaluation result (i.e., the probability that a seat occupancy state is correctly classified by the evaluation model).

[0012] A metric is provided and evaluated, taking into account at least one value from the specified measure to output a value for that measure. The hyperparameters of the evaluation model are then set accordingly, with the metric being optimized using an optimization procedure. The evaluation model is then created with the set hyperparameters for further use in training with training data for a procedure to automatically determine the occupancy status of a seating arrangement with at least one seat.

[0013] Using the method described in the first aspect, an evaluation model, particularly a machine learning model, can be created that achieves more balanced recognition accuracy for different seat occupancy states. By using a non-linear measure, especially compared to a linear relationship between accuracy and hit rate, classes with poorer results are given greater weight than with a linear relationship. The evaluation model is therefore less prone to errors, particularly in the area of ​​underrepresented classes. For example, training data for young children is not as easy to generate as data for an average adult. The recognition accuracy of the evaluation model created in this way can thus be further improved.This method is particularly advantageous for the machine learning of a radar-based system for detecting seat occupancy. 2023PF03001.

[0014] The term "seat occupancy status" of a seating arrangement with at least one seat, as used herein, refers in particular to information indicating whether or to what extent the seating arrangement, or at least one of its seats, is occupied by an object, especially a thing or a person. In a simple example, the seat occupancy status can simply indicate the presence or absence of an object. In a more advanced example, if at least one object is present on the seating arrangement, or on one or more of its seats, it can provide information about the type or other property of the object, such as its spatial extent.

[0015] The term "evaluation model," as used herein, refers to a model, particularly a mathematical one, that uses a radar point cloud or one or more characteristic parameters as input(s) to deliver an evaluation result based on these parameters, in this case, one of several predefined possible seat occupancy states of the seating arrangement. The evaluation model can, in particular, be a mathematical estimator, where the radar point cloud represents empirical data as a sample and the evaluation result is an estimated value determined based on this sample. The evaluation model can, in particular, be a "machine learning model" (or a computer model).(Synonymous with "machine learning model"), which here refers in particular to a mathematical, especially statistical, model for making predictions or decisions, created using at least one machine learning algorithm based on example data referred to as training data, without the algorithm(s) being explicitly programmed to make such predictions or decisions. In particular, decision tree-based machine learning models (decision trees) are machine learning models.

[0016] The term "metric," as used herein, refers in particular to a mathematical mapping that assigns a numerical value to one or more values ​​of the measure, especially a combination of values ​​such as a mean. The metric is used in particular for hyperparameter optimization by feeding its value into an optimization procedure in order to adjust the hyperparameters accordingly. Specifically, weights in the evaluation model can be adjusted to achieve a specific behavior of the evaluation model. 2023PF03001

[0017] The term "parameter," as used herein, refers in particular to a value relating to the seat occupancy status, as it may also occur in the actual operation of the evaluation model during a procedure for detecting seat occupancy. The parameters may, in particular, indicate whether a seat is occupied or not, how a seat is occupied (e.g., adult or child), and, where applicable, other circumstances such as the vehicle's condition (e.g., whether it is moving or stationary).

[0018] The term "hyperparameter," as used herein, refers specifically—in contrast to the parameters explained previously—to a parameter of the evaluation model that is determined during the model's creation in order to control the model during training. The hyperparameters are thus determined and defined before the actual model training.

[0019] The term "radar point cloud," as used herein, refers in particular to a set of points in a vector space obtained by radar scanning of at least one object's surface, which exhibits a typically unorganized spatial structure ("cloud"). In the case of a radar point cloud, the points of the radar point cloud can be referred to as "radar points." A (radar) point cloud can be described in particular by the (radar) points it contains. The radar points, in turn, can each be described in particular by their spatial coordinates, which, for each radar point, indicate the location of the reflection of an emitted radar signal from an object's surface, as measured during the radar scan. Additional attributes, such as the measured Doppler velocity or a signal-to-noise ratio (SNR), can be recorded for the radar points.

[0020] Any terms used herein, such as "comprises," "includes," "features," "has," "with," or any other variant thereof, are intended to cover non-exclusive inclusion. For example, a method or apparatus that includes or features a list of elements is not necessarily limited to those elements but may include other elements not expressly listed or inherent in such method or apparatus.

[0021] Furthermore, unless expressly stated otherwise, "or" refers to an inclusive or and not an exclusive "or". For example, condition A or B is satisfied by either of the following: A is true (or exists) and 2023PF03001

[0022] B is false (or not present), A is false (or not present) and B is true (or present), and both A and B are true (or present).

[0023] The terms "ein" or "eine," as used here, are defined as "one or more." The terms "ein anderer" and "ein Weitere," as well as any other variant thereof, are to be understood as "at least one more."

[0024] The term “plural” or “several”, as used here, is to be understood in the sense of “two or more”.

[0025] The terms "configured" or "set up" (and any variations thereof) used herein to fulfill a specific function are understood, within the meaning of the invention, to mean that the corresponding device already exists in a configuration or setting in which it can perform the function, or at least that it is adjustable—i.e., configurable—so that it can perform the function after appropriate adjustment. Configuration can be achieved, for example, by adjusting parameters of a process sequence or by using switches or similar devices to activate or deactivate functionalities or settings. In particular, the device can have several predetermined configurations or operating modes, so that configuration can be carried out by selecting one of these configurations or operating modes.

[0026] The following describes various exemplary embodiments of the method, which, unless expressly excluded or technically impossible, can be combined with each other and with the other described aspects of the present solution.

[0027] In some embodiments, the measure (F) for an accuracy at which the proportion of correctly output results by the evaluation model with the provided hyperparameters is less than the total number of corresponding evaluation results indicates a hit rate that is higher than that of a corresponding linear relationship between accuracy and hit rate. It is understood that at an accuracy of 0, the hit rate is 0 for both a linear and a non-linear relationship; likewise, the hit rate is always 1 (100%) if all evaluation results are correct. Due to the continuous nature of the curves, which can be assumed in any case, it follows that while in a proportion range as described above the measure always has a higher 2023PF03001

[0028] The hit rate is particularly high when the curve deviates significantly from a linear relationship, while the curve flattens out towards the end. This means that already good results (especially for well-represented classes) are evaluated hardly differently than with a linear relationship. However, poorer results (especially for underrepresented classes) receive a higher hit rate value from this measure, so that even in this range, a suitably trained model will deliver good results.

[0029] In some embodiments, the measure is the F-measurement, where the F-measurement is calculated according to the following formula, where P is the accuracy ("precision") and R is the hit rate ("recall"):

[0030] In this case, the F-measurement can also be called the Fi-measurement, where accuracy and hit rate are weighted equally. It should be noted that other weightings are also conceivable in principle, in which case the F-measurement is generally referred to as F.a -measure can be called (for positive values ​​of a). The formula then reads:

[0031] In some embodiments, providing the parameters includes providing a set of parameters associated with multiple seat occupancy states, and determining the measure for each combination of parameters from the set of parameters, where the metric is defined as a mean value calculated for the set of parameters and the respective measure. The mean value can, for example, be the geometric mean. Calculating the mean value ensures that all values ​​are optimized, and in particular maximized, by the optimization procedure. A set of parameters can, for example, contain combinations of parameters where a value is predetermined, such as a specific seat in the seating arrangement or whether the engine is running or not. A combination of driver size, seat, and vehicle state might, for example, be: a small woman (AF05) sitting in the driver's seat with the vehicle stationary.The measure can then be determined (calculated) for all possible combinations of the selected parameters. 2023PF03001.

[0032] In some embodiments, the optimization method is a Bayesian optimization. In particular, the value of the metric can be maximized using Bayesian optimization. By using the described measure, especially the F-measure, it is not necessary to first normalize the values ​​of the measure, since the measure is already designed such that its values ​​always lie in the interval [0; 1].

[0033] In some implementations, the optimization procedure is performed with a predetermined number of iterations. While, for example, Bayesian optimization theoretically allows any number of iterations, it can be advantageous to limit the number of iterations and, after this number, adjust the hyperparameters so that they represent the most promising hyperparameters for the evaluation model.

[0034] In some embodiments, the seating arrangement comprises multiple seats, with the seat occupancy state being an individual or cumulative occupancy state of the seats. In other words, a seat occupancy state can encompass one or more seats of the vehicle and their respective occupancy (see also the discussion of radar point cloud cluster formation below).

[0035] In some embodiments, the seat occupancy state includes at least one type of seat occupancy for at least one seat in the seating arrangement. In particular, determining the seat occupancy state can include determining a type of seat occupancy. The type of seat occupancy can include at least one unoccupied (i.e., free) seat, a seat occupied by an adult, a seat occupied by a child, a seat occupied by a child seat with a toddler, and a seat occupied by an infant carrier with a baby. Additionally, it can be provided to detect whether a seat is occupied, for example, by an object or a doll. A person can be simulated, for example, using the established THUMS human model. Here, the values ​​AM95, AM50, and AF05, for example, represent different adult persons (tall / heavy man, medium / medium-weight man, and small / light woman, respectively).Children can be identified by their age, e.g. 4YO or 6YO for a four-year-old or six-year-old child.

[0036] By determining not only whether a seat is occupied, but also how, and outputting the corresponding information, more nuanced control of, for example, a seatbelt warning system or airbag system (2023PF03001) is possible. For instance, certain airbags can only be activated if an adult occupies a seat, while an airbag can (or must) be deactivated if a seat is occupied, but not by an adult, but rather by, for example, a rear-facing infant car seat. Furthermore, when using a child car seat or infant carrier, which are typically not secured with a seatbelt but rather with a special attachment system (e.g., "Isofix"), it is advantageous if a seatbelt warning is not issued, thus avoiding unnecessary warning messages that are unhelpful and may be perceived as annoying by the user.

[0037] In some embodiments, the parameters also include the vehicle's state. This state can specifically include whether the vehicle is stationary or moving and / or whether the engine is running. Considering the vehicle's state can positively influence the evaluation model, as this can affect the radar measurement result, since varying levels of vehicle vibration can impact the radar monitoring of the vehicle's interior.

[0038] A second aspect of the solution presented here concerns a method, particularly a computer-implemented one, for training an evaluation model—created according to a method based on a first aspect—for the automated detection of the occupancy status of a seating arrangement with at least one seat. In this method, measurement data representing an associated radar point cloud is acquired. This radar point cloud is obtained from a radar scan of a spatial area that at least partially surrounds the seating arrangement and is assigned to one of several predefined possible occupancy states. Training data is then generated from the measurement data and provided as input to the evaluation model. The model's output is an evaluation result that corresponds to the occupancy status of the seating arrangement.The evaluation model created according to the first aspect is less susceptible to unbalanced training data, since real (training) data for average adults, in particular, can be obtained more easily in larger quantities than, for example, for small children or very tall or heavy adults. 2023PF03001.

[0039] A third aspect of the solution presented here concerns a method, particularly a computer-implemented one, for the automated detection of the occupancy status of a seating arrangement with at least one seat (or equivalently: seat), especially a seat in or for a vehicle, such as an automobile (e.g., truck, car, or bus). The method comprises: (i) acquiring, in particular receiving or generating, measurement data that represent an associated radar point cloud.Each radar point cloud is or was obtained based on a radar scan of a spatial area that at least partially surrounds the seating arrangement; (ii) determining, in particular estimating, a seat occupancy state of the seating arrangement using an evaluation model that, depending on the radar point cloud, delivers one of several predefined possible seat occupancy states of the seating arrangement as an evaluation result; and (iii) outputting information defined as a function of the evaluation result. The procedure according to the third aspect uses an evaluation model, in particular a machine learning model, which was trained using the procedure according to the second aspect.

[0040] Using the method described in the third aspect, a radar point cloud obtained by means of radar scanning of the area surrounding the seating arrangement allows for an evaluation result (particularly in the sense of a prediction or classification) that characterizes the occupancy status of the seating arrangement. This enables the implementation of radar-based solutions, especially in the vehicle context (particularly for automobiles), that can reliably detect a seat occupancy status (particularly exclusively) using radar and, based on this, activate, deactivate, or control / regulate certain functionalities or systems, such as a seatbelt warning system or an airbag system, either completely or selectively.

[0041] The information to be output can, in particular, represent the evaluation result itself. It can also be a detectable signal, especially one perceptible to a human sense, such as a warning, a control signal for activating a signal source, or a data signal carrying the information.

[0042] The following are various exemplary embodiments of the method according to the third aspect, which, unless expressly excluded or technically impossible, can be combined with each other as desired and with the other aspects of the present solution described in 2023PF03001.

[0043] In some embodiments, the seating arrangement has multiple seats, for which an occupancy status must be determined individually or cumulatively within the framework of the method. For each (individual) radar point cloud, the set of its radar points is subdivided into several clusters, each containing a subset of the radar points, by means of cluster formation, depending on the respective spatial location of the radar points in relation to the seats, in order to individually assign each seat to the cluster closest to it. The determination of the occupancy status of the seating arrangement is carried out for each seat depending on the radar point cloud determined for the respective associated cluster, in order to obtain an evaluation result, in particular a classification result, that characterizes the occupancy status of the respective seat.The information to be displayed is then defined depending on the respective individual evaluation results for the different seats.

[0044] In some related embodiments, each radar point cloud is segmented into several clusters by assigning each seat a subset of the radar points from the respective radar point cloud as a cluster, depending on their respective positions, such that the radar points of the cluster lie within a defined closed spatial area, in particular a cuboid, in the vicinity of the seat. This enables particularly simple and computationally efficient cluster formation and thus seat-related seat occupancy detection, whereby the location (position and orientation) and shape of the spatial area is or can be defined such that it strongly overlaps the spatial area typically occupied by a typical object to be detected, in particular a person, on a seat in the seating arrangement.

[0045] In the case of a multi-seat seating arrangement, this enables seat-specific, i.e., individual, seat occupancy detection, which is particularly advantageous or even necessary when a seat-specific response to the detected occupancy is required, for example, by activating, deactivating, or otherwise controlling a specific functionality or system for a particular seat, such as a seat-specific airbag system, a seat-specific seatbelt reminder, or a seat-specific seat heater, depending on its detected occupancy. In some embodiments of 2023PF03001, cluster formation can be achieved in such a way that the clusters are disjoint, so that no radar point is assigned to two different clusters.

[0046] In some embodiments, the radar point cloud, or each individual radar point cloud, is segmented into several clusters by assigning a subset of the radar points to each seat as a cluster, depending on their respective position, in such a way that the radar points of the cluster lie within a defined, closed, particularly cuboid, spatial region in the vicinity of the seat. The assignment can be carried out in such a way that each radar point is assigned to the cluster of the seat nearest to it. In this way, the radar point cloud can be divided into clusters, i.e., subsets of the radar point cloud located in the vicinity of the respective seats, so that the determination of seat-specific occupancy states can be carried out in a targeted and therefore highly reliable manner based on the cluster assigned to the respective seat.

[0047] In some embodiments, outputting the information includes controlling a signal source based on the information, causing the signal source to output a defined signal depending on the control. The signal source can be, in particular, an audio source, an optical signal source, especially a display device for images or text, and / or a haptic actuator, or a combination of at least two of the aforementioned signal sources. Thus, the detected seat occupancy status can be communicated to a user based on the signaling or used to control another technical system, such as an airbag system.

[0048] In some of these embodiments, the signal source is controlled depending on the information in such a way that it outputs a signal, defined in particular by the control, when the information results from an evaluation result according to which at least one seat of the seating arrangement is occupied and / or a selected predetermined seat occupancy state exists.

[0049] In some embodiments, the method further comprises: (i) detecting a seatbelt fastening state of at least one seat of the seating arrangement or receiving seatbelt information characterizing this seatbelt fastening state; (ii) wherein the signal source is controlled depending on the seatbelt information and the information from the evaluation result such that it outputs a seatbelt fastening warning signal when, according to the information, at least one seat of the seating arrangement is occupied and / or a selected predetermined 2023PF03001

[0050] The system detects seat occupancy and indicates that the corresponding seat belt is not fastened. This allows for a radar-based seat belt fastening check and warning system, particularly with regard to detection.

[0051] In some embodiments, the individual radar points of the radar point cloud are each represented by a position of the respective radar point in three-dimensional space and by at least one of the following parameters: (i) a Doppler shift value of the radar signal relative to the respective radar point; (iii) a signal-to-noise ratio value of the radar signal relative to the respective radar point. These parameters can be used, in particular, for pre-filtering the radar point cloud as part of a pre-processing step preceding feature determination.

[0052] A fourth aspect of the present solution concerns a system, in particular a data processing device, for the automated detection of the occupancy status of a seating arrangement with at least one seat, in particular with at least one vehicle seat in or for a vehicle. The system includes a data processing device that is configured, in particular by means of a corresponding computer program, to execute the procedure according to the third aspect for detecting the occupancy status.

[0053] A fifth aspect of the present solution relates to a computer program or computer program product comprising instructions which, when executed on the data processing device of the system according to the fourth aspect, cause the system to execute the procedure according to the third aspect.

[0054] The computer program can be stored, in particular, on a non-volatile data carrier. Preferably, this is a data carrier in the form of an optical data carrier or a flash memory module. This can be advantageous if the computer program as such is to be handled independently of a processor platform on which the one or more programs are to be executed. In another implementation, the computer program can exist as a file on a data processing unit, in particular on a server, and be downloadable via a data connection, for example the Internet or a dedicated data connection, such as a proprietary or local network. In addition, the computer program can have a plurality of interacting individual program modules. The modules can, in particular, be configured to [2023PF03001] or at least be usable in such a way that they can be used in the sense of distributed computing (i.e., distributed computing)."Distributed computing" is performed on different devices (computers or processor units) that are geographically separated and connected via a data network.

[0055] The system according to the fourth aspect can accordingly have a program memory in which the computer program is stored. Alternatively, the system can also be configured to access an external computer program, for example on one or more servers or other data processing units, via a communication link, in particular to exchange data with it that is used during the execution of the procedure or computer program or represents outputs of the computer program.

[0056] A sixth aspect of the present solution relates to a vehicle comprising: (i) a seating arrangement with at least one seat; (ii) a radar sensor for at least section-by-section radar scanning of the seating arrangement; and (iii) a system according to the fourth aspect for automatically detecting a, in particular, respective, seat occupancy state of the seating arrangement as a function of at least section-by-section radar scanning of the seating arrangement performed by the radar sensor, in particular according to a method according to the third aspect.

[0057] The features and advantages explained in relation to one aspect of the present solution also apply accordingly to the other aspects of the solution.

[0058] Further advantages, features and application possibilities of the present solution will become apparent from the following detailed description in conjunction with the drawings.

[0059] This shows:

[0060] Fig. 1 schematically shows an exemplary embodiment of a vehicle equipped with a system for automatically detecting the occupancy status of a seating arrangement in the vehicle;

[0061] Fig. 2 schematically shows the vehicle from Fig. 1, where the passenger seat is occupied;

[0062] Fig. 3A is an exemplary two-dimensional representation of a radar point cloud recorded by a radar sensor of the vehicle from Fig. 2; 2023PF03001

[0063] Fig. 3B is an exemplary representation of a clustering of the radar point cloud from Fig. 3A according to the positions of the individual seats of the seating arrangement;

[0064] Fig. 4 is a flowchart illustrating an exemplary embodiment of a method for the automated detection of a seat occupancy status of a seating arrangement;

[0065] Fig. 5 is a flowchart illustrating an exemplary embodiment of a method for creating an evaluation model for the automated detection of a seat occupancy status of a seating arrangement; and

[0066] Fig. 6 shows an example of a curve for a measure.

[0067] In the figures, identical reference symbols denote identical, similar, or corresponding elements. Elements depicted in the figures are not necessarily shown to scale. Rather, the various elements depicted in the figures are represented in such a way that their function and general purpose are understandable to a person skilled in the art. Connections and couplings between functional units and elements shown in the figures can, unless expressly stated otherwise, also be implemented as indirect connections or couplings. Functional units can, in particular, be implemented as hardware, software, or a combination of hardware and software.

[0068] First, the detection of seat occupancy in a vehicle using a radar system is described with reference to Figures 1 to 4, since the method according to the present invention is advantageously applicable in this context. The present invention is then explained in particular with reference to Figures 5 and 6.

[0069] The exemplary embodiment of a vehicle 100, schematically depicted in Fig. 1, has a seating arrangement 105 with five individual seats or seating positions 105a to 105e. Each of the seats 105a to 105e is suitable for accommodating one person as a passenger of the vehicle 100. The vehicle 100 further has a radar sensor 110, which is mounted on the ceiling inside the vehicle cabin and configured to scan the seating arrangement 105, at least substantially, using radar beams. Accordingly, the seats 105a to 105e, in particular their seating surfaces, are located, at least predominantly, within a field of view 10a that can be scanned by the radar sensor 110.Furthermore, vehicle 100 2023PF03001 has a system 1 15 for the automated detection of a seat occupancy state of the seat arrangement 105 depending on a radar scan of the seat arrangement 105 carried out by the radar sensor 1 10, at least sectionally with respect to the observation field 110a.

[0070] The system 115 comprises, in particular, a data processing unit 115a with at least one microprocessor and a signal-connected memory 115b in which a computer program configured for carrying out the method for automatically detecting the occupancy status of the seat arrangement 105, as described below with reference to Fig. 4, is stored. Furthermore, the sensor data generated by the radar sensor 110 during radar scanning, or information already obtained from it through further processing, may be stored or be stored in the memory 115b.

[0071] The vehicle 100 shown in Fig. 2 corresponds to the vehicle in Fig. 1, except that here the passenger seat 105b is occupied by a person E. The subsequent discussion of Figs. 3A and 3B refers to the configuration in Fig. 2.

[0072] Reference is now made to Figures 3A and 3B, which each represent a radar point cloud, whereby, for the purpose of representation, the respective, inherently three-dimensional radar point cloud was reduced to two dimensions by projecting the positions of the radar points of the radar point cloud onto a plane spanned by two of its dimensions.

[0073] Figure 3A illustrates an exemplary radar point cloud 305, as it was acquired as a result of a radar scan of the seating arrangement 105 by the radar sensor 110 during a defined time interval (measurement period). The position of the individual radar points within the radar point cloud 305 can be represented by spatial coordinates; for example, Cartesian coordinates X and Y can be assigned to the plane of the drawing and to each individual point. In reality, if the dimensional reduction due to the drawing is disregarded, a third coordinate Z for the third spatial dimension is also required.

[0074] If, during radar scanning, not only the spatial positions of the points where the radar beam is reflected by the scanned objects are recorded as coordinates, but also a corresponding Doppler shift is measured, then the individual radar points can be classified according to the magnitude of this Doppler shift, in particular divided into two different classes 2023PF03001. The latter can be done, for example, by comparing the Doppler shift with a predefined shift threshold that corresponds to a specific shift velocity. Depending on the result of the comparison, those radar points 310 that, according to the value of their associated Doppler shift, have no velocity or a velocity of the object surface at the reflection point that is below the shift wave can be classified as "static" radar points (in the Fig.3A and 3B are each represented by a filled black circle). Conversely, those radar points 315 that exhibit a Doppler shift above the shift threshold can be classified as “dynamic” radar points 315 (represented by a black ring in Figs. 3A and 3B).

[0075] Classifying radar points 310 and 315 according to their Doppler shift is not strictly necessary; however, it can be used to process radar point cloud 305, particularly as part of preprocessing prior to evaluation, and especially to filter based on the classification. For example, such filtering could be performed to consider only dynamic radar points 315 for evaluation, in order to detect only moving objects.

[0076] Figure 3B shows the same radar point cloud 305 as Figure 3A. However, in addition, selected spatial regions 325a to 325e, which are cuboid (in the 3D case) or rectangular (in the present 2D representation), are shown here. These spatial regions correspond to the respective locations of the individual seats 105a to 105e. The definition of these spatial regions 325a to 325e can now be used to cluster the radar point cloud 305, whereby each radar point 310 or 315 is assigned, where possible, to the spatial region 325a to 325e in which it lies. All radar points not located in one of the spatial regions 325a to 325e can be disregarded. It can be seen in particular that the regions 320 with a particularly high radar point density are located in the area of ​​the front passenger seat 105b, where, according to Figure 2, person E is located.

[0077] Fig. 4 shows a flowchart illustrating an exemplary embodiment 400 of a method for automatically detecting the occupancy status of a seating arrangement. The method can, in particular, be implemented as a computer-implemented method. For this purpose, it can, in particular, be stored as a computer program in memory 115b of system 115 and be executable on the data processing unit 115a. 2023PF03001

[0078] In method 400, a radar point cloud 305 is acquired by receiving radar measurement data in a step 410, in the present example from the radar sensor 1 10 of the vehicle 100, and processing it to form one or more radar point clouds.

[0079] The resulting radar point cloud 305 can then be clustered in a further process 420 by checking for each of its radar points whether it lies within one of the defined spatial areas 325a to 325e (see Fig. 3B) and, if so, in which one. Thus, each of the points can be assigned either to one of the spatial areas 325a to 325e or to the other observation field. All radar points that lie within the same spatial area 325a to 325e are grouped into a respective cluster. As a result, each of the seats 105a to 105e is assigned a corresponding cluster of radar point cloud 305. This forms the basis for subsequently conducting an individual evaluation for each seat 105a to 105e to determine whether the respective seat 105a to 105e was occupied or unoccupied while radar point cloud 305 was being generated.

[0080] To facilitate the subsequent evaluation of the clustered radar point cloud 305, a corresponding parameter K can be determined for each cluster in a process 430, whereby this parameter K can be defined, in particular, as the number of radar points in the cluster. If no filtering according to the Doppler shift value has taken place, this can be a combined count of both the static and the dynamic radar points 310 and 315, respectively. However, if the static radar points 310 have been filtered out beforehand, it is only a count of the dynamic radar points 315.

[0081] The evaluation of the parameter K for the cluster corresponding to seat 105b can now be performed (the same can be done analogously for the respective clusters corresponding to the other seats). For this purpose, in process 440, the parameter K (or a time series of the parameter K) is provided as input to an evaluation model. This can be, in particular, a machine learning-based model, such as an artificial neural network or a decision tree-based model. The training and, if applicable, validation data used for the preceding training can be structured in such a way that they contain, according to the type of parameter K, a multitude of different radar point clouds or clusters thereof, and, for each series of K, an assigned correct class of a classification of possible seat occupancy states. Thus, the model in 2023PF03001

[0082] The systems are trained and validated using supervised learning methods. In the simplest case, seat occupancy states indicate whether a seat is occupied or not. However, more advanced classifications are also conceivable, where, in the case of an object's presence, the respective class additionally specifies the type of object, for example, whether it is moving or stationary, and, in the case of a moving object, specifically whether it is a person (generally recognizable by a breathing pattern in the course of the parameter K).

[0083] If, in process 440, a seat occupancy status for the seating arrangement 105 has been determined using the evaluation model, in particular for one or more of its seats 105a to 105e individually, this result can be output as corresponding information in process 445, for example at a user interface of the vehicle or in the form of data for further processing by one or more other systems, in particular systems of the vehicle.

[0084] In the present example, this information is to be used in particular to check whether, depending on the occupancy status of a respective seat 105a to 105e and the result of a check as to whether a corresponding seat belt has been fitted for this seat or not, a seat belt warning signal should be issued or not.

[0085] In process 450, it can be checked whether the seat belt is fastened for the relevant seat (here, for example, seat 105b). In step 455, a function of the vehicle 100 can be controlled depending on the seat occupancy information output in process 445 and the seat belt status determined in process 450. Specifically, this can be done by activating a signal source in process 455 to output a seat belt status signal, particularly visual and / or audible, to indicate to one or more other vehicle occupants that a seat is occupied but the seat belt is not fastened. The process then branches back to step 410 to start another loop iteration.

[0086] The evaluation model is created before the actual application in procedure 400 and before training. In particular, the hyperparameters are optimized accordingly, as explained below. A flowchart illustrating an exemplary embodiment of procedure 500 for creating the evaluation model is shown in Fig. 5. 2023PF03001

[0087] In step 510, a dataset with various seat occupancy states (parameters) is first provided. This is processed in the evaluation model. In particular, this serves to optimize the hyperparameters before the created evaluation model is finally trained with the training data (step 560). For hyperparameter optimization, a metric is applied which is optimized using an optimization method 501 (here Bayesian optimization).

[0088] The dataset contains various seat occupancy states, including people, objects, and empty seats. For each seat, a decision must be made as to whether a seatbelt reminder (i.e., a seatbelt warning signal) should be issued, as explained above. This decision is based on the presence of a person and their size (for children, age is used, as it correlates strongly with size). The dataset is highly unbalanced with respect to passenger age and size. Therefore, the different types of passengers (or seat occupancy states) are weighted to achieve optimal overall classification without causing poor detection accuracy for an underrepresented group of passengers (e.g., AM95). (Detection accuracy for frequently occurring groups, such as medium-sized adults (like AM50), should be relatively good anyway.)These weights are optimized using Bayesian optimization to obtain a more balanced overall result.

[0089] A key component is the selection of the metric. The metric described below is therefore based, in particular, on a non-linear relationship between accuracy and hit rate. Accuracy represents the proportion of correct results generated by the evaluation model with the provided hyperparameters out of all corresponding evaluation results, while hit rate represents the probability that the evaluation model with the provided hyperparameters correctly generates a corresponding evaluation result, as will be explained in more detail below. The measure is defined for each possible combination, as described below.

[0090] First, a series of parameters is selected, i.e., various seat occupancy states for which a balance is to be established, e.g., whether the engine is running or not, or the occupancy of a specific seat 105a to 105e. A combination would be, for example: AF05 (small woman) in the driver's seat, while 2023PF03001 the vehicle is stationary. This is a combination of the person's size, the seat, and the vehicle's condition.

[0091] The accuracy is calculated for all possible combinations of the selected parameters. For each of these calculated accuracies, a hit rate is then determined using a suitable measure, in particular the F1 measure as shown in Fig. 6. Due to the non-linear relationship, poorer results, which can occur particularly with classes that are underrepresented in the training data, are assigned a higher hit rate compared to a linear relationship.

[0092] The geometric mean of each of these values ​​is then calculated. This ensures that all values ​​are maximized, not just one. Because the Fi measure is used, the resulting value lies within the range of 0 to 1 even without normalization. This value is then used as input for Bayesian optimization, which aims to maximize this value.

[0093] For Bayesian optimization 501, a fixed number of iterations (m) is defined before starting. Bayesian optimization optimizes the metric described above to set the hyperparameters ("hyperparameter optimization"). Bayesian optimization 501 treats the function defining the metric as a random function that it attempts to estimate. The number of iterations is fixed because, due to the implementation, it theoretically has no limit. The hyperparameters are provided, with a range of input values ​​specified for each hyperparameter, and Bayesian optimization 501 selects random values ​​for the first few iterations (here: 10) (step 520).After 10 iterations, in which the evaluation model is trained with the parameters from the data set and the metric is evaluated (step 530), the Bayesian optimization 501 selects the most promising hyperparameters (step 540).

[0094] In the next (m-1) iterations, where the evaluation model is further trained with the parameters from the dataset and the metric is evaluated accordingly (step 550), the most promising hyperparameters are always selected. Specifically, the next selected hyperparameters are those from the range of input values ​​with the highest values. After the chosen number of iterations has been completed (here, a total of m), the result of the process (i.e., the Bayesian optimization 501) 2023PF03001 is the set of hyperparameters that was evaluated and leads to the lowest measured value.

[0095] As mentioned above, in step 560 the evaluation model is finally trained with the selected hyperparameters and saved. It turns out that a better balance for the different groups can be achieved in the evaluation results.

[0096] Figure 6 shows an example of a curve for a measure F, specifically the "F-measure" (i.e., the so-called "Fi-measure") for a set of 100 evaluation results. Due to the non-linear relationship between accuracy P and hit rate R, the curve flattens out in the upper range, which in turn means that classes with poorer results are given more weight than with a linear relationship (see curve A in Figure 6). For example, when using the Fi-measure, the accuracy of 50 correct evaluation results (out of 100) is not assigned a hit rate T of 0.5, but a higher value of approximately 0.6.

[0097] While at least one exemplary embodiment has been described above, it should be noted that a large number of variations exist. It should also be noted that the described exemplary embodiments are merely non-limiting examples, and it is not intended to restrict the scope, applicability, or configuration of the devices and methods described herein. Rather, the preceding description will provide the person skilled in the art with guidance for implementing at least one exemplary embodiment. It is understood that various modifications to the function and arrangement of the elements described in an exemplary embodiment can be made without derogating from the subject matter defined in the appended claims and their legal equivalents. 2023PF03001

[0098] REFERENCE MARK LIST

[0099] E person in the passenger seat

[0100] 100 vehicles

[0101] 105 Seating arrangement

[0102] 105a-e seats or seating places

[0103] 110 radar sensor

[0104] 110a Observation field of the radar sensor 110

[0105] 115 System for automated seat occupancy detection

[0106] 115a Data processing unit

[0107] 115b memory

[0108] 305 radar point cloud

[0109] 310 static radar points

[0110] 315 dynamic radar points

[0111] 320 areas of radar point cloud 305 with high radar point density

[0112] 325a-e Spatial areas for cluster definition

[0113] 400 methods for the automated detection of seat occupancy status

[0114] 410-455 individual processes or procedural steps within the procedure 400

[0115] 500 methods for creating an evaluation model

[0116] 501 Optimization methods

[0117] 510-560 individual processes or procedural steps within the procedure 500

[0118] F measure

[0119] P Accuracy

[0120] R hit rate

[0121] A Recognition accuracy

Claims

2023PF03001 REQUIREMENTS 1. Method for creating an evaluation model for automated detection of the occupancy status of a seating arrangement (105) with at least one seat (105a-e), wherein the method comprises: Providing parameters that are assigned to at least one of several predefined possible seat occupancy states of the seat arrangement (105); and Providing hyperparameters for the evaluation model, where the hyperparameters are configured to be set based on a metric; Determining a measure (F) that indicates a non-linear relationship between an accuracy (P) and a hit rate (R), where the accuracy (P) indicates the proportion of the evaluation results correctly output by the evaluation model with the provided hyperparameters out of the total of the corresponding evaluation results, and the hit rate (R) indicates the probability with which the evaluation model with the provided hyperparameters correctly outputs a corresponding evaluation result; Providing a metric and evaluating the metric, whereby the metric takes into account at least one value from the specified measure in order to output a value for the specified measure; Setting the hyperparameters, whereby the metric is optimized using an optimization procedure; and Creating the evaluation model with the set hyperparameters for training with training data for a method for automatically determining the occupancy status of a seating arrangement (105) with at least one seat (105a-e).

2. Method according to claim 1, wherein the measure (F) for an accuracy at which the proportion of evaluation results correctly output by the evaluation model with the provided hyperparameters is less than the total number of corresponding evaluation results indicates a hit rate that is greater than that of a corresponding linear relationship between accuracy and hit rate.

3. Method according to claim 1 or 2, wherein the dimension (F) is the F-dimension, wherein the F-dimension is calculated according to the following formula: 24 / 28 2023PF03001 where P is the accuracy (P) and Fl is the hit rate (R).

4. Method according to one of the preceding claims, wherein providing the parameters comprises providing a set of parameters that are assigned to multiple seat occupancy states and the measure (F) is determined for combinations of parameters of the set of parameters, wherein the metric is defined as a mean value which is formed for the set of parameters and the respective measure (F).

5. Method according to any one of the preceding claims, wherein the The optimization method is a Bayesian optimization.

6. Method according to any one of the preceding claims, wherein the Optimization procedures are performed with a predetermined number of iterations.

7. Method according to any of the preceding claims, wherein the seating arrangement (105) comprises a plurality of seats (105a-e), wherein the seat occupancy state is an individual or cumulative seat occupancy state of the seats (105a-e).

8. Method according to one of the preceding claims, wherein the seat occupancy state comprises at least one type of seat occupancy for at least one seat of the seating arrangement.

9. Method according to one of the preceding claims, wherein the parameters further comprise a state of the vehicle.

10. Method for training an evaluation model created according to a method according to one of the preceding claims for the automated detection of a seat occupancy state of a seating arrangement (105) with at least one seat (105a-e), wherein the method comprises: acquiring measurement data representing an associated radar point cloud (305), wherein the radar point cloud (305) is based on a radar scan of the seating arrangement (105) at least sectionally 2023PF03001 surrounding area was or is being obtained and is assigned to one of several predefined possible seating occupancy states of the seating arrangement (105); and Generating training data from the measurement data, whereby the training data is provided as input data to the evaluation model in order to obtain an evaluation result as its output, which is assigned to the seat occupancy state of the seat arrangement (105).

11. Method for automatically detecting the occupancy status of a seating arrangement (105) with at least one seat (105a-e), wherein the method comprises: Acquisition of measurement data representing an associated radar point cloud (305), wherein each radar point cloud (305) was or is obtained on the basis of a radar scan of a spatial area surrounding the seating arrangement (105) at least section by section; Determining a seat occupancy state of the seating arrangement (105) using an evaluation model trained according to a method according to claim 10, which, depending on the radar point cloud, delivers one of several predefined possible seat occupancy states of the seating arrangement (105) as an evaluation result; and Output of information defined based on the evaluation result.

12. Method according to claim 1, wherein the output of the information comprises controlling a signal source depending on the information in order to cause the signal source to output a defined signal depending on the control, wherein the signal source is controlled depending on the information such that it outputs a signal when the information results from an evaluation result according to which at least one seat of the seating arrangement (105) is occupied and / or a selected predetermined seat occupancy state exists, wherein the method further comprises: Detecting a seatbelt fastening state of at least one seat of the seating arrangement (105) or receiving seatbelt information characterizing this seatbelt fastening state; wherein the signal source is controlled in such a way as to depend on the seatbelt information and the information from the evaluation result, such that it outputs a seatbelt fastening indicator signal when, according to the information, at least one 2023PF03001 The seat of the seating arrangement (105) is occupied and / or a selected predetermined seat occupancy state exists and seat belt information indicates that the associated seat belt of the seat is not fastened.

13. System (115) for automatically detecting a seat occupancy state of a seating arrangement (105) with at least one seat (105a-e), wherein the system (115) has a data processing device configured to perform the method according to claim 10 or 11 for detecting the seat occupancy state.

14. Computer program or computer program product comprising instructions which, when executed on the data processing device of the system (115) according to claim 13, cause the system (115) to execute the method according to claim 10 or 11.

15. Vehicle (100) comprising: a seating arrangement (105) with at least one seat (105a-e); a radar sensor (110) for at least section-wise radar scanning of the seating arrangement (105); and a system (115) according to claim 13 for automatically detecting a seat occupancy state of the seating arrangement (105) depending on at least section-wise radar scanning of the seating arrangement (105) performed by the radar sensor (110). 27 / 28

Images