Method for producing an evaluation model for detecting a seat occupancy state of a seat arrangement

Abstract

The invention relates to a method for producing an evaluation model, in particular a machine learning model, for automated detection of a seat occupancy state of a seat arrangement (105) having at least one seat (105a-e), in which method parameters that are assigned to at least one of a plurality of predefined possible seat occupancy states of the seat arrangement (105), as well as hyperparameters for the evaluation model, are provided, wherein the hyperparameters are configured to be adjusted on the basis of a metric. A detection unit is determined, wherein the detection precision indicates a discrepancy between the seat occupancy state assigned to the parameters and an evaluation result provided by the evaluation model with the provided hyperparameters. A metric is provided and evaluated, wherein the metric takes into account a difference between the detection precision and a specified target value in order to output a value for the determined detection precision, wherein the target value is defined as a detection precision at which the metric reaches a maximum. The hyperparameter of the evaluation model is then adjusted accordingly, wherein the metric is optimised by means of an optimisation method in order to obtain an optimum for the output value and to adjust in this way the hyperparameter for which the metric reaches the optimum. The evaluation model is produced with the adjusted hyperparameters for further use for training using training data for a method for automated of a seat occupancy state of a seat arrangement (105) having at least one seat (105a-e).

Description

[0001] Method for creating an evaluation model for detecting the occupancy status of a seating arrangement

[0002] The present invention relates to a method for creating an evaluation model, in particular a machine learning model, for use in a method for detecting a seat occupancy state of a seating arrangement with at least one seat. The invention also relates to a method for training the evaluation model, as well as a method, a computer program, and a system configured to execute the method, each for detecting a seat occupancy state using such a trained evaluation model.

[0003] In various situations, it may be necessary to automatically determine the current seat occupancy status of a seating arrangement with at least one seat. Such a situation can occur particularly in vehicles, for example in motor vehicles, where a vehicle configuration or the activation, deactivation, and / or control of one or more vehicle functions is to take place depending on a current seat occupancy status. For example, it is known in motor vehicles to issue an acoustic or visual indication to the vehicle occupants to fasten seat belts or to control the activation or deactivation of airbags depending on a detected seat occupancy status.

[0004] Methods are known for the automated detection of a current seat occupancy status, in particular the arrangement of vehicle seats in a vehicle, in which the interior of a vehicle is monitored. In particular, methods are known that use radar technology for the automated detection of a current seat occupancy status. By scanning the vehicle interior with radar sensors, measurement data is generated in the form of radar point clouds. Based on the measured radar point cloud, the current seat occupancy can be determined.

[0005] To evaluate the radar point clouds, a corresponding evaluation model is used, which can be a machine learning (ML) model. Known methods using the machine learning model can also detect the type of seat occupancy, for example, whether a seat is occupied by an adult or a child. Methods are also known that are capable of performing an even more differentiated evaluation of the seat occupancy status, particularly with regard to the height and weight of an adult or the age of a child. For example, it may be necessary to distinguish between a six-year-old and a four-year-old in order to take the different heights of the children into account.

[0006] When creating the ML model, the model's hyperparameters are adjusted using optimization methods. A metric can be used for the recognition accuracy of an evaluation result for known parameters. This metric is based on the difference between the correct result (e.g., the actual seat occupancy state) and a result of the evaluation model. For imbalanced datasets, i.e., datasets in which, for example, one class is overrepresented and another underrepresented, the choice of a metric can lead to evaluation results for the overrepresented class, which may already be of very good quality, being further improved, since these metrics, on average, compensate for poorer evaluation results (of the underrepresented class), which, conversely, are not further improved.In the context of a seat belt warning system, it may therefore happen that the already good detection of average adults is continually improved, whereas the possibly inadequate detection of children is not improved.

[0007] It is an object of the present invention to provide an improved solution for the automated detection of a seat occupancy state of a seating arrangement having at least one seat. In particular, an improved solution for generating an evaluation model and for the automated detection of a seat occupancy state of a seating arrangement is to be provided, which offers more balanced detection accuracy across different classes.

[0008] This object is achieved according to the teaching of the independent claims. Various embodiments and further developments of the invention are the subject of the dependent claims.

[0009] A first aspect of the solution presented here relates to a method, in particular a computer-implemented method, for creating an evaluation model for automated detection of a seat occupancy state of a seating arrangement having at least one seat. The method provides parameters associated with at least one of several predefined possible seat occupancy states of the seating arrangement, as well as hyperparameters for the evaluation model, wherein the hyperparameters are configured to be adjusted based on a metric. A detection accuracy is determined, wherein the detection accuracy indicates a distance between the seat occupancy state associated with the parameters and an evaluation result delivered by the evaluation model with the provided hyperparameters.A metric is provided and evaluated, wherein the metric takes into account a difference between the recognition accuracy and a predetermined target value in order to output a value for the determined recognition accuracy, wherein the target value indicates a recognition accuracy at which the metric achieves an optimum. The hyperparameters of the evaluation model are then adjusted accordingly, wherein the metric is optimized using an optimization method. The evaluation model is created with the adjusted hyperparameters for further use for training with training data for a method for automatically determining a seat occupancy state of a seating arrangement with at least one seat.

[0010] Using the method according to the first aspect, an evaluation model, in particular a machine learning model, can be created with which a more balanced recognition accuracy can be achieved for different seat occupancy states. By using a predetermined target value, it is possible, in particular, to slow down any further improvement in recognition accuracy beyond the target value, while further improving recognition accuracies for ranges below the target value. In other words, an improvement beyond the target value is penalized by the metric, whereas known metrics typically work without such a target value in order to optimize recognition accuracy to a maximum.

[0011] The method is particularly advantageous for machine learning of a radar-based system for detecting seat occupancy. In particular, the detection of certain seat occupancy states, which are relatively poorly detected using conventional metrics and optimization methods, can be improved. The detection of certain seat occupancy states, which are relatively better detected using conventional metrics and optimization methods, may be slightly impaired. However, since these are already detected with sufficient reliability and the detection remains above a value that is considered sufficient, this does not result in a disadvantage in practice. Rather, a more consistent detection accuracy that is sufficient for a larger number of different seat occupancy states can be achieved.The term "seat occupancy state" of a seating arrangement with at least one seat, as used herein, is to be understood in particular as information indicating whether or to what extent the seating arrangement or at least one of its seats is occupied by an object, in particular a thing or a person. In a simple example, the seat occupancy state can only indicate the presence or absence of an object, or in a more advanced example, in the case of the presence of at least one object on the seating arrangement or one or more of its seats, it can provide information about the type or other property, such as a spatial extent, of the object.

[0012] The term “evaluation model,” as used herein, refers to a model, particularly a mathematical one, that uses a radar point cloud or one or more parameters characterizing it as input variables in order to provide an evaluation result depending thereon, in this case one of several predefined possible seat occupancy states of the seating arrangement. The evaluation model can, in particular, be a mathematical estimation function, wherein the radar point cloud represents empirical data as a sample and the evaluation result represents an estimated value determined depending thereon. The evaluation model can, in particular, be a “machine learning model” (orA machine learning model can be a "machine learning model" (synonymous with "machine learning model"), which here is understood in particular to mean a mathematical, in particular statistical, model for making predictions or decisions created using at least one machine learning algorithm on the basis of sample data referred to as training data, without the algorithm(s) being explicitly programmed to make such predictions or decisions. In particular, decision tree-based machine learning models (English "decision trees") are machine learning models.

[0013] The term "metric," as used herein, refers in particular to a mathematical mapping that assigns a numerical value to one or more recognition accuracies. The metric is used, in particular, for hyperparameter optimization by feeding its value into an optimization procedure to then adjust the hyperparameters accordingly. In particular, weights in the evaluation model can be adjusted accordingly to achieve a specific behavior of the evaluation model.

[0014] The term "parameter," as used herein, refers in particular to a value relating to the seat occupancy status, as it may also occur during the actual operation of the evaluation model in a method for detecting a seat occupancy status. The parameters can, in particular, indicate whether a seat is occupied or not, how a seat is occupied (e.g., by an adult or a child), and possibly other circumstances, such as the state of the vehicle (e.g., whether it is moving or stationary).

[0015] The term "hyperparameter," as used herein, refers specifically—in contrast to the previously discussed parameters—to a parameter of the evaluation model that is determined during the creation of the evaluation model to guide the model during training. Hyperparameters are thus determined and specified before the actual training of the model.

[0016] The term "radar point cloud," as used herein, refers in particular to a set of points in a vector space obtained by radar scanning of at least one object surface, which has a typically unorganized spatial structure ("cloud"). In the case of a radar point cloud, the points of the radar point cloud can be referred to as "radar points." A (radar) point cloud can be described in particular by the (radar) points it contains. The radar points, in turn, can each be described in particular by their spatial coordinates, which indicate, for each radar point, a location of the reflection of an emitted radar signal on an object surface, measured during the radar scanning. In addition to the radar points, attributes such as measured Doppler velocity or a signal-to-noise ratio (SNR) can also be recorded.

[0017] The terms "comprises," "includes," "includes," "has," "has," "with," or any other variation thereof, as used herein, are intended to cover non-exclusive inclusion. For example, a method or apparatus that includes or has a list of elements is not necessarily limited to those elements, but may include other elements not expressly listed or that are inherent in such a method or apparatus.

[0018] Furthermore, unless expressly stated to the contrary, "or" refers to an inclusive "or" and not an exclusive "or." For example, a condition A or B is satisfied by one of the following conditions: A is true (or present) and B is false (or absent), A is false (or absent) and B is true (or present), and both A and B are true (or present). The terms "a" or "an" as used herein are defined to mean "one or more." The terms "another" and "a further," and any other variations thereof, are to be construed to mean "at least one further."

[0019] The term “plurality” or “several” as used here shall be understood to mean “two or more”.

[0020] The terms “configured” or “set up” to fulfil a specific function (and respective variations thereof), as used herein, are to be understood within the meaning of the invention as meaning that the corresponding device is already in a configuration or setting in which it can carry out the function or is at least adjustable – i.e. configurable – so that it can carry out the function after being set accordingly. The configuration can be carried out, for example, by appropriately setting parameters of a process sequence or of switches or the like for activating or deactivating functionalities or settings. In particular, the device can have a plurality of predetermined configurations or operating modes, so that the configuration can be carried out by selecting one of these configurations or operating modes.

[0021] In the following, various exemplary embodiments of the method are described, each of which, unless expressly excluded or technically impossible, can be combined with each other as well as with the other aspects of the present solution described.

[0022] In some embodiments, the target value is smaller than a maximum recognition accuracy, so that the value of the metric is maximum for the target value. In this way, preference is given to hyperparameters for which recognition accuracy is not maximized at all costs (which, as explained above, can lead to an imbalance between already strong results and weak results). In particular, the recognition accuracy can be determined with a value in the interval [0; 1] and the target value can be set to a value less than 1. The metric reaches a maximum value for values ​​less than 1, e.g. 0.9 or 0.95. Evaluation results that are far from the target value are penalized, i.e. not only low values ​​as usual (i.e. values ​​with a large distance to the "optimal" value 1), but also values ​​greater than the target value.In some embodiments, providing the parameters comprises providing a set of parameters associated with multiple seat occupancy states, and determining the recognition accuracy for each combination of parameters in the set of parameters, wherein the metric is defined as an average value calculated for the set of parameters and the respective recognition accuracy. The average value can be, for example, the geometric mean. A set of parameters can, for example, contain combinations of parameters for which a value is predetermined, e.g., a specific seat of the seating arrangement or whether the engine is running or not. A combination of driver size, seat, and vehicle state can, for example, be: a small woman (AF05) is sitting in the driver's seat while the vehicle is stationary. The recognition accuracy can then be determined (calculated) for all possible combinations of the selected parameters.For each calculated recognition accuracy, the difference to the target value can then be calculated.

[0023] In some embodiments, the metric includes the natural exponential function, which includes as an argument the magnitude of a difference between the recognition accuracy and the target value. A metric may look like this (including normalization to obtain a value between 0 and 1 ("normalized")): raw normalized = where n is the number of combinations of the selected parameters (e.g., n=3: child in a moving vehicle, no child in a moving car, child in a stationary car, no child in a stationary car). Accuracy is the recognition accuracy, and target is the target value.

[0024] In some embodiments, the optimization method is Bayesian optimization. In particular, the above-mentioned normalized value of the metric can be minimized using Bayesian optimization.

[0025] In some embodiments, the optimization process is performed with a predetermined number of iterations. For example, while Bayesian optimization theoretically allows any number of iterations, it may be advantageous to limit the number of iterations and, after this number, adjust the hyperparameters to represent the most promising hyperparameters for the evaluation model.

[0026] In some embodiments, the seating arrangement comprises a plurality of seats, wherein the seat occupancy state is an individual or cumulative seat occupancy state of the seats. In other words, a seat occupancy state can include one or more seats of the vehicle and their respective occupancy (see also the discussion on clustering a radar point cloud below).

[0027] In some embodiments, the seat occupancy state comprises at least one type of seat occupancy for at least one seat of the seating arrangement. In particular, determining the seat occupancy state can comprise determining a type of seat occupancy. The type of seat occupancy can comprise at least one of an unoccupied (i.e., free) seat, a seat occupied by an adult, a seat occupied by a child, a seat occupied by a child seat with a toddler, and a seat occupied by a baby seat with a baby. In addition, it can be provided to detect whether a seat is occupied, for example, by an object or a doll. A person can be simulated, for example, using the established THUMS human model. The values ​​AM95, AM50, and AF05 represent, for example, different adults (tall / heavy man, medium / medium-weight man, or small / light woman).Children can be identified by their age, e.g. 4YO or 6YO for a four-year-old or six-year-old child, respectively.

[0028] By determining not only whether a seat is occupied, but also how, and outputting the corresponding information, more differentiated control of a seat belt warning system or airbag system, for example, can be achieved. For example, certain airbags can only be activated when an adult is occupying a seat, whereas an airbag can (or must) be deactivated if a seat is occupied but not by an adult, for example by a baby seat that is positioned rearward facing. When using a child seat or baby seat, which are not usually secured with a seat belt but using a special attachment (“Isofix”), for example, it is also advantageous if a seat belt warning is not output, thus avoiding unnecessary warning messages that are unhelpful and may be annoying to the user.In some embodiments, the parameters further include a state of the vehicle. The state of the vehicle can include, in particular, whether the vehicle is stationary or moving and / or whether the engine is running or not. Taking the vehicle state into account can have a positive effect on the evaluation model, as it can influence the result of the radar measurement, as different vibrations of the vehicle can affect the result of the radar monitoring of the vehicle interior.

[0029] A second aspect of the solution presented here relates to a method, in particular a computer-implemented method, for training an evaluation model created according to a method according to a first aspect for the automated detection of a seat occupancy state of a seating arrangement with at least one seat. In the method, measurement data are acquired which represent an associated radar point cloud, wherein the radar point cloud was or is obtained based on a radar scan of a spatial area surrounding the seating arrangement at least in part and is associated with one of several predefined possible seat occupancy states of the seating arrangement. Training data is then generated from the measurement data, wherein the training data is provided as input data to the evaluation model in order to obtain an evaluation result as its output which is associated with the seat occupancy state of the seating arrangement.

[0030] A third aspect of the solution presented here relates to a method, in particular a computer-implemented method, for the automated detection of a seat-occupancy state, in particular a seat-related one, of a seating arrangement with at least one seat (or equivalently: seat), in particular a seat in or for a vehicle, such as an automobile (e.g., truck, car, or bus). The method comprises: (i) acquiring, in particular receiving or generating, measurement data representing an associated radar point cloud.Each radar point cloud is or was obtained based on a radar scan of a spatial area surrounding the seating arrangement at least in part; (ii) determining, in particular estimating, a seat occupancy state of the seating arrangement using an evaluation model that, depending on the radar point cloud, provides one of several predefined possible seat occupancy states of the seating arrangement as an evaluation result; and (iii) outputting information defined depending on the evaluation result. The method according to the third aspect uses an evaluation model, in particular a machine learning model, which was trained using the method according to the second aspect.Using the method according to the third aspect, an evaluation result characterizing the seat occupancy status of the seat arrangement (particularly in the sense of a prediction or classification) can be obtained based on a radar point cloud obtained by radar scanning of a spatial area surrounding the seat arrangement. This allows radar-based solutions to be implemented, particularly in the vehicle context (particularly for automobiles), that can reliably detect a seat occupancy status (particularly exclusively) based on radar and, based on this, can activate, deactivate, or control / regulate certain functionalities or systems, such as a seat belt warning system or an airbag system, entirely or selectively.

[0031] The information to be output can, in particular, represent the evaluation result itself. It can also be a detectable signal, particularly one that can be perceived by the human senses, such as a warning, a control signal for controlling a signal source, or a data signal carrying the information.

[0032] Various exemplary embodiments of the method according to the third aspect are described below, which can each be combined with each other as well as with the other aspects of the present solution described, unless this is expressly excluded or is technically impossible.

[0033] In some embodiments, the seating arrangement has a plurality of seats for which a seat occupancy state is to be determined individually or cumulatively within the scope of the method. For each (individual) radar point cloud, the set of its radar points is divided into several clusters, each containing a subset of the radar points, by means of cluster formation depending on the respective spatial position of the radar points in relation to the seats, in order to individually assign to each of the seats a cluster that is spatially closest to it. The seat occupancy state of the seating arrangement is determined for each of the seats depending on the radar point cloud determined for the respectively associated cluster in order to obtain an evaluation result, in particular a classification result, that characterizes a seat occupancy state of the respective seat.The information to be output is then defined depending on the respective individual evaluation results for the various seats. In some associated embodiments, each radar point cloud is segmented into several clusters by assigning a subset of the radar points of the respective radar point cloud to each of the seats as a cluster depending on their respective position such that the radar points of the cluster are located in a defined, closed spatial area, in particular a cuboid, in the vicinity of the seat. This enables particularly simple and low-computational-intensive cluster formation and thus seat-related seat occupancy detection, wherein the location (position and orientation) and shape of the spatial area is or can be defined such that it strongly overlaps the spatial area usually occupied by a typical object to be detected, in particular a person, on a seat of the seating arrangement.

[0034] This enables seat-specific, i.e., individual, seat occupancy detection for each seat in the case of a multi-seat seating arrangement. This is particularly advantageous or even necessary if a seat-specific response is to be made to the detected seat occupancy, for example by activating or deactivating, or otherwise controlling, a specific functionality or system for a specific seat depending on its detected occupancy, such as a seat-specific airbag system, a seat-specific seat belt warning, or a seat-specific seat heating system. In some embodiments, cluster formation can be effected in particular such that the clusters are disjoint, so that no radar point is assigned to two different clusters.

[0035] In some embodiments, the or each individual radar point cloud is segmented into a plurality of clusters by assigning a subset of the radar points to each of the seats as a cluster depending on their respective position, in particular uniquely for each radar point, such that the radar points of the cluster are located in a defined, closed, in particular cuboid-shaped, spatial area in the vicinity of the seat. The assignment can in particular be carried out such that each radar point is assigned to the cluster of the seat closest to it. In this way, the radar point cloud can be divided into clusters, i.e. subsets of the radar point cloud localized in the vicinity of the respective seats, so that seat-specific seat occupancy states can be determined in a targeted manner and therefore with a high degree of reliability on the basis of the cluster assigned to the respective seat.

[0036] In some embodiments, outputting the information comprises controlling a signal source as a function of the information in order to cause the signal source to output a defined signal as a function of the control. The signal source can be, in particular, an audio source, an optical signal source, in particular a display device for images or text, and / or a haptic actuator, or a combination of at least two of the aforementioned signal sources. Thus, based on the signaling, the detected seat occupancy status can be communicated to a user or used to control another technical system, such as an airbag system.

[0037] In some of these embodiments, the signal source is controlled in dependence on the information such that it outputs a signal, in particular defined by the control, when the information results from an evaluation result, according to which at least one seat of the seat arrangement is occupied and / or a selected predetermined seat occupancy state exists.

[0038] In some embodiments, the method further comprises: (i) detecting a seat belt fastening state of at least one seat of the seat assembly or receiving seat belt information characterizing this seat belt fastening state; (ii) wherein the signal source is controlled as a function of the seat belt information and the information from the evaluation result such that it outputs a seat belt fastening indication signal if, according to the information, at least one seat of the seat assembly is occupied and / or a selected predetermined seat occupancy state exists and seat belt information indicates that the associated seat belt of the seat is not fastened. This makes it possible to achieve a radar-based seat belt fastening checking and warning system, particularly with regard to detection exclusively.

[0039] In some embodiments, the individual radar points of the radar point cloud are each represented by a position of the respective radar point in three-dimensional space and by at least one of the following parameters: (i) a Doppler shift value of the radar signal at the respective radar point; (iii) a signal-to-noise ratio value of the radar signal at the respective radar point. These parameters can be used, in particular, for pre-filtering the radar point cloud as part of a pre-processing step prior to feature determination.

[0040] A fourth aspect of the present solution relates to a system, in particular a data processing device, for the automated detection of a, in particular respective, seat occupancy state of a seating arrangement with at least one seat, in particular with at least one vehicle seat in or for a vehicle. The system comprises a data processing device configured, in particular by means of a corresponding computer program, to execute the method according to the third aspect for detecting the seat occupancy state.

[0041] A fifth aspect of the present solution relates to a computer program or computer program product comprising instructions which, when executed on the data processing device of the system according to the fourth aspect, cause the system to carry out the method according to the third aspect.

[0042] The computer program can in particular be stored on a non-volatile data carrier. This is preferably a data carrier in the form of an optical data carrier or a flash memory module. This can be advantageous if the computer program as such is to be handled independently of a processor platform on which the one or more programs are to be executed. In another implementation, the computer program can be present as a file on a data processing unit, in particular on a server, and can be downloaded via a data connection, for example the Internet or a dedicated data connection, such as a proprietary or local network. In addition, the computer program can have a plurality of interacting individual program modules. The modules can in particular be configured or at least be usable in such a way that they can be used in the sense of distributed computing (DC).“Distributed computing” is carried out on different devices (computers or processor units) that are geographically separated from each other and connected via a data network.

[0043] The system according to the fourth aspect can accordingly comprise a program memory in which the computer program is stored. Alternatively, the system can also be configured to access an external computer program, for example, available on one or more servers or other data processing units, via a communication connection, in particular to exchange data with it that is used during the execution of the method or computer program or that represents outputs of the computer program.

[0044] A sixth aspect of the present solution relates to a vehicle comprising: (i) a seating arrangement with at least one seat; (ii) a radar sensor for at least partially radar scanning the seating arrangement; and (iii) a system according to the fourth aspect for automatically detecting a, in particular a respective, seat occupancy state of the seating arrangement as a function of an at least partially radar scan of the seating arrangement carried out by the radar sensor, in particular according to a method according to the third aspect.

[0045] The features and advantages explained with regard to one aspect of this solution also apply accordingly to the other aspects of the solution.

[0046] Further advantages, features and possible applications of this solution will become apparent from the following detailed description in conjunction with the drawings.

[0047] It shows:

[0048] Fig. 1 schematically shows an exemplary embodiment of a vehicle equipped with a system for automatically detecting a seat occupancy state of a seat arrangement in the vehicle;

[0049] Fig. 2 shows schematically the vehicle from Fig. 1 , with the passenger seat occupied;

[0050] Fig. 3A is an exemplary two-dimensional representation of a radar point cloud recorded by a radar sensor of the vehicle from Fig. 2;

[0051] Fig. 3B is an exemplary representation of a clustering of the radar point cloud from Fig. 3A according to the positions of the individual seats of the seating arrangement;

[0052] Fig. 4 is a flowchart illustrating an exemplary embodiment of a method for automatically detecting a seat occupancy state of a seat arrangement; and

[0053] Fig. 5 is a flowchart illustrating an exemplary embodiment of a method for creating an evaluation model for the automated detection of a seat occupancy state of a seat arrangement.

[0054] In the figures, like reference numerals designate like, similar, or corresponding elements. Elements shown in the figures are not necessarily drawn to scale. Rather, the various elements shown in the figures are depicted in such a way that their function and general purpose will be understood by those skilled in the art. Connections and couplings between functional units and elements shown in the figures can also be implemented as indirect connections or couplings, unless expressly stated otherwise. Functional units can, in particular, be implemented as hardware, software, or a combination of hardware and software.

[0055] First, the detection of seat occupancy in a vehicle using a radar system will be described with reference to Figs. 1 to 4, since the method according to the present invention can be advantageously applied in this context. The present invention will then be explained in particular with reference to Fig. 5.

[0056] The exemplary embodiment of a vehicle 100 schematically illustrated in Fig. 1 has a seating arrangement 105 with five individual seats or seating locations 105a to 105e. Each of the seats 105a to 105e is suitable for accommodating a person as a passenger of the vehicle 100. The vehicle 100 further has a radar sensor 110, which is mounted on the ceiling within the vehicle cabin and is configured to scan the seating arrangement 105, at least substantially, using radar beams. Accordingly, the seats 105a to 105e, in particular their seating surfaces, are each at least predominantly located within an observation field 110a that can be scanned by the radar sensor 110.In addition, the vehicle 100 has a system 115 for automatically detecting a seat occupancy state of the seat arrangement 105 as a function of a radar scan of the seat arrangement 105 carried out by the radar sensor 110, at least in sections with respect to the observation field 110a.

[0057] The system 115 comprises, in particular, a data processing unit 115a with at least one microprocessor and a memory 115b connected to the microprocessor for signal transmission, in which a computer program configured to carry out the method for automatically detecting a seat occupancy state of the seat arrangement 105, described below with reference to Fig. 4, is stored. Furthermore, the sensor data generated by the radar sensor 110 during radar scanning or information already obtained therefrom through further processing can be or will be stored in the memory 115b.

[0058] The vehicle 100 shown in Fig. 2 corresponds to the vehicle from Fig. 1 , but here the passenger seat 105b is occupied by a person P. In the further discussion of Figs. 3A and 3B, reference is made to the constellation from Fig. 2. Reference is now made below to Figs. 3A and 3B, each of which represents a radar point cloud, wherein, for the purpose of representability, the respective, essentially three-dimensional radar point cloud has been reduced to two dimensions by projecting the positions of the radar points of the radar point cloud onto a plane spanned by two of its dimensions.

[0059] Fig. 3A illustrates an exemplary radar point cloud 305, as acquired as a result of a radar scan of the seating arrangement 105 by the radar sensor 110 during a specified time interval (measurement period). The position of the individual radar points within the radar point cloud 305 can be represented by spatial coordinates; for example, Cartesian coordinates X and Y can be assigned to the drawing plane and, accordingly, to each individual point. In reality, if the dimensionality reduction due to the drawing is ignored, a third coordinate Z is also added for the third spatial dimension.

[0060] If, during radar scanning, not only the spatial positions of the points at which the radar beam is reflected by the scanned objects are recorded as coordinates, but also a respective Doppler shift is measured, then the individual radar points can be classified depending on the magnitude of this Doppler shift, in particular divided into two different classes. The latter can be achieved, for example, by comparing the Doppler shift with a predefined shift threshold corresponding to a specific shift velocity. Depending on the result of the comparison, those radar points 310 which, according to the value of their assigned Doppler shift, have no velocity or a velocity of the object surface at the reflection point that lies below the displacement wave can be classified as "static" radar points (in Fig.3A and 3B, each represented by a filled black circle. Conversely, those radar points 315 that exhibit a Doppler shift above the shift threshold can be classified as "dynamic" radar points 315 (represented by a black ring in Figs. 3A and 3B, each represented by a black ring).

[0061] The classification of radar points 310 and 315 according to their Doppler shift is not mandatory, but it can be used to process radar point cloud 305, particularly as part of a preprocessing step prior to its evaluation, particularly to filter it depending on the classification. For example, such filtering could be performed in such a way that only dynamic radar points 315 are considered for evaluation, for example, to detect only moving objects.

[0062] Fig. 3B shows the same radar point cloud 305 as in Fig. 3A. In addition, however, cuboid-shaped (3D case) or rectangular in the present 2D representation, selected spatial regions 325a to 325e are shown here, which are spatially assigned to the respective location of the individual seats 105a to 105e. The definition of these spatial regions 325a to 325e can now be used to cluster the radar point cloud 305, whereby each radar point 310 or 315 is assigned, as far as possible, to the spatial region 325a to 325e in which it is located. All radar points not located in one of the spatial regions 325a to 325e can be disregarded below. Alternatively or additionally, points located outside can also be assigned to the nearest spatial region, provided they do not exceed a previously defined distance from the spatial region center.In particular, it can be seen that the areas 320 with a particularly high radar point density are located in the area of ​​the passenger seat 105b, on which the person P is located according to Fig. 2.

[0063] Fig. 4 shows a flowchart illustrating an exemplary embodiment 400 of a method for automatically detecting a seat occupancy state of a seat arrangement. The method can be embodied, in particular, as a computer-implemented method. For this purpose, it can be stored, in particular, in the memory 115b of the system 115 as a computer program and be executable on the data processing unit 115a.

[0064] In the method 400, a radar point cloud 305 is acquired by receiving radar measurement data, in the present example from the radar sensor 110 of the vehicle 100, in a step 410 and further processing them to form one or more radar point clouds.

[0065] The now existing radar point cloud 305 can then be clustered in a further process 420 by checking for each of its radar points whether it lies within one of the defined spatial areas 325a to 325e (see Fig. 3B) and, if so, in which one. Thus, each of the points can be assigned either to one of the spatial areas 325a to 325e or to the other observation field. All radar points that lie within the same spatial area 325a to 325e are combined into a respective cluster. As a result, each of the seats 105a to 105e is assigned a corresponding cluster of the radar point cloud 305. This forms the basis for an individual evaluation for each seat 105a to 105e as to whether the respective seat 105a to 105e is or was occupied or not while the radar point cloud 305 was being formed.

[0066] To facilitate the subsequent evaluation of the clustered radar point cloud 305, a corresponding characteristic K or a plurality of characteristic K with preferably different properties can be determined for each of the clusters in a process 430, wherein this characteristic K can be defined in particular as the number of radar points in the cluster. If no filtering according to the Doppler shift value has taken place, this can involve a joint count of both the static and the dynamic radar points 310 and 315, respectively. However, if the static radar points 310 were previously filtered out, only the dynamic radar points 315 are counted.

[0067] The evaluation of the parameter K for the cluster for seat 105b can now take place (the same can be done analogously for the respective clusters for the other seats). For this purpose, in process 440, the parameter K (or a temporal progression of the parameter K) is made available as an input to an evaluation model. This can in particular be a model based on machine learning, such as an artificial neural network or a decision tree-based model (decision tree(s)). The training and, if applicable, validation data used for the preceding training can be structured such that they contain, according to the type of parameter K, a plurality of different radar point clouds or clusters thereof, as well as an assigned correct class of a classification of possible seat occupancy states for each progression of K. The model can thus be trained and validated in the sense of supervised learning.In the simplest case, seat occupancy states indicate whether the seat is occupied or not. However, more sophisticated classifications are also conceivable, in which, in the case of the presence of an object, the respective class additionally indicates the type of object, for example, whether it is moving or stationary, and in the case of a moving object, in particular, whether it is a person (generally recognizable in particular based on a breathing pattern in the course of the characteristic K).

[0068] If, in process 440, a seat occupancy state for the seat arrangement 105 was determined based on the evaluation model, in particular for one or more of its seats 105a to 105e individually, this result can be output as corresponding information in process 445, for example at a user interface of the vehicle or in the form of data for further processing by one or more other systems, in particular systems of the vehicle.

[0069] In the present example, this information is to be used in particular to check whether or not to issue a belt warning signal depending on the seat occupancy status of a respective seat 105a to 105e and the result of a check as to whether or not a corresponding seat belt has been fastened for this seat.

[0070] For this purpose, in process 450, it can be checked whether the seat belt for the relevant seat (here, for example, for seat 105b) is fastened, and in step 455, a functionality of the vehicle 100 can be controlled depending on the information on the seat occupancy state output in process 445 and the status of the seat belt determined in process 450. In particular, this can be done in such a way that in process 455, a signal source is activated to output a belt status signal, in particular a visual and / or acoustic one, in order to signal to one or more other occupants of the vehicle, if necessary, that a seat is occupied but the seat belt is not fastened there. The method then branches back to step 410 to start another loop run.

[0071] The evaluation model is created before the actual application in method 400 and before training. In particular, the hyperparameters are optimized accordingly, as explained below. A flowchart illustrating an exemplary embodiment of a method 500 for creating the evaluation model is shown in Fig. 5.

[0072] In step 510, a data set with various seat occupancy states (parameters) is first provided. This data set is processed in the evaluation model. In particular, this serves to optimize the hyperparameters before the created evaluation model is finally trained with the training data (step 560). For hyperparameter optimization, a metric is applied, which is optimized using an optimization method 501 (here, Bayesian optimization).

[0073] In particular, the dataset contains various seat occupancy states with people, objects, and empty seats. For each seat, a decision must be made as to whether a seatbelt reminder (i.e., a seatbelt warning signal) should be issued, as explained above. The decision is based on the presence of a person and their size (for children, age is used because it is strongly correlated with size). The dataset is highly imbalanced in terms of age and size of the passengers. Therefore, the different types of passengers (or seat occupancy states) are weighted to achieve an optimal overall classification without an underrepresented group of passengers (e.g., AM95, AF05) having poor recognition accuracy (while recognition accuracy for frequently occurring groups, e.g., medium-sized adults (such as AM50), is likely to be relatively good anyway).These weights are optimized using Bayesian optimization to obtain a more balanced overall result.

[0074] A key component is the selection of the metric. The metric described below is therefore specifically designed to penalize overconfidence by setting an upper limit for the metric, i.e., a target value for recognition accuracy (or, for short, accuracy). Thus, the optimization of the metric does not aim to achieve 100% recognition accuracy (value 1), but rather, for example, 95% (value 0.95), in order to obtain the desired more balanced overall result. The target value for accuracy is set for each possible combination, as described below.

[0075] First, a set of parameters is selected, i.e., various seat occupancy states for which equilibrium is to be established, e.g., whether the engine is running or not, or the occupancy of a specific seat 105a to 105e. For example, a combination would be: AF05 (small woman) in the driver's seat while the vehicle is stationary. This is a combination of the person's size, the seat, and the vehicle's condition.

[0076] The accuracy is calculated for all possible combinations of the selected parameters. For each of these calculated accuracies, the absolute value of the difference between the specified target value and the calculated accuracy for each combination of parameters is calculated. These absolute values ​​are then used as exponents in the natural exponential function. This ensures that classes that are further away from the desired target are penalized more severely than classes that are already close. The geometric mean of each of these values ​​is then calculated. This ultimately maximizes all values, not just one of them. Finally, the obtained value is normalized so that it lies in the range 0 to 1. Smaller values ​​are better, so we can use this as direct input for Bayesian optimization, which then attempts to minimize this value.

[0077] The metric can therefore be implemented as the following function: where n is the number of combinations of the selected parameters (e.g., n=3: child in a moving vehicle, no child in a moving car, child in a stationary car, no child in a stationary car). The normalized value "normalized" therefore takes on values ​​between 0 and 1 (i.e., in the interval [0; 1]). The recognition accuracy is denoted by "accuracy," and the target value is denoted by "target."

[0078] For Bayesian optimization 501, a fixed number of iterations (m) is specified before starting. Bayesian optimization optimizes the metric described above to tune the hyperparameters (“hyperparameter optimization”). Bayesian optimization 501 treats the above-mentioned function, which defines the metric, as a random function that it attempts to estimate. The number of iterations is fixed because, due to the implementation, there is theoretically no limit to this in Bayesian optimization. The hyperparameters are provided, with a range of input values ​​specified for each hyperparameter, and Bayesian optimization 501 selects random values ​​for the first few iterations (here: 10) (step 520).After 10 iterations, during which the evaluation model is trained with the parameters from the dataset and the metric is evaluated (step 530), the Bayesian optimization 501 selects the most promising hyperparameters (step 540).

[0079] In the next (m-1) iterations, during which the evaluation model is further trained or retrained with the parameters from the data set, and the metric is evaluated accordingly (step 550), the most promising hyperparameters are always selected. In particular, the next selected hyperparameters are those from the range of input values ​​that offer the greatest potential for minimizing the metric. After the selected number of iterations has been run through (here a total of m), the result of the process (i.e., the Bayesian optimization 501) is the set of hyperparameters that was evaluated and led to the lowest measured value. As already mentioned above, in a step 560 the evaluation model is finally trained with the selected hyperparameters and saved. It can be seen that a better balance for the different groups can be achieved in the evaluation result.While at least one exemplary embodiment has been described above, it should be appreciated that a wide variety of variations exist. It should also be understood that the described exemplary embodiments are merely non-limiting examples and are not intended to limit the scope, applicability, or configuration of the devices and methods described herein. Rather, the foregoing description will provide one skilled in the art with guidance for implementing at least one exemplary embodiment, it being understood that various changes in the operation and arrangement of the elements described in an exemplary embodiment may be made without departing from the subject matter as defined in the appended claims, as well as their legal equivalents.

[0080] LIST OF REFERENCE SYMBOLS

[0081] P Person in the passenger seat

[0082] 100 vehicles

[0083] 105 Seating arrangement

[0084] 105a-e Seats or seating positions

[0085] 110 radar sensor

[0086] 110a Observation field of radar sensor 110

[0087] 115 System for automated detection of a seat occupancy status

[0088] 115a Data processing unit

[0089] 115b memory

[0090] 305 radar point cloud

[0091] 310 static radar points

[0092] 315 dynamic radar points

[0093] 320 areas of the radar point cloud 305 with high radar point density

[0094] 325a-e Spatial areas for cluster definition

[0095] 400 Methods for the automated detection of a seat occupancy status

[0096] 410-455 individual processes or procedural steps within the framework of procedure 400

[0097] 500 procedures for creating an evaluation model

[0098] 501 optimization methods

[0099] 510-560 individual processes or procedural steps within the framework of procedure 500

Claims

CLAIMS Method for creating an evaluation model for automated detection of a seat occupancy state of a seating arrangement (105) with at least one seat (105a-e), the method comprising: Providing parameters associated with at least one of several predefined possible seat occupancy states of the seat arrangement (105); and Providing hyperparameters for the evaluation model, wherein the hyperparameters are configured to be set based on a metric; Determining a recognition accuracy, wherein the recognition accuracy indicates a distance between the seat occupancy state associated with the parameters and an evaluation result provided by the evaluation model with the provided hyperparameters; Providing a metric and evaluating the metric, wherein the metric takes into account a difference between the recognition accuracy and a predetermined target value in order to output a value for the determined recognition accuracy, wherein the target value indicates a recognition accuracy at which the metric reaches an optimum; Setting the hyperparameters, whereby the metric is optimized using an optimization procedure; and Creating the evaluation model with the adjusted hyperparameters for training with training data for a method for automating a seat occupancy state of a seating arrangement (105) with at least one seat (105a-e). The method according to claim 1, wherein the target value is less than a maximum recognition accuracy, so that the value of the metric for the target value is maximum. The method according to claim 1 or 2, wherein providing the parameters comprises providing a set of parameters associated with multiple seat occupancy states, and the recognition accuracy is determined for each combination of parameters of the set of parameters, wherein the metric is defined as an average value formed for the set of parameters and the respective recognition accuracy. Method according to one of the preceding claims, wherein the metric contains the natural exponential function, which comprises as an argument the magnitude of a difference between the recognition accuracy and the target value. Method according to one of the preceding claims, wherein the The optimization method is a Bayesian optimization. Method according to one of the preceding claims, wherein the The optimization method is performed with a predetermined number of iterations. The method according to any one of the preceding claims, wherein the seating arrangement (105) has a plurality of seats (105a-e), the seat occupancy state being an individual or cumulative seat occupancy state of the seats (105a-e). The method according to any one of the preceding claims, wherein the seat occupancy state comprises at least one type of seat occupancy for at least one seat of the seating arrangement. The method according to any one of the preceding claims, wherein the parameters further comprise a state of the vehicle.A method for training an evaluation model created according to a method according to one of the preceding claims for automated recognition of a seat occupancy state of a seating arrangement (105) having at least one seat (105a-e), the method comprising: acquiring measurement data representing an associated radar point cloud (305), the radar point cloud (305) being obtained or being obtained on the basis of a radar scan of a spatial region surrounding the seating arrangement (105) at least in sections and being associated with one of a plurality of predefined possible seat occupancy states of the seating arrangement (105); and. Generating training data from the measurement data, wherein the training data are made available to the evaluation model as input data in order to obtain as its output an evaluation result which is associated with the seat occupancy state of the seat arrangement (105).

11. A method for the automated detection of a seat occupancy state of a seating arrangement (105) with at least one seat (105a-e), the method comprising: Acquiring measurement data representing an associated radar point cloud (305), wherein each radar point cloud (305) was or is obtained on the basis of a radar scan of a spatial region surrounding the seat arrangement (105) at least in sections; Determining a seat occupancy state of the seat arrangement (105) using an evaluation model that has been trained according to a method according to claim 10 and, depending on the radar point cloud, provides one of several predefined possible seat occupancy states of the seat arrangement (105) as an evaluation result; and Output of information defined depending on the evaluation result.

12. The method according to claim 1 1, wherein the outputting of the information comprises controlling a signal source in dependence on the information in order to cause the signal source to output a defined signal in dependence on the control, wherein the signal source is controlled in dependence on the information in such a way that it outputs a signal when the information results from an evaluation result, according to which at least one seat of the seat arrangement (105) is occupied and / or a selected predetermined seat occupancy state exists.

13. The method of claim 12, further comprising: Detecting a seat belt fastening state of at least one seat of the seat arrangement (105) or receiving seat belt information characterizing this seat belt fastening state; wherein the signal source is controlled as a function of the seat belt information and the information from the evaluation result such that it outputs a seat belt fastening indication signal if, according to the information, at least one seat of the seat arrangement (105) is occupied and / or a selected predetermined seat occupancy state exists and seat belt information indicates that the associated seat belt of the seat is not fastened.

14. System (115) for the automated detection of a seat occupancy state of a seat arrangement (105) having at least one seat (105a-e), wherein the system (115) comprises a data processing device configured to Detecting the seat occupancy state, to carry out the method according to one of claims 11 to 13. Computer program or computer program product, comprising instructions which, when executed on the data processing device of the system (115) according to claim 14, cause the system (115) to Method according to one of claims 11 to 13. Vehicle (100), comprising: a seat arrangement (105) with at least one seat (105a-e); a radar sensor (110) for at least partially radar scanning the seat arrangement (105); and a system (115) according to claim 14 for automatically detecting a seat occupancy state of the seat arrangement (105) as a function of an at least partially radar scan of the seat arrangement (105) performed by the radar sensor (110).

Images