Method and system for detecting seat occupancy in a seating arrangement based on radar point clouds

Abstract

A method for automatic detection of a seat occupancy state of a seat arrangement having at least one seat, in particular for each seat, comprises receiving or generating measurement data which in each case represents one assigned radar point cloud for each measurement frame of a sequence of a plurality of temporally consecutive measurement frames, whereby the measurement data represents a sequence of radar point clouds corresponding to the sequence of measurement frames, each radar point cloud of the sequence having been obtained or obtained based on a radar scan of a spatial region surrounding at least a partial section of the seat arrangement, performed at a measurement time or during a measurement period assigned to the respective measurement frame; accumulating a plurality of radar point clouds of the sequence to obtain an accumulated radar point cloud comprising radar points from each of the individual radar point clouds combined as part of the accumulation process; detecting a seat occupancy state of the seat arrangement based on an evaluation model which returns as an evaluation result one of a plurality of predetermined possible seat occupancy states of the seat arrangement in response to the accumulated radar clouds; and outputting information determined based on the evaluation result.

Description

[Technical field]

[0001] The invention relates to a method in each case for automatically detecting the seat occupancy of a seating arrangement with at least one seat, a computer program and a system arranged to implement said method. Summary of the Invention

[0002] In various situations, it may be necessary to automatically detect the current seat occupancy state of a seating arrangement having at least one seat. Such situations may arise in particular in vehicles, e.g., motorized vehicles, in which the configuration of the vehicle or the activation, deactivation, and / or control of one or more vehicle functions is performed in response to the current seat occupancy state. For example, in motorized vehicles, it is known to output an acoustic or visual warning to vehicle occupants to fasten their seat belts or to control the activation or deactivation of airbags in response to the detected seat occupancy state.

[0003] For the automatic detection of the current seat occupancy state of one or more seats, in particular an arrangement of vehicle seats in a vehicle, so-called seat occupancy mats are known for this purpose, which are integrated into the seats (usually one per seat) and use pressure-sensitive sensors to detect whether the respective seat is occupied or not. Depending on the sensor signals or sensor data of these sensors, the seat occupancy state of the seats is detected, usually by means of a threshold comparison.

[0004] These known solutions therefore require the seat to be equipped with an in-seat sensor and typically are unable to distinguish between various seat occupancy states other than "occupied" and "unoccupied." Furthermore, the sensors typically must be installed into the seat at the factory, making retrofitting difficult or impossible and precluding the ability to detect seat occupancy states for seats not equipped in this manner.

[0005] The object of the present invention is to provide an improved solution for automatic seat occupancy detection of a seating arrangement having at least one seat.

[0006] That object is achieved by the teaching of the independent claims. Various embodiments and developments of the invention are the subject of the dependent claims.

[0007] A first aspect of the solution presented herein relates to a method, in particular implemented by a computer, for automatic seat occupancy detection, in particular on an individual seat basis, of a seat arrangement having at least one seat position (or equivalent: seat), in particular a seat in or for a vehicle such as a motor vehicle (e.g. a lorry, car or bus), comprising: (i) receiving or generating measurement data representative of an assigned radar point cloud for each measurement frame of a sequence of a plurality of temporally consecutive measurement frames, whereby the measurement data represents a sequence of radar point clouds corresponding to the sequence of measurement frames. Each radar point cloud of the sequence has been obtained or is obtainable based on a radar scan of a spatial region surrounding at least a section of the seating arrangement, performed at a measurement time or during a measurement period assigned to the respective measurement frame; (ii) accumulating a plurality of radar point clouds of the sequence to obtain an accumulated radar point cloud comprising radar points, in particular all radar points, from each of the individual radar point clouds combined as part of the accumulation process; (iii) detecting, in accordance with the accumulated radar point clouds, a seat occupancy state of the seating arrangement, based on an evaluation model which returns as an evaluation result one of a plurality of predetermined possible seat occupancy states of the seating arrangement; and (iv) outputting a portion of information determined based on the evaluation result.

[0008] The term "seat occupancy state" of a seating arrangement having at least one seat, when used herein, is understood to mean information indicating in particular whether or to what extent the seating arrangement or at least one of its seats is occupied or occupied by an object, in particular by a person or an object. In a simple example, the seat occupancy state may only indicate the presence or absence of an object, but in a further developed example, in case of the presence of at least one object in the seating arrangement or in one or more of its seats, a statement may be made regarding the nature or another characteristic of the object, for example regarding its spatial extent.

[0009] The term "radar point cloud" as used herein is understood to mean in particular a collection of points in a vector space obtained by a radar scan of at least one object surface, which typically has an unorganized spatial structure ("cloud"). In the case of a radar point cloud, the points of the radar point cloud can be called "radar points". A (radar) point cloud can be described in particular by the (radar) points contained therein. The radar points can then each be described in particular by their spatial coordinates, each of these radar points identifying the position of the reflection of the emitted radar signal on the object surface measured in the radar scan. In addition to the radar points, attributes such as the measured Doppler velocity or the signal-to-noise ratio (SNR) can be obtained.

[0010] The term "accumulated radar point cloud", as used herein, is understood to mean a radar point cloud resulting from the accumulation of a plurality of individual radar point clouds, particularly in the mathematical sense of a union, whereby the accumulated radar point cloud includes radar points, particularly all radar points, from each of the individual radar point clouds that have been combined as part of the accumulation.

[0011] In particular, an accumulated radar point cloud may include more radar points than each of the individual radar point clouds included in the accumulation, and thus the term "accumulating" should be understood to mean such grouping of multiple individual radar point clouds to form an accumulated radar point cloud.

[0012] The term "measurement frame" as used here is understood to mean in particular a measurement time or measurement period in a defined, in particular periodically, chronological sequence of successive measurement points or measurement periods in or during which a respective measurement is or has been performed, in this example a radar scan of at least one object surface, to which each measurement frame is assigned the result of its associated measurement, in this case a radar point cloud generated for the measurement frame by the radar scan.

[0013] The term "evaluation model", when used herein, is understood to mean in particular a mathematical model, in which in particular the accumulated radar point cloud or one or more parameters characteristic thereof are used as input variables to return an evaluation result accordingly, in this case one of a plurality of predefined possible seat occupancy states of the seat arrangement. In particular, the evaluation model may be a mathematical estimation function, in which the accumulated radar point cloud represents empirical data as a random sample and the evaluation result represents an estimation to be detected accordingly. The evaluation model may in particular be a "machine learning model", which is understood here to mean a mathematical, in particular statistical, model for making predictions or detections by at least one machine learning algorithm, in particular based on example data, known as training data, without the algorithm being explicitly programmed to make such predictions or detections. In particular, a decision tree-based model for machine learning is a machine learning model.

[0014] In particular, the output information may represent the evaluation result itself, but it may also be a signal, in particular a signal detectable by human senses such as an alarm, a control signal for activating a signal source, or a data signal carrying information.

[0015] As used herein, the terms "comprise," "contain," "include," "have," "having," or other variations are intended to cover a non-exclusive inclusion. By way of example, a method or apparatus that includes or has a list of elements is not therefore necessarily limited to those elements, but can include other elements not expressly included in the list or inherent to such method or such apparatus.

[0016] Further, unless expressly stated otherwise, "or" means an inclusive or or not an exclusive or. For example, condition A or B is satisfied by one of the following conditions: A is true (or exists) and B is false (or does not exist), A is false (or does not exist) and B is true (or exists), and both A and B are true (or exist).

[0017] The terms "a" or "an," as used herein, are defined as "one or more." The terms "another" and "further" and other variations thereof should be understood to mean "at least one other."

[0018] The term "plurality" as used herein should be understood to mean "two or more."

[0019] The terms "configured" or "designed" to perform a particular function (and any variations thereof) as may be used herein are understood for the purposes of the present invention to mean that the corresponding apparatus already exists in a configuration or setting in which it can perform that function, or that the apparatus is at least adjustable, i.e. configurable, so that it can perform that function after appropriate adjustments. The configuration can be applied, for example, by appropriate setting of process sequence parameters, or by appropriate setting of switches or the like for activating or deactivating the function or setting. In particular, the apparatus may be provided with a number of predefined configurations or modes of operation, whereby the configuration can be performed by selection of one of these configurations or modes of operation.

[0020] With the method according to the first aspect, an assessment result characterizing (in particular in the sense of prediction or classification) the seat occupancy state of a seat arrangement can be obtained based on a radar point cloud collected from a time sequence of real or simulated radar scans of a spatial region surrounding the seat arrangement. In particular in the vehicle context (in particular for automobiles), radar-based solutions can be implemented in which the seat occupancy state can be reliably detected (in particular exclusively) by radar and on this basis certain functions or systems, such as seat belt warning systems or airbag systems, can be totally or selectively activated, deactivated or controlled.

[0021] In many radar measurement techniques, the number of radar points generated per radar point cloud depends on whether or in what way the object is moving. The use of at least one accumulated radar point cloud, compressed by accumulation of several radar point clouds successive in time, serves in particular to increase the number of radar points on the basis of which an evaluation for detecting the seat occupancy is performed. The quality of the evaluation can be increased, in particular the error rate can be reduced. This is particularly important when the individual radar point clouds obtained for the individual measurement frames have in each case only a small number of radar points (known as "sparse point clouds"), whereby without accumulation the evaluation may be inaccurate or prone to errors due to an insufficient database. This may occur in particular in the case of stationary or slightly moving objects, for example when there is a stationary object, a person barely moving or a very small object on the seat.

[0022] In the following, various exemplary embodiments of the method are described, each of which may be combined as desired with one another and with other aspects of the solution, unless expressly excluded or technically impossible.

[0023] In some embodiments, the accumulation of a plurality of radar point clouds of a sequence is performed a plurality of times to obtain an accumulated radar point cloud, which includes radar points, in particular all radar points, from each of the individual radar point clouds combined as part of the respective accumulation, and during each accumulation only the radar point clouds of an individual subset of the sequence assigned thereto are accumulated, whereby at least two subsets are different. The seat occupancy state of the seat arrangement is detected as a function of at least two accumulated radar point clouds. In this way, in particular the time curve of the radar scan can be used to detect the seat occupancy state of the seat arrangement. Based on such a time curve, in particular the movement or a certain movement pattern of one or more objects on the seat arrangement, and thus the presence of the objects or the objects themselves, can be detected with higher reliability than is usually possible based on one accumulated radar point cloud. This further improves the quality and reliability of the automatic detection of the seat occupancy state of the seat arrangement.

[0024] In some embodiments, the method further comprises determining, for each of at least two accumulated radar point clouds taken into account when detecting the seat occupancy state, a respective value of at least one determined characteristic value for characterizing the radar point cloud. The evaluation model is determined or is to be determined in such a way that, when detecting the seat occupancy state of the seat arrangement, an evaluation result is determined depending on a respective value of the at least one characteristic value for the at least two accumulated radar point clouds taken into account when detecting the seat occupancy state. In this way, the evaluation model can be determined and applied in a simplified manner, since instead of the entire accumulated radar point cloud, only the value of the at least one characteristic value has to be taken into account as input variable. In particular, the time curve of the values ​​can reflect a movement or a specific movement pattern of one or more objects on the seat arrangement, so that the evaluation model can detect the seat occupancy state of the seat arrangement in a particularly reliable manner on this basis.

[0025] In particular, in some embodiments, the subsets are selected on a rolling basis to form a regular sequence of subsets, with the first of the subsets having a number N of consecutive point sets, and each subsequent further subset emerging from the respective preceding subset by replacing the preceding point set of the preceding subset with the next M point sets in the sequence in the subsequent subset, based on their order in sequence M, where 0 < M < N applies.

Number

[0026] In particular, the following value ranges have been found to be advantageous in ensuring high quality of the method: N is selected or is to be selected such that 4 < N < 8; and / or M is selected or is to be selected such that M = 1 or M = 2.

[0027] A particularly suitable selection is N = 6 and M = 1. In this case, each respective radar point cloud of the N = 6 temporally consecutive frames is combined to form a subset or an accumulated radar point cloud based thereon. For example, if the measurement frames are numbered consecutively 1, 2, 3,... etc. in their time series, the first accumulated radar point cloud can be generated by accumulating the individual radar point clouds of measurement frames 1 to 6.

[0028] A second accumulated radar point cloud can then be generated by accumulating the individual radar point clouds of measurement frames 2-7, etc. In this rolling process, the first measurement frame of the preceding subset is therefore no longer included in the immediately following subset, but is replaced by the next measurement frame that follows it. Overall, this results in a time sequence of accumulated radar point clouds, each of which is determined by the accumulation of N individual radar point clouds. In this way, a time curve of successively accumulated radar point clouds can be generated, again with the aforementioned advantages.

[0029] In some embodiments, when detecting the seat occupancy of a seat arrangement, an evaluation result is detected for each characteristic value as a function of a series of values ​​of the characteristic values ​​for a subset of the series corresponding to a series of subsets, which also reduces the complexity of the input variables for the evaluation model, in particular by reducing the computational effort and allowing high performance, in particular real-time performance, to be achieved with less powerful or less computational resources.

[0030] In some embodiments, the respective value series is analyzed for at least one of the characteristic values ​​to detect whether a periodic curve of the characteristic values, in particular a curve corresponding to a periodic breathing pattern, is detected there. Then, depending on the result of the analysis, an evaluation result is detected. In particular, not only can the presence of any object on a seat of the seating arrangement be detected, but also a discrimination can be made with high reliability between living organisms present on the seating arrangement, in particular between mammals such as pets (e.g. dogs) and humans. In particular, depending on the detection or non-detection of such a periodic curve, it is possible to display information that is output according to the method. In particular, one or more detected frequencies of the periodic curve can be taken into account, so that in particular the information depends on whether the frequency is within a certain frequency range, such as a typical breathing frequency range. In particular, a seat belt tightening warning function or an airbag system can be controlled depending on whether a breathing rate, and therefore with high probability a human one, is detected on the seating arrangement or on a particular seat thereof.

[0031] In some embodiments, the characteristic value or one of the characteristic values ​​is or will be determined by the number of radar points in the respective accumulated radar point cloud or by an amount dependent on that number. If, in a radar scan in a measurement frame, the number of radar points of the radar point cloud generated depends on the degree to which the scanned object is moving, the degree of that movement can thus be represented in the characteristic value, in particular with regard to the aforementioned detection of the breathing frequency of the object.

[0032] In some embodiments, the sequence of measurement frames is or will be defined such that temporally successive measurement frames follow each other periodically with a frequency f, where 6 fps≦f≦9 fps. In particular, a frequency f of 7 fps≦f≦8 fps has been shown to be advantageous (fps=frames per second, i.e. measurement frames per second).

[0033] In some embodiments, the assessment model comprises a trained machine learning model. At least data representative of the accumulated radar point cloud or values ​​of one or more characteristic values ​​defined thereon are provided as input data to the machine learning model in order to obtain an assessment result as output thereof. This allows for the implementation of a particularly flexible and adaptable assessment model, whereby machine learning can be used to continuously improve the assessment model and thereby improve the quality and reliability of the seat occupancy recognition. In particular, the assessment result can indicate a class of seat occupancy classification. In particular, the machine learning model can be a decision tree-based model or a model based on an artificial neural network.

[0034] In some embodiments ("first group"), the seat arrangement comprises a number of seats, the seat status of which is detected individually or cumulatively as part of the method. For each (individual) radar point cloud, the set of radar points is subdivided by clustering into a number of clusters, each of which comprises a subset of radar points, depending on the respective spatial location of the radar points with respect to the seats, such that one of the clusters that is spatially closest to it is assigned to each of the seats individually (hence, clustering has already been performed at the level of the individual radar point clouds). The accumulation of the radar point clouds of the sequence is performed in clusters, and for each cluster, only the radar points of the respective accumulated radar point clouds that belong to said cluster are accumulated to form a respective accumulated radar point cloud for each cluster. The detection of the seat occupancy state of the seat arrangement comprises an individual detection of each individual seat occupancy state of each seat depending on the accumulated radar point clouds detected for the respective associated cluster, in order to obtain an evaluation result, in particular a classification result, characterizing the seat occupancy state of each seat. Then, depending on the individual evaluation results for the various seats, the information to be output is determined.

[0035] In some embodiments of the first group, each radar point cloud is segmented into a number of clusters by each of the seats being assigned a subset of the radar points of the respective radar point cloud as a cluster depending on their respective location, whereby the radar points of a cluster are located in a defined closed spatial region, in particular a rectangular parallelepiped, in the vicinity of the seat. This allows a particularly simple and computationally inexpensive clustering and thus seat-related seat occupancy detection, the location (position and orientation) and the shape of the spatial region being defined or can be defined such that it strongly overlaps with the spatial region occupied by typical objects, in particular people, that are usually detected on the seats of a seating arrangement.

[0036] In some other embodiments ("second group"), the seat arrangement also comprises a number of seats whose seat status status is detected individually or cumulatively as part of the method. However, here, for each accumulated radar point cloud, its respective set of radar points is subdivided by clustering into a number of clusters each including a subset of radar points depending on the respective spatial location of the radar points with respect to the seat, such that each seat is assigned individually one of the clusters that is spatially closest to it (wherein the clustering is performed at the level of the previously accumulated individual radar point clouds). The detection of the seat occupancy status of the seat arrangement comprises an individual detection of the occupancy status of each individual seat for each seat depending on the accumulated radar point clouds detected for each associated cluster, in order to obtain an evaluation result characterizing the seat occupancy status of each seat. The information to be output is determined depending on each individual evaluation result for the various seats.

[0037] Thus, the second group of embodiments is an alternative to the first group of embodiments. In both cases, in the case of a multi-seat seating arrangement, seat-related, i.e. seat-specific, seat occupancy detection is made possible, which is particularly advantageous or even necessary if a seat-related response is made by activating or deactivating or otherwise controlling for a particular seat a specific function or system, such as a seat-related airbag system, a seat-related seat belt warning or a seat-related seat heater, depending on the detected seat occupancy.

[0038] In some embodiments of the second group, each accumulated radar point cloud is segmented into a number of clusters by assigning each of the seats, according to their respective location, to a subset of the radar points of the respective accumulated radar point cloud as a cluster, whereby the radar points of a cluster are located in a defined closed spatial region, in particular a rectangular parallelepiped, in the vicinity of the seat. This allows a simplified and less computationally intensive and therefore seat-related seat occupancy detection, also compared to the first group, where the location (position and orientation) and shape of the spatial region are defined or can be defined such that it strongly overlaps with the spatial region occupied by typical objects, in particular people, that are usually detected on the seat of a seating arrangement.

[0039] Thus, each cluster comprises a subset, particularly true, of the set of radar points of the respective overall individual radar point cloud (for the first group) or pooled radar point cloud (for the second group). In some embodiments, the clustering can be performed in a manner that makes the clusters particularly discontinuous, such that a radar point cannot be assigned to two different clusters.

[0040] In some embodiments, the or each individual or accumulated radar point cloud is segmented into a plurality of clusters by assigning a subset of the radar point cloud to each of the seats as a cluster according to their respective positions, in particular uniquely for each radar point, such that the radar points of a cluster are located in a closed, in particular rectangular, spatial region defined in the vicinity of the seats. In particular, the assignment can be made in such a way that each radar point is assigned to the cluster of the seat in the closest vicinity to it. Thus, the radar point cloud can be divided into clusters, i.e. into subsets of the radar point cloud localized in the vicinity of the respective seats, whereby based on the cluster assigned to the respective seats, the seat-specific seat occupancy can be detected in a targeted manner and therefore with high reliability.

[0041] In some embodiments, outputting the information comprises activating a signal source in response to the information, causing the signal source to output a signal determined in response to said activation. The signal source may in particular be an audio source, an optical signal source, a display device, in particular for images or texts, and / or a tactile actuator or a combination of at least two of the aforementioned signal sources. This means that the signal transmission can inform a user of the detected seat occupancy state or can be used to control another technical system, such as an airbag system.

[0042] In some of these embodiments, the signal source is enabled in response to the information so as to output a signal, specifically as determined by the enablement, when the information results from an evaluation result that at least one seat of the seating arrangement is occupied and / or a selected predetermined seat occupancy state exists.

[0043] In some embodiments, the method further comprises: (i) receiving seat belt information detecting or identifying a seat belt fastening state of at least one seat in the seating arrangement; and (ii) enabling a signal source in response to the seat belt information and information from the evaluation result to output a seat belt fastening warning signal if, based on the information, the seat belt information indicates that at least one seat in the seating arrangement is occupied and / or a selected predefined seat occupancy state exists and the associated seat belt is not fastened. In this way, a radar-based seat belt checking and warning system can be realised, particularly with respect to detection only.

[0044] In some embodiments, individual radar points of the radar point cloud are represented by their respective radar point positions in three-dimensional space and by at least one of the following parameters: (i) a Doppler shift value of the radar signal at the relevant radar point; (iii) a signal-to-noise ratio value of the radar signal at the relevant radar point. These parameters can be used to pre-filter the radar point cloud, particularly as part of a pre-processing stage before feature extraction.

[0045] In particular, in some of these embodiments, the detection of the seat occupancy of the seat arrangement based on the evaluation model is performed exclusively, or at least predominantly in a numerical sense, based on those radar points whose Doppler shift value is equal to or greater than a predefined non-zero shift threshold. Thus, only or at least predominantly so-called dynamic radar points are used as the basis of the evaluation, i.e. such radar points are used as indicative of a movement of the scanned object, whose Doppler shift value is equal to or greater than the shift threshold. This can in particular be used to further improve the quality, in particular the reliability, of the method, since static, i.e. essentially stationary, points on the object surface, for example on the seat surface of the seat, are not included in the evaluation or are less numerous than points that show movement and are therefore more likely to be assigned to a living being, in particular a human or an animal. This allows the output information resulting from the evaluation (for example for an airbag control or seat belt warning system) to be used, in particular depending on whether a living being or a stationary object surface is detected.

[0046] A second aspect of the solution relates to a system, in particular a data processing device, for automatic detection of a seat occupancy state, in particular of any, seating arrangement with at least one seat, in particular by at least one vehicle seat in or for a vehicle, where the system comprises a data processing device configured to execute, in particular by means of a corresponding computer program, the method according to the first aspect for detecting a seat occupancy state.

[0047] A third aspect of the solution relates to a computer program or computer program product comprising instructions which, when executed on a data processing device of a system according to the second aspect, cause the system to perform the method according to the first aspect.

[0048] The computer program can in particular be stored on a non-volatile data carrier, which is preferably an optical data carrier or a data carrier in the form of a flash memory module. This can be advantageous if the computer program is processed as such independently of the processor platform on which the program or programs are executed. In another embodiment, the computer program can be present as a file in the data processing unit, in particular on a server, and can be downloaded via a data link, for example via a dedicated data link such as a proprietary or local network or via the Internet. Furthermore, the computer program can have a number of individual interacting program modules. In particular, the modules can be configured in such a way that they can be used in a distributed computing sense in different devices (computers or processor units) that are geographically separated and connected to each other via a data network, or can be used in any case.

[0049] The system according to the second aspect may accordingly have a program memory in which a computer program is stored, or it may be arranged to access, via a communications link, computer programs available externally, for example on one or more servers or other data processing units, in particular for exchanging therewith data used during the execution of a method or computer program or data constituting the output of a computer program.

[0050] A fourth aspect of the present solution relates to a vehicle comprising (i) a seating arrangement having at least one seat, (ii) a radar sensor for radar scanning at least a section of the seating arrangement, and (iii) a system according to the second aspect for automatic detection of the seat occupancy state of the seating arrangement, in particular of each of the seats, in response to the radar scanning of at least a section of the seating arrangement performed by the radar sensor.

[0051] The features and advantages explained with regard to the first aspect of the solution apply correspondingly to the further aspects of the invention.

[0052] Further advantages, features and application possibilities of the solution can be seen in the following detailed description in conjunction with the figures. [Brief description of the drawings]

[0053] In the drawings: [Figure 1] FIG. 1 shows a schematic representation of an exemplary embodiment of a vehicle, which is equipped with a system for automatic seat occupancy detection of a seating arrangement of the vehicle. [Diagram 2] FIG. 2 shows diagrammatically the vehicle from FIG. 1, where a baby lying in a baby seat on the passenger seat is present as an object to be detected. [Figure 3A] FIG. 3A shows an exemplary two-dimensional depiction of a (single) radar point cloud corresponding to a single measurement frame captured by the radar sensor of the vehicle from FIG. [Figure 3B] FIG. 3B illustrates an exemplary two-dimensional depiction of an accumulated radar point cloud generated by accumulation from individual radar point clouds of each of multiple successive measurement frames recorded according to FIG. 3A. [Figure 3C] FIG. 3C shows an example depiction of clustering of the accumulated radar point cloud of the embodiment of FIG. 3B according to the position of individual seats in the seating arrangement. [Figure 4] FIG. 4 shows a flow diagram illustrating an exemplary embodiment of a method for automatically detecting seat occupancy of a seating arrangement. [Figure 5A] FIG. 5A shows an exemplary depiction of curves of characteristic values ​​for individual radar point clouds for a series of measurement frames. [Figure 5B] FIG. 5B shows an exemplary depiction of a curve of characteristic values ​​for an already accumulated radar point cloud for the same series of measurement frames as FIG. 5A. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0054] In the figures, the same reference numbers indicate identical, similar or mutually corresponding elements. The elements shown in the figures are not necessarily drawn to scale. Rather, the various elements shown in the figures are reproduced in such a way that their function and general purpose can be understood by a person skilled in the art. The connections and couplings between the functional units or elements shown in the figures can also be implemented as indirect connections or couplings, unless expressly stated otherwise. The functional units can in particular be implemented as hardware, software or a combination of hardware and software.

[0055] The exemplary embodiment of the vehicle 100, which is shown diagrammatically in Fig. 1, comprises a seating arrangement 105 with five individual seats or seating positions 105a-105e. Each of the seats 105a-105e is suitable for accommodating one human being as a passenger of the vehicle 100. The vehicle 100 further comprises a radar sensor 110, which is mounted in the passenger compartment on its ceiling and which is configured so that it can at least substantially scan the seating arrangement 105 by means of a radar beam. The seats 105a-105e, in particular their seating surfaces, are therefore at least in each case largely located within an observation field 110a which can be scanned by the radar sensor 110. Furthermore, the vehicle 100 comprises a system 115 for automatic detection of a seat occupancy state of the seating arrangement 105 in response to a radar scan of at least a section of the seating arrangement 105 relative to the observation field 110a, which is performed by the radar sensor 110.

[0056] The system 115 comprises in particular a data processing unit 115a having at least one microprocessor and a memory 115b in signal connection therewith, in which a computer program is stored and which is configured to execute the method described below with reference to Figure 4 for the automatic detection of the seat occupancy state of the seating arrangement 105. Furthermore, sensor data generated by the radar sensor 110 during a radar scan or information already obtained therefrom by further processing can be or is stored in the memory 115b.

[0057] The vehicle 100 shown in Fig. 2 corresponds to the vehicle of Fig. 1, but in this case a baby seat B on which a baby lies is arranged in the passenger seat 105b as an object to be detected as part of the seat occupancy check. In the further following description of Figs. 3A to 5B, reference is made to the configuration from Fig. 2.

[0058] In the following, reference is made to Figures 3A-3C, which each show a radar point cloud and, for clarity of depiction, each radar point cloud, which is three-dimensional in nature, has been reduced to two dimensions by projection of the radar point locations of the radar point cloud onto a plane spanned by two of its dimensions.

[0059] 3A shows an exemplary single radar point cloud 300 acquired within a single measurement frame, i.e. as a result of a radar scan of the seat arrangement 105 by the radar sensor 110 during a defined time interval (measurement period). The positions of the individual radar points within the radar point cloud 300 can be represented by spatial coordinates, for example Cartesian coordinates X and Y can be assigned to the plane of the drawing and thus to the individual points. In reality, if the dimensional reduction due to the drawing is to be ignored, a third coordinate Z should be added for the third spatial dimension.

[0060] If during a radar scan not only the spatial position of the location where the radar beam is reflected from the scanned object is obtained as a coordinate, but also the Doppler shift is measured, then the individual radar points can be classified based on the magnitude of this Doppler shift, in particular into two different classes. The latter can be achieved, for example, by comparing the Doppler shift with a predefined shift threshold, which corresponds to a certain shift speed. Depending on the result of the comparison, those radar points 310 that have no speed based on their associated Doppler shift value or have a speed of the object surface at the reflection point that is smaller than the shift threshold can be classified as "static" radar points (indicated in each case by a filled black circle in Figs. 3A-3C). Conversely, those radar points 315 that have a Doppler shift larger than the shift threshold can be classified as "dynamic" radar points 315 (indicated in each case by a black ring in Figs. 3A-3C).

[0061] Although the classification of radar points 310 and 315 according to their Doppler shift is not essential, it can be used to process radar point cloud 300, in particular in the context of pre-processing performed before its evaluation, in particular to filter it according to the classification. For example, this filtering can be performed in such a way that only dynamic radar points 315 are taken into account for the evaluation, e.g. to detect only moving objects.

[0062] FIG. 3B shows an accumulated radar point cloud 305, which was generated by accumulating a number of individual radar point clouds, for example by accumulating six individual radar point clouds (i.e., N=6) from six, particularly consecutive, measurement frames. In this example, the individual radar point cloud 300 from FIG. 3A is one of these radar point clouds that is combined by accumulation, so that its radar points 310 and 315 are also seen in FIG. 3B. Comparing FIG. 3A and FIG. 3B, it is easy to see that due to accumulation, the average radar point density in FIG. 3B is significantly higher than that in FIG. 3A. In particular, it can be seen that a particularly high radar point density is prevalent in each circled region 320, which includes in particular both static radar points 110 and dynamic radar points 115. This indicates a high probability (high radar point density) of the presence of at least partially moving objects (high point density of dynamic radar points).

[0063] N=6 represents a particularly advantageous choice for N since it covers both multiple periods of the periodic breathing process recognized in FIG. 3B and the accumulation in terms of the number of individual radar point clouds that are combined by accumulation is short enough to obtain all the important data (such as the breathing pattern itself and the length of the breath) for subsequent evaluation in the resulting accumulated radar point cloud 305.

[0064] FIG. 3C shows the same radar point cloud 305 as in FIG. 3B. In addition, however, selected spatial regions 325a-325e, cuboidal (in the 3D case) or rectangular in the present 2D representation, are now shown, which are spatially assigned to the respective positions of the individual seats 105a-105e. The definition of these spatial regions 325a-325e can be used to cluster the accumulated radar point cloud 305, each radar point 310 or 315 being assigned, as far as possible, to the spatial region 325a-325e in which it is located. Any radar point that is not located in one of the spatial regions 325a-325e can then be ignored. In particular, it can be seen that the region 320 with high radar point density is located in the region of the passenger seat 105b, where the baby seat B with the baby is located according to FIG. 2.

[0065] 4 shows a flow diagram illustrating an exemplary embodiment 400 of a method for automatically detecting the seat occupancy state of a seating arrangement, which method can in particular be embodied as a computer-implemented method. To this end, it can in particular be stored as a computer program in the memory 115b of the system 115 and executable on the data processing unit 115a.

[0066] In the method 400, an initialization process 405 is first performed, whereby initial values ​​for various parameters of the method are set. In particular, a total number i for measurement frames and a counter j for the generated and accumulated radar point clouds can be initialized simultaneously, for example as i=0 and j=0. Furthermore, in particular a parameter N can be initialized here, which specifies the number of individual radar point clouds corresponding to an appropriate number of different measurement frames (e.g. radar point cloud 300), which are to be combined into an accumulated radar point cloud (e.g. accumulated radar point cloud 305). Furthermore, a further parameter M can be initialized, which specifies how many new measurement frames are added in the next rolling step as part of the rolling accumulation, while simultaneously removing a corresponding number of old measurement frames. Furthermore, the number L of measurement frames can be defined as a further parameter that must be included in total as part of the rolling accumulation before an evaluation of the generated sequence of accumulated radar point clouds is performed with respect to the detection of the seat occupancy. In this example, exemplary values ​​N=6, M=1 and L=48 are set.

[0067] After the initialization process 405 is completed, another phase of the method 400 is started, in which radar measurement data are received, in this example from the radar sensor 110 of the vehicle 100, and further processed to form one or several accumulated radar point clouds. For this purpose, in a process 410, firstly radar measurement data for a new measurement frame is received from a time sequence of a plurality of measurement frames, in particular from a periodic sequence. At the same time, the total number i is incremented to indicate the reception of radar measurement data of a new measurement frame. Then, in a further process 415, it is checked whether radar measurement data for N measurement frames have already been received. If not (415-no), the process 410 is repeated until radar measurement data for N measurement frames have been received, each representing a single radar point cloud measured at a time or period corresponding to the respective measurement frame.

[0068] Then, in a further process 420, the received N first individual radar point clouds 300 are accumulated to form an accumulated radar point cloud 305. Optionally, filtering can be performed to ensure that static radar points are filtered out, i.e. radar points having a Doppler shift value smaller than a predefined Doppler shift threshold. The Doppler shift threshold can also be optionally initialized, in particular in the initialization process 405.

[0069] The now existing accumulated radar point cloud 305 can then be clustered in a further process 425 in which each of the radar points is checked to find out whether and if so which of the spatial regions 325a-325e (see FIG. 3C) in which it is defined. Each and every one of the points can thus be assigned to one of the spatial regions 325a-325e or to another observation field. All radar points located in the same spatial region 325a-325e are combined to form a cluster. As a result, each of the sheets 105a-105e is assigned a corresponding cluster of the accumulated radar point cloud 305. This forms the basis for the fact that for each sheet 105a-105e an individual assessment can be made as to whether or not the respective sheet 105a-105e is occupied or not or was occupied or not during the N measurement frames in which the accumulated radar point cloud 305 was formed.

[0070] To facilitate subsequent evaluation of the clustered accumulated radar point cloud 305, a corresponding characteristic value K(i) can be found for each of the clusters in process 430, which characteristic value K(i) can be defined as, among other things, the number of radar points in the cluster. If no Doppler shift value filtering has been performed, this can be the combined total number of both static and dynamic radar points 310, 315, respectively. However, if the static radar points 310 have previously been filtered, for example, in process 420 discussed above, this will only include the total number of dynamic radar points 315.

[0071] Since the subsequent evaluation will be based on the curve of the characteristic value K(i) over the value range of the total number i defined by the parameter L, a check is made in process 435 to detect whether a total of L measurement frames have already been processed. If not (435-no), the method returns to step 410 in order to receive another measurement frame and detect further points and their characteristic values ​​of the new accumulated radar point cloud. If not (435-yes), the procedure can continue to detect seat occupancy in process 440, as described below.

[0072] However, reference will first be made to Figures 5A and 5B, which show example curves of values ​​of the characteristic value K(i) for clusters formed for sheet 105b or spatial region 325b in Figures 5A and 5B, respectively.

[0073] In Fig. 5A, for comparison with Fig. 5B, the number of radar points counted for each individual measurement frame i, K(i), is shown without accumulation. The total number K(i) for the first L=48 measurement frames is shown here.

[0074] In contrast, in FIG. 5B according to method 400, a corresponding diagram of curve 505 of characteristic value K(i) is shown, where a rolling accumulation was performed over each of N=6 measurement frames. The total number i now corresponds to the index of the first measurement frame that is considered as part of the accumulation. In this way, the accumulated number of radar points resulting from the accumulation of the first N=6 measurement frames 1-6 is assigned the total value i=1 in the diagram of FIG. 5B. And the accumulated number of radar points from the subsequent accumulation of measurement frames 2-7 is assigned the value i=2, and so on.

[0075] According to the initialization of N=6 and M=1 selected in this example, six measurement frames or their associated individual radar point clouds 300 are thus accumulated, and during the transition from one accumulation to the next, the first measurement frame is removed from the next accumulation and replaced by the next measurement frame not yet considered in the sequence of measurement frames. This is shown in Fig. 5B by a square bracket in the lower left part of the figure. The rolling accumulation proceeds until a number of measurement frames determined by the third parameter L has been processed in total, so that in this example a total of L=48 measurement frames are considered as part of the accumulation for the formation of the various accumulated radar point clouds.

[0076] Comparing the curves of K(i) from Fig. 5A and Fig. 5B, it is easy to see that the curve of K(i) from Fig. 5B is smoothed compared to that of Fig. 5A. However, the characteristic periodic patterns that may be caused in particular by the breathing movement of the baby B on the seat 105b detected during the radar measurement remain present with a proper selection of N=6. However, the larger the value of N, the more likely such patterns are to be flattened or disappear as part of the smoothing, which means that N should be specified appropriately here with a view to preserving these patterns.

[0077] Now referring again to FIG. 4, an evaluation of the characteristic value K(i) curve for the cluster of seats 105b can be performed (the same process can be performed for each cluster for the other seats). In process 440, the characteristic value K(i) curve from FIG. 5B is provided as an input variable to an evaluation model. This can in particular be a machine learning based model such as an artificial neural network or a decision tree based model. The training data and optionally the validation data used for the previous training can be structured such that they each contain curves of characteristic values ​​K(i) corresponding to the type shown in FIG. 5B for a number of different radar point clouds or clusters thereof, and for each curve of K(i) an assigned correct class of classification of possible seat occupancy states. This makes it possible to train and validate the model as part of a supervised learning procedure. In the simplest case, the occupancy state of a seat indicates whether the seat is occupied or not. However, more advanced classifications are also conceivable, in which, if an object is present, each class additionally specifies the type of object, e.g. whether it is a moving or stationary object and, in particular in the case of a moving object, whether it is a human (which in principle can be identified by the breathing pattern, in particular in the characteristic value K(i) curve).

[0078] As an alternative to machine learning models, particularly simple comparison algorithms can also be used, in which the curves of characteristic values ​​K(i), in the sense of a mathematical function, are used as input variables and compared with one or more comparison thresholds to obtain a classification result. In particular, it is conceivable to carry out a frequency analysis of K(i), for example by means of a Fourier transform, in order to detect the respiratory frequency on which the classification is based.

[0079] If in process 440 a seat occupancy state is detected based on the evaluation model for the seat arrangement 105, in particular for one or more of its seats 105a-105e individually, this result can be output in process 445 as corresponding information, for example at a user interface of the vehicle or in the form of data for further processing by one or more other systems, in particular a system of the vehicle.

[0080] In this example, this information is used, inter alia, to check whether a belt warning signal should be issued depending on the seat occupancy status of a particular seat 105a-105e and the result of a test to detect whether the corresponding seat belt for that seat is fastened or not.

[0081] To do this, in process 450 it can be checked whether the seat belt for the seat of interest (here, for example, for seat 105b) is fastened and in step 455 the functionality of vehicle 100 is controlled depending on the information on the occupancy status of the seat output in process 445 and on the status of the seat belt detected in process 450. In particular, this can be implemented such that in process 455 a signal source for outputting in particular an optical and / or acoustic seat belt status signal is enabled, if necessary, in order to inform one or more other occupants of the vehicle that the seat is occupied but the seat belt therein is not fastened. The method then branches to step 410, starting another loop pass.

[0082] Although at least one exemplary embodiment has been described, it should be noted that there are many variations in this regard. It is also noted here that the described exemplary embodiments constitute only non-limiting examples, and that they are not intended to limit the scope, applicability, or configuration of the apparatus and methods described herein. Instead, the above description provides those skilled in the art with suggestions for implementing at least one exemplary embodiment, and it is understood that various changes in the functional means and arrangement of elements described in the exemplary embodiments can be made without departing from the subject matter herein, as defined in the appended claims or their legal equivalents, respectively. List of References B Baby in a baby seat 100 Vehicles 105 Seat Layout 105a-e Seat or seat position 110 Radar Sensor 110a Observation field of radar sensor 110 115 System for automatic seat occupancy detection 115a Data Processing Unit 115b Memory 300 (single) radar points 305 Accumulated radar point clouds 310 Static Radar Points 315 Dynamic Radar Points 320 Areas of the accumulated radar point cloud 305 that have high radar point density 325a-e Spatial domains for cluster definition 400 Method for automatically detecting seat occupancy 405-455 Individual process or method steps within the context of method 400 500 Curve of characteristic value K(i) for a single radar point cloud 505 Curve of characteristic value K(i) for cumulative radar point cloud

Claims

1. A method for automatically detecting a seat occupancy state of a seating arrangement (105) having at least one seat (105a-e), comprising: receiving or generating, for each measurement of a sequence of a plurality of temporally consecutive measurement frames, measurement data representative of one assigned radar point cloud (300), whereby the measurement data represents a sequence of radar point clouds corresponding to said sequence of measurement frames, each radar point cloud (300) of said sequence having been obtained or derived based on a radar scan of a spatial region surrounding at least a portion of a section of said seat arrangement (105) performed at a measurement time or during a measurement period assigned to the respective measurement frame; accumulating the sequence of multiple radar point clouds, as part of the accumulation process, to obtain an accumulated radar point cloud (305) that includes radar points (310, 315) from each of the individual radar point clouds combined; detecting a seat occupancy state of the seating arrangement (105) based on an assessment model that returns as an assessment result one of a plurality of predefined possible seat occupancy states of the seating arrangement (105) in response to the accumulated radar constellation; and outputting information determined based on the evaluation results; The method includes:

2. the accumulation of the plurality of radar point clouds of the sequence is performed multiple times to obtain an accumulated radar point cloud (305) that includes radar points (310, 315) from each of the individual radar point clouds combined as part of each accumulation, during each accumulation, only the radar point cloud of a distinct subset of the sequence assigned thereto is accumulated, whereby at least two of the subsets are different; the seat occupancy state of the seating arrangement (105) is detected in response to at least two of the accumulated radar point clouds; The method of claim 1.

3. determining, for each of the at least two accumulated radar point clouds taken into account when detecting the seat occupancy state, a respective value of at least one defined characteristic value (K(i)) for characterizing the radar point cloud; the evaluation model is determined or will be determined such that, when detecting the seat occupancy state of the seat arrangement (105), the evaluation result is determined as a function of a respective value of the at least one characteristic value (K(i)) for the at least two accumulated radar point clouds taken into account when detecting the seat occupancy state, The method of claim 2.

4. the subsets are selected on a rolling basis to form an ordered series of subsets, a first of the subsets having a number N of consecutive sets of points in the sequence, each subsequent further subset emerging from a respective preceding subset by replacing the preceding set of points of the preceding subset with the next M sets of points in the sequence in the next subset based on their order in the sequence M; [0010] applies, where 0<M<N, The method according to claim 3.

5. N is selected or will be selected such that 4<N<8; and / or M is selected or will be selected to be M=1 or M=2; The method according to claim 4.

6. when detecting the seat occupancy state of the seat arrangement (105), the evaluation result is detected for each characteristic value (K(i)) depending on the series of respective values ​​of the characteristic value (K(i)) for the subset of the series corresponding to the series of subsets, The method according to claim 4 or 5.

7. for at least one of said characteristic values, said series of their respective values ​​is analyzed to detect whether a periodic curve of said characteristic value (K(i)) is detected therein; The evaluation result is detected based on the result of the analysis. The method according to claim 5.

8. the characteristic value or one of the characteristic values ​​is or will be determined by a number or a quantity corresponding to a number of radar points (310, 315) in each accumulated radar point cloud; The method according to any one of claims 3 to 7.

9. 9. The method according to claim 1, wherein the sequence of measurement frames is or is to be defined such that temporally successive measurement frames therein follow each other periodically with a frequency f, where 6 fps≦f≦9 fps.

10. the assessment model comprises a trained machine learning model; data representative of at least the accumulated radar point clouds (305) or values ​​of one or more characteristic values ​​defined therefor are provided as input data to the machine learning model in order to obtain the assessment result as its output; The method according to any one of claims 1 to 9.

11. said seating arrangement (105) comprising a plurality of seats (105a-e) for which a seat occupancy state is detected individually or cumulatively as part of said method; for each radar point cloud (300), the set of radar points (310, 315) is subdivided by clustering into a plurality of clusters each including a subset of the radar points (310, 315) according to their respective spatial locations with respect to the sheets (105a-e), such that each of the sheets (105a-e) is individually assigned one of the clusters located spatially closest thereto; the accumulation of the radar point clouds of the sequence is performed in clusters, and for each cluster, only the radar points (310, 315) that belong to the cluster of each accumulated radar point cloud are accumulated to form a respective accumulated radar point cloud (305) for each cluster; and said detection of the seat occupancy status of the seating arrangement (105) comprises an individual detection of a respective individual seat occupancy status for each of said seats (105a-e) in response to the accumulated radar point cloud detected for each associated cluster to obtain an assessment result characterizing the seat occupancy status of each seat (105a-e); The information to be output is determined based on the results of each individual evaluation of the different sheets (105a-e). The method according to any one of claims 1 to 10.

12. each of the sheets (105a-e) is assigned a subset of the radar points (310, 315) of a respective radar point cloud (300) as a cluster according to their respective positions, such that each radar point cloud (300) is segmented into a plurality of clusters, whereby the radar points (310, 315) of a cluster are located in a defined closed spatial region in the vicinity of the sheets (105a-e); The method of claim 11.

13. the seating arrangement (105) comprises a plurality of seats (105a-e), the seat occupancy status of which is detected individually or cumulatively as part of the method; for each accumulated radar point cloud (305), its respective set of radar points (310, 315) is subdivided by clustering into a plurality of clusters, each cluster including a subset of the radar points (310, 315), according to the respective spatial locations of the radar points (310, 315) with respect to the sheets (105a-e), in order to individually assign to each of the sheets (105a-e) one of the clusters located spatially closest thereto; said detecting the seat occupancy status of said seating arrangement (105) comprises detecting a respective individual seat occupancy status for each of said seats (105a-e) in response to said accumulated radar point cloud detected for each associated cluster to obtain an assessment result characterizing the seat occupancy status of each seat (105a-e); The information to be output is determined based on the results of each individual evaluation of the different sheets (105a-e). The method according to any one of claims 1 to 10.

14. each accumulated radar point cloud (305) is segmented into a plurality of clusters by each of said sheets (105a-e) being assigned a subset of said radar points (310, 315) of each accumulated radar point cloud as a cluster according to their respective positions, such that the radar points (310, 315) of said clusters are located in defined closed spatial regions in the vicinity of said sheets (105a-e); The method of claim 13.

15. A method according to any one of claims 1 to 14, wherein the output of the information comprises enabling a signal source in response to the information, causing the signal source to output a signal determined in response to the enabling.

16. 16. The method according to claim 15, wherein the signal source is enabled in response to the information to output a signal if the information results from an evaluation result that at least one seat of the seat arrangement (105) is occupied and / or a selected predetermined seat occupancy state exists.

17. detecting a seat belt fastening state of at least one seat in the seating arrangement (105) or receiving seat belt information characteristic of said seat belt fastening state; the signal source is enabled in response to the seat belt information and the information from the evaluation result, whereby based on the information, it is adapted to output a seat belt fastening warning signal if at least one seat of the seat arrangement (105) is occupied and / or a selected predetermined seat occupancy state exists, indicating seat belt information that the associated seat belt is not fastened.

17. The method of claim 16.

18. The individual radar points (310, 315) for each radar point cloud (300) are represented by their respective radar point positions in three-dimensional space and by at least one of the following parameters: - the Doppler shift value of said radar signal at said relevant radar point; the signal-to-noise ratio value of said radar signal at said relevant radar point, The method according to any one of claims 1 to 17.

19. the individual radar points (310, 315) of each radar point cloud (300) are represented by a respective radar point position in three-dimensional space and by the Doppler shift value of the radar signal for each respective radar point (310, 315); the detection of the seat occupancy state of the seat arrangement (105) based on the evaluation model is performed exclusively or at least predominantly in a numerical sense based on those radar points (310, 315) whose Doppler shift value is equal to or greater than a predetermined non-zero shift threshold value, 20. The method of claim 18.

20. A system (115) for automatically detecting a seat occupancy state of a seating arrangement (105) having at least one seat (105a-e), the system (115) comprising a data processing device configured to perform the method according to any one of claims 1 to 19 for detecting the seat occupancy state.

21. A computer program or computer program product comprising instructions which, when executed on a data processing device of a system (115) according to claim 20, causes said system (115) to carry out a method according to any one of claims 1 to 19.

22. A seating arrangement (105) having at least one seat (105a-e); a radar sensor (110) for radar scanning at least a section of the seating arrangement (105); A system (115) according to claim 20 for automatic detection of a seat occupancy state of the seating arrangement (105) in response to a radar scan of at least a section of the seating arrangement (105) performed by the radar sensor (110), and A vehicle (100) comprising:

Images