Device and method for monitoring the fluid intake of a vehicle occupant

Abstract

In a device and method for monitoring a vehicle occupant's fluid intake, an image is captured and corresponding image data is generated. Based on this image data, a drinking vessel and the occupant's drinking action are detected. Using the image data and / or weight information, a drinking volume is determined that corresponds to the volume of liquid consumed by the vehicle occupant during the drinking action.

Description

[0001] The invention relates to a device and a method for monitoring the fluid intake of a vehicle occupant using at least one interior camera, which is configured to capture at least one image depicting an area of ​​the vehicle interior encompassing at least part of a vehicle seat and to generate image data corresponding to the image. A processing unit is configured to process the image data.

[0002] For safe participation in road traffic, it is crucial that drivers can concentrate sufficiently on the diverse demands of various traffic situations and thus be able to react quickly to the behavior of other road users. However, due to the constant concentration required on the road, or even advanced age, drivers often forget to drink enough fluids. Insufficient fluid intake leads to impairments such as headaches and disorientation, which are associated with a high risk of accidents in road traffic. Other vehicle occupants also experience a significant decline in well-being due to inadequate fluid intake. Therefore, there is a need to support vehicle occupants in maintaining adequate fluid intake.

[0003] Document DE 10 2014 203 204 A1 discloses a device for holding a liquid-filled container with a temperature sensor and a control unit. The control unit issues a warning to the vehicle occupants when the temperature exceeds a predetermined limit.

[0004] Document DE 10 2022 125 658 A1 discloses a holding device for drinking containers in a vehicle. The holding device comprises at least one base plate configured to support an object and a cavity formed in a body of the holding device. The cavity is configured to receive the base plate and to at least partially receive the object supported by the base plate. The holding device is configured to change the position of the base plate within the cavity depending on the object.

[0005] Document JP 2010-186251 A discloses a device for monitoring a driver's condition. The system includes a driver monitoring unit for capturing information about a driver's facial appearance and whether their eyes are open or closed. This is achieved using facial image information of the driver captured by a driver monitoring camera. Based on facial color or temperature, it can be determined whether the driver has consumed alcohol.

[0006] The object of the invention is to provide a device and a method for simple monitoring of the fluid intake of a vehicle occupant.

[0007] This problem is solved by a device having the features of claim 1 and by a method having the features of the independent method claim. Advantageous embodiments are specified in the dependent claims.

[0008] The device with the features of claim 1 enables the volume of liquid consumed by a vehicle occupant during drinking to be easily determined as the drinking volume. Based on this, the fluid intake of the vehicle occupant can be monitored.

[0009] A vehicle occupant can be a driver, a front passenger, or a passenger in the back seat. The area of ​​the vehicle interior encompassing part of a vehicle seat can, for example, be the area that includes the head and neck of a vehicle occupant seated in a vehicle seat. Furthermore, the area can also include a holder and / or a storage compartment for objects.

[0010] The drinking vessel can be, for example, a bottle, a cup, or a thermos. It can have a base made of glass, metal, and / or plastic and an outer surface with transparent and / or opaque areas. The drinking vessel can be detected, for example, by the processing unit performing a pattern recognition procedure based on the image data. In this way, an object in the image can be assigned to the pattern "drinking vessel" based on its shape, color, surface reflective properties, and any visible liquid it may contain. Alternatively or additionally, the drinking vessel can be detected using artificial intelligence, which can identify drinking vessels and, preferably, their volume.

[0011] The act of drinking can be detected, for example, by having the processing unit process image data using a computer vision method. This allows specific arrangements of body parts to be associated with the action of drinking. These specific arrangements of body parts can include, for example, the position of the larynx as it changes during swallowing, the shape of the mouth as it becomes rounded, and / or the shape of the cheeks.

[0012] The weight sensor can determine the mass of a drinking vessel or other object, such as a solid food item, held in a holder or placed on a shelf. The weight sensor can be positioned, for example, so that an object in the holder rests on the sensor, or so that the weight of the holder and the object together is measured. The weight of the holder can then be subtracted from the total weight, allowing the object's weight to be easily determined. Alternatively, the object's weight can act directly on the weight sensor, enabling the object's mass to be determined.

[0013] The camera can be integrated into the vehicle's interior rearview mirror or its housing. It can also be positioned above the driver and / or passenger in the area of ​​the vehicle's headliner. Furthermore, the camera can be mounted on the B-pillar and / or A-pillar on the driver's and / or passenger's side, and / or on a central display unit within the vehicle. By positioning cameras in multiple locations within the vehicle, the area captured inside the vehicle can be expanded compared to capturing the interior with only one camera. For example, the interior camera can be an RGB camera. This allows for particularly good capture of patterns on a drinking vessel. It is also advantageous if the interior camera is an infrared (IR) camera.This allows for particularly precise determination of the fill level of a hot or cold beverage in a drinking vessel, especially if the vessel is a thin-walled cup. Furthermore, the use of IR images is advantageous because these images are independent of ambient light. It is also beneficial if the indoor camera is a time-of-flight (TOF) camera. This makes determining the volume of a drinking vessel particularly easy. It is especially advantageous if the indoor camera includes an RGB camera unit, an IR camera unit, and / or a TOF camera unit. This allows the aforementioned advantages to be combined.

[0014] It is advantageous if the processing unit is designed to determine at least one parameter for calculating the drinking volume from the image data. This parameter could be, for example, the volume of the drinking vessel and / or a change in the vessel's angle of inclination during drinking. The angle of inclination during drinking depends, in particular, on the desired flow rate and the remaining volume of liquid in the vessel. The change in the vessel's angle of inclination during drinking correlates with the relative decrease in the liquid volume within the vessel, allowing for easy determination of the drinking volume. The vessel volume is detected, for example, by the processing unit determining a volume marking on the vessel based on the image data.

[0015] Alternatively or additionally, the parameter for determining the drinking volume can include the number of swallows taken by the vehicle occupant during the drinking process. Each swallow can be assigned a volume of liquid that the occupant ingests with each sip. This makes it particularly easy to determine the occupant's drinking volume. It is advantageous if each swallow can be assigned a volume of liquid specific to that occupant, which is ingested with each sip. This allows for a particularly accurate and straightforward determination of the occupant's drinking volume.

[0016] Furthermore, the parameter for determining the drinking volume can alternatively or additionally include the duration the drinking vessel remains at the vehicle occupant's mouth during the drinking process. Based on this duration and a known and / or average flow rate for the vehicle occupant when drinking from the vessel, the volume of liquid consumed by the vehicle occupant during the drinking process can, for example, be easily determined.

[0017] The parameter for determining the drinking volume can alternatively or additionally include the volume of the drinking vessel and the fill level of the drinking vessel before and after drinking. The fill level before and after drinking can be determined, for example, based on the liquid level in the drinking vessel and its height / volume. This allows for a very simple determination of the vehicle occupant's drinking volume.

[0018] Furthermore, it is advantageous if the processing unit is designed to determine a parameter for calculating the drinking volume, either alternatively or additionally, based on the weight information. For example, the weight of the drinking vessel before and after drinking can be determined. Based on the weight difference and the density of the liquid or a reference density, the drinking volume of the vehicle occupant can be determined simply and accurately.

[0019] It is also advantageous if the processing unit determines the drinking volume based on several parameters derived from the image data and / or weight information. This allows the drinking volume to be determined simply and accurately.

[0020] It is advantageous if the device includes a communication unit configured to communicate data with a database. The processing unit is configured to classify the detected drinking vessel based on the image data and data stored in the database. Based on this classification of the drinking vessel and data from the database and / or another database, the processing unit can determine at least one ingredient and / or the quantity of an ingredient in the liquid contained in the drinking vessel. Furthermore, it is advantageous if the processing unit is configured, alternatively or additionally, to determine the calorific value of the liquid contained in the drinking vessel based on the classification of the drinking vessel and data from the database and / or another database.Furthermore, the processing unit can determine the amount of the ingested ingredient and / or the calorific value per serving of the beverage. This allows for the monitoring of additional factors relevant to the well-being of the vehicle occupant.

[0021] The database and / or the additional database can contain, for example, data on beverage brands, beverage names, and beverage types, such as orange juice, apple juice, coffee, tea, water, cola, Fanta, milk, and mixed drinks like latte. The data can also include nutritional information and ingredient lists. For example, a drinking vessel can be classified as a water bottle of a specific brand, and the mineral content and calorific value of the water it contains can be assigned to it.

[0022] It is further advantageous if the processing unit is equipped to add the volume of fluid consumed during a detected drinking event to the total fluid intake of the vehicle occupant since the start of the journey. Based on the total fluid intake and the journey time since the start of the journey, the processing unit can calculate a fluid intake per unit of time and compare this fluid intake per unit of time with a preset, stored initial threshold. Using an output unit equipped to provide information to the vehicle occupant, the processing unit can then issue a fluid intake recommendation to the vehicle occupant if the initial threshold is not met. This effectively supports the vehicle occupant in improving their concentration and well-being.

[0023] The total fluid intake of a vehicle occupant since the start of the journey describes the total volume of fluid consumed since the beginning of the trip. The start of the journey could be, for example, sitting down in the driver's seat. The first threshold could be, for example, a theoretical fluid requirement per unit of time, based on the daily needs of an adult. The fluid intake recommendation could be, for example, a request to drink water or a suggestion to take a drinking break. The output device could be, for example, a display unit or a speaker in the vehicle. It is advantageous if the output device is a smartphone or smartwatch. Then, when the first threshold is not reached, the processing unit can use the communication unit to transmit the fluid intake recommendation to the smartphone and / or smartwatch. The smartphone and / or smartwatch can then announce the fluid intake recommendation.This allows the attention of the vehicle occupant to be effectively and easily directed to the drinking recommendation.

[0024] It is advantageous if the processing unit is designed to add the amount of the ingredient ingested with the liquid, relative to the volume consumed, to the total intake of that ingredient by the vehicle occupant since the start of the journey. Based on this total intake and the journey time since the start, the processing unit can then determine an intake of the ingredient per unit of time and compare this intake per unit of time with a preset, stored second threshold. If the intake falls below this second threshold, the processing unit can issue an intake recommendation to the vehicle occupant using the output unit. This effectively supports the vehicle occupant in improving their concentration and well-being.It is still advantageous if the ingredient intake recommendation is provided using a smartphone and / or smartwatch.

[0025] The total amount of the substance ingested by the vehicle occupant since the start of the journey describes the total quantity of the substance ingested since the beginning of the journey. The second limit value could, for example, be a theoretical intake requirement per unit of time, based on the daily requirement for an adult.

[0026] Furthermore, it is advantageous if the processing unit is designed to add the calorific value related to the volume of fluid consumed to the total calorie intake of the vehicle occupant since the start of the journey. Based on the total calorie intake and the journey time since the start of the journey, the processing unit can calculate a calorie intake per unit of time and compare this calorie intake per unit of time with a preset, stored third threshold. Using the output unit, the processing unit can issue a calorie intake recommendation to the vehicle occupant if the calorie intake falls below the third threshold. This very effectively supports the vehicle occupant in improving their concentration and well-being. It is also advantageous if the calorie intake recommendation is issued via smartphone and / or smartwatch.

[0027] The total calorie intake of the vehicle occupant since the start of the journey describes the total number of calories consumed since the beginning of the journey. The third limit could, for example, be a theoretical intake requirement per unit of time, based on the daily requirement for an adult.

[0028] In one embodiment, the processing unit is located in a mobile unit or a central processing facility. The device then communicates data wirelessly with the processing unit via the communication unit. The mobile unit could be, for example, a smartphone or a smartwatch. The central processing facility could be, for example, a data center of a nutritional counseling service provider.

[0029] It is advantageous if the processing unit is configured to detect a solid food item and a vehicle occupant eating it, based on the image data. The processing unit is further configured to determine, based on the image data and / or weight information obtained using the weight sensor or another weight sensor, the quantity consumed, corresponding to the amount of solid food eaten by the vehicle occupant. Based on the image data and database data, the processing unit can classify the detected solid food item. Furthermore, based on the classification of the solid food item and data from the database and / or another database, the processing unit can determine at least one ingredient and the quantity of that ingredient in the solid food item.Alternatively or additionally, the processing unit can determine the calorific value of the solid food based on its classification and data from the database and / or other databases. Furthermore, the processing unit can determine the amount of the ingested ingredient per serving and / or the calorific value per serving. This provides the vehicle occupant with particularly comprehensive and effective support in improving their concentration and well-being.

[0030] Solid food items can be detected, for example, by the processing unit performing a pattern recognition procedure based on the image data. This allows the object in the image to be assigned to the pattern "solid food" based on its shape, color, and surface reflective properties. Alternatively or additionally, solid food items can be detected using artificial intelligence that is capable of detecting solid food items.

[0031] The eating process can be detected, for example, by having the processing unit process image data using a computer vision method. This allows specific arrangements of body parts to be associated with the action "eating." These specific arrangements of body parts can include, for example, the position of the larynx, which changes position during swallowing, the shape of the mouth during chewing, and the shape of the cheeks while chewing.

[0032] The database and / or the additional database can also contain data on food brands, food names, and food types, such as apple pie, croissant, salami pizza, strawberry yogurt, orange, or chocolate bar. The data can further include nutritional information and ingredient lists. For example, a solid food item can be classified as a chocolate bar of a specific brand, and its ingredients and energy value can be assigned to it.

[0033] It is advantageous if the processing unit is designed to determine the number of swallows during the eating process based on the image data. Each swallow by a vehicle occupant can be assigned a quantity of food that the occupant consumes with each swallow. Furthermore, it is advantageous if the processing unit alternatively or additionally determines the weight of the solid food before and after eating, based on the weight information. The processing unit can then determine the quantity consumed based on the determined number of swallows during eating and / or based on the determined weight of the solid food before and after eating. This allows the amount of solid food consumed to be determined particularly easily and accurately.

[0034] Furthermore, it is advantageous if the processing unit is designed to add the determined amount of the ingested ingredient, related to the portion of food consumed, to the total amount of that ingredient ingested by the vehicle occupant since the start of the journey. It is also advantageous if the processing unit is designed to add the calorific value, related to the portion of food consumed, to the total calorie intake of the vehicle occupant since the start of the journey. This provides the vehicle occupant with particularly comprehensive and effective support in improving their concentration and well-being.

[0035] In one embodiment, the processing unit is configured to output information regarding the classification of the drinking vessel via the output unit. The processing unit is further configured to receive confirmation or correction of this classification information via an input unit. Based on the correction, the processing unit can then adjust the classification of the drinking vessel. This allows for the correction of an incorrect classification, enabling more accurate determination of the quantity of contents and the calorie intake.

[0036] It is further advantageous if the processing unit is alternatively or additionally configured to output information regarding the classification of the solid food via the output unit. The processing unit is also configured to receive confirmation or correction of this classification information via the input unit. Based on the correction, the processing unit can then adjust the classification. This allows for the correction of an incorrect classification, enabling more accurate determination of the ingredient quantity and calorie content.

[0037] In one embodiment, a temperature measuring unit determines the ambient temperature of the vehicle occupant. The processing unit can then adjust the first, second, and / or third limit values ​​depending on the determined ambient temperature. Alternatively or additionally, the processing unit can also adjust the first, second, and / or third limit values ​​depending on the interior temperature, the exterior temperature, the amount of sunlight entering the vehicle, whether the convertible mode is in use, the occupant's gender, age, weight, average waking hours, individual settings, and / or the amount of liquids and / or solid food already consumed.

[0038] The method with the features of the dependent method claim has the same advantages as the claimed device. In particular, the method can be further developed with the features of the dependent claims directed to the device and the aforementioned embodiments.

[0039] Examples of implementation are explained in more detail below with reference to the figures. These show: Fig. 1a perspective schematic representation of a vehicle cockpit; Fig. 2a first image taken using the interior camera, depicting an initial consumption situation; Fig. 3. A second image taken using the interior camera, depicting a second consumption situation; and Fig. 4. A schedule for monitoring the fluid intake of a vehicle occupant.

[0040] Fig. Figure 1 shows a perspective schematic representation of a cockpit 100 of a vehicle 102.

[0041] A driver 104 sits in a driver's seat 106 of the vehicle 102. The cockpit 100 also includes a central information display (CID) 108, a head-up display 110, and a graphic instrument cluster 112, which are arranged in a dashboard 114 of the vehicle 102. The aforementioned display elements 108, 110, and 112 each form a functional unit of an output unit of the vehicle 102, which is designed to output information to the vehicle occupant 104. At least one loudspeaker 116 and one microphone 117 of the vehicle 102 are also arranged in the cockpit 100. The loudspeaker 116 forms a functional unit of the output unit, and the microphone 117 forms a functional unit of the input unit. The functional units of the output unit and the input unit also serve as playback units for audio and / or video playback.

[0042] The cockpit 100 also includes a steering wheel 118 with controls 120, a gear selector 122, pedals 124, and an input unit 126 with a rotary dial and push-button function and / or a touch input panel. This input unit 126 is also referred to as the Ergo Commander.

[0043] In the upper area of ​​the windshield 128 of the vehicle 102, an interior rearview mirror 132 and an interior camera 134, mounted on the windshield 128 in the area of ​​this interior rearview mirror 132, are arranged. The interior camera 134 is designed to capture images of an area of ​​the vehicle interior encompassing at least part of a vehicle seat 106. The interior camera 134 can be a monocular camera or a stereo camera, preferably a camera with an IR camera unit, an RGB camera unit, and / or a TOF camera unit.

[0044] An image 200 taken by camera 134 of the first consumption situation and another image 300 taken by camera 134 of the second consumption situation are in Fig. 2 and Fig. 3 shown. The camera 134 is specifically designed to capture several successive images, particularly in the form of a video stream, to generate image data corresponding to the images, and to transmit this data to a control unit 138 serving as a processing unit.

[0045] The control unit 138 is also connected to a temperature measuring unit 150. The temperature measuring unit 150 is designed to determine the ambient temperature of the driver 104. The control unit 138 has data outputs 140 and data inputs 142, which serve to connect to other units of the vehicle 102, for example, to additional cameras, sensors, input and output units, and control units of assistance systems.

[0046] The control unit 138 further comprises a communication module 144, which is configured to establish a connection with a telecommunications network, in particular a mobile communications network. Car-to-car communication and infrastructure-to-car communication are easily possible via this communication module 144. Data can be transmitted to and from the vehicle 102 via this communication module 144, in particular to and / or from a central database and / or another database of the vehicle manufacturer and / or a service provider. In a further embodiment, data can be transmitted via the communication module 144 to a processing unit 138, which is arranged in a mobile unit or a central processing facility.

[0047] Fig. Figure 2 shows a first image 200 taken using the interior camera 134, depicting a first consumption situation.

[0048] A driver 104 is sitting in the driver's seat 106, and a passenger 206 is sitting in the passenger seat 202. The passenger 206 holds the opening of a transparent drinking bottle 208, containing an orange liquid 210, to her mouth 212. The bottle 208 is tilted so that the liquid 210 flows out of it. A label 216 is attached to the outside of the bottle 208, showing a drawing of an orange and the words "ORANGE" and "1L". The passenger 206 drinks from the bottle 208, so that her mouth 212 is rounded, her cheeks 222 are distorted, and her swallowing muscles and larynx 218 are moving. In contrast, the driver 104 has his mouth 214 open, and his swallowing muscles and larynx 220 are not moving and his cheeks 224 are not deformed.A holder 228, 230 for storing items such as drinking vessels 208 and solid food items is integrated into the center console 226 of the vehicle 102, one on the passenger side 206 and one on the driver's side 104. Each holder 228, 230 includes a cylindrical recess designed to hold an item securely. Additionally, each holder 228, 230 includes a weight sensor 234, 236 for determining the weight of an item located in the holder 228, 230. The holder 228 on the passenger side 206 is empty because the passenger 206 has removed the bottle 208 from it. The weight of the bottle 208 was detected by the weight sensor 234 of the holder 228 on the passenger side 206 before the bottle 208 was removed. The holder 230 on the side of the driver 104 contains a croissant 228, the weight of which is detected by the weight sensor 236 of the holder 230.The temperature measuring unit 150 detects a temperature of 20°C in the vehicle interior. The one shown in Figure 200... Fig. The first consumption situation shown in this example takes place 30 minutes after the start of the journey.

[0049] The interior camera 134 generates image data corresponding to image 200, which is processed by the control unit 138. Based on the image data, the control unit 138 detects the drinking bottle 208 and its volume. Furthermore, the control unit 138 detects the drinking action of the passenger 206, who drinks the liquid 210 from the drinking bottle 208.

[0050] The drinking process is detected, for example, by the control unit 138 processing the image data using a computer vision method. This allows specific arrangements of body parts to be assigned to the action "drinking process". For example, the position of the larynx 218 of the passenger 206 during swallowing and the shape of the mouth 212 and cheeks 222 during drinking can be assigned to the action "drinking process".

[0051] The passenger 206 places the drinking bottle 208 back in the holder 228 on the passenger's side after drinking. Placing the drinking bottle 208 is in Fig. 2 not shown, but the drinking bottle 208, which was placed down after this drinking process, is shown in picture 300. Fig. 3 shown. The control unit 138 then determines a drinking volume based on the image data, which corresponds to the volume of the liquid 210 ingested by the passenger 206 during the drinking process.

[0052] For this purpose, the control unit 138 can determine at least one parameter for calculating the drinking volume based on the image data. The parameter determined from the image data for calculating the drinking volume can, for example, include the volume of the drinking bottle 208 and the change in the angle of inclination of the drinking bottle 208 during the drinking process. The volume of the drinking bottle 208 is detected, for example, by the control unit 138 determining from the image data that the label 216 displays the inscription "1L", whereby the control unit 138 detects a volume of one liter for the drinking bottle 208 based on this information. Alternatively or additionally, the interior camera 134 can include a time-of-flight (TOF) camera unit. The control unit 138 can then detect the volume of the drinking bottle 208 based on the image data from the TOF camera unit.

[0053] Alternatively or additionally, the parameter for determining the drinking volume can also include the number of swallows made by the passenger 206 during the drinking process and / or the duration the drinking vessel 208 remains at the passenger 206's mouth 212 during the drinking process. Furthermore, the parameter for determining the drinking volume, derived from the image data, can alternatively or additionally include the volume of the drinking bottle 208 and the fill level of the drinking bottle 208 before and after the drinking process.

[0054] Furthermore, the control unit 138 alternatively or additionally determines the volume of liquid 210 consumed by the passenger 206, based on the weight information obtained by the weight sensor 234 in the holder 228 on the passenger's side. For this purpose, the control unit 138 can determine the weight of the drinking bottle 208 before and after drinking, based on the weight information. Based on the weight of the drinking bottle 208 before and after drinking, the control unit can then determine a parameter for calculating the volume of liquid consumed. Using this parameter, determined from the weight information, the control unit 138 can then calculate the volume of liquid 210 consumed by the passenger 206. In this embodiment, the control unit 138 calculates a volume of 250 ml.

[0055] In another embodiment, the indoor camera 134 comprises at least one IR camera unit, and the drinking vessel 208 is a paper cup containing hot coffee. A portion of the outer surface of the paper cup located below the coffee level is hotter than a portion of the outer surface located above the coffee level. Based on the image data from the indoor camera 134 with an IR camera unit, the coffee level in the paper cup can then be determined particularly easily, thus also allowing the parameter for determining the drinking volume to be easily ascertained.

[0056] Furthermore, the communication unit 144 can communicate data with the database. The database contains data on beverage names and types, e.g., orange juice, apple juice, coffee, tea, water, cola, Fanta, milk, and mixed drinks such as latte. The control unit 138 can then, for example, determine from the image data that the word "ORANGE" is displayed on the drinking bottle 208 and can further determine that the liquid 210 contained in the bottle 208 is orange in color. Based on this processed image data and data stored in the database, the control unit 138 can classify the detected drinking bottle 208 as an orange juice bottle 208. In another embodiment, a drinking vessel 208 can also be classified as a water bottle, coffee mug, thermos flask with tea, or beer can.

[0057] Furthermore, the control unit 138 can determine at least one ingredient and the quantity of that ingredient in the orange juice 210 contained in the orange juice bottle 208, based on the classification of the drinking bottle 208 as an orange juice bottle 208 and data from the database and / or the further database. In this embodiment, the at least one ingredient is vitamin C. The database and / or the further database contains data on the vitamin C content in a specific volume of the orange juice 210 contained in the orange juice bottle 208. The database and / or the further database also contain data on the external appearance and ingredients, such as vitamin C, of ​​liquids in commercially packaged drinking containers, e.g., fruit juice bottles.The drawing of the orange on the orange juice bottle 208 can be assigned to a brand, so that the vitamin C content in the orange juice 210 in the commercially packaged drinking vessel 208 of this brand can be determined based on the data in the database and / or the additional database. In this embodiment, the control unit 138 determines that the orange juice 210 contains 30 mg of vitamin C in 100 ml of liquid 210. In another embodiment, the commercially packaged drinking vessel could also be a Coca-Cola bottle, a Red Bull can, a beer bottle, or a fruit juice carton of another brand. Based on the data in the database and / or the additional database, the control unit 138 can determine the vitamin C content of 75 mg in the 250 ml of orange juice 210 consumed by the passenger 206.

[0058] In the same manner as described for determining the at least one ingredient and the quantity of the ingredient, the control unit 138 can alternatively or additionally determine the calorific value of the orange juice 210 contained in the orange juice bottle 208. Furthermore, the control unit can also determine the calorific value of 112.5 kcal for the 250 ml of orange juice 210 consumed by the passenger 206 in the same manner.

[0059] Furthermore, the control unit 138 can add the volume of fluid consumed by the passenger 206 to her total fluid intake since the start of the journey. Based on this total fluid intake and the journey time, the control unit 138 can then calculate a fluid intake per unit of time, e.g., per minute, and compare this fluid intake per minute with a preset, stored first limit value. In one embodiment, the first limit value could correspond to a theoretical fluid requirement per minute for an adult, which could be determined, for example, from a daily requirement of 2 liters at an ambient temperature of 20°C for the passenger 206 and an average waking time of 16 hours. Thus, the first limit value could, for example, be 2.1 ml / min.

[0060] Subsequently, if the first threshold is not met, the control unit 138 can issue a drinking recommendation to the passenger 206 using the output units 108, 110, 112, 116. With a drinking volume of 250 ml and a driving time of 30 minutes until the first consumption event, the drinking volume per minute for the passenger 206 corresponds to 8.3 ml / min. The control unit 138 compares the drinking volume per unit of time of 8.3 ml / min with the first threshold of 2.1 ml / min and does not issue a drinking recommendation to the passenger 206, as the first threshold is not met.

[0061] The control unit 138 can further add the vitamin C content related to the passenger's fluid intake to the passenger's total vitamin C intake since the start of the journey. Based on the total vitamin C intake and the journey time since the start of the journey, the control unit 138 can determine a vitamin C intake per unit of time, e.g., per minute, and compare this vitamin C intake per minute with a preset, stored second limit value. In one embodiment, the second limit value can correspond to a theoretical vitamin C requirement per minute for an adult, which can be determined, for example, from a daily requirement of 100 mg at the passenger's ambient temperature of 20°C and an average waking time of 16 hours. Thus, the second limit value could, for example, be 0.10 mg / min.

[0062] Subsequently, if the second threshold is not met, the control unit 138 can issue a vitamin C intake recommendation to the passenger 206 using the output units 108, 110, 112, and 116. With an intake of 75 mg and a driving time of 30 minutes until the first consumption, the vitamin C intake corresponds to 2.5 mg / min. The control unit 138 compares the intake of 2.5 mg / min with the second threshold of 0.10 mg / min and does not issue a vitamin C intake recommendation to the passenger 206, as the second threshold is not met.

[0063] Furthermore, the control unit 138 can add the calorific value related to the passenger's (206) fluid intake to the passenger's total calorie intake since the start of the journey. Based on the total calorie intake and the journey time since the start of the journey, the control unit 138 can determine a calorie intake per unit of time, e.g., per minute, and compare this calorie intake per minute with a preset, stored third limit value. In one embodiment, the third limit value can correspond to a theoretical calorie requirement per minute for an adult, which can be determined, for example, from a daily requirement of 2000 kcal at an ambient temperature of 20°C for the passenger 206 and an average waking time of 16 hours. Thus, the third limit value could, for example, be 2.1 kcal / min.

[0064] Subsequently, if the third threshold is not met, the control unit 138 can issue a calorie intake recommendation to the passenger 206 using the output units 108, 110, 112, and 116. With a calorie intake of 112.5 kcal and a driving time of 30 minutes until the first consumption event, the calorie intake per minute for the passenger 206 corresponds to 3.8 kcal / min. The control unit 138 compares the calorie intake per unit of time of 3.8 kcal / min with the third threshold of 2.1 kcal / min and does not issue a calorie intake recommendation to the passenger 206, as the third threshold is not met.

[0065] In one embodiment, the control unit 138 can be located in a mobile unit or a central processing device. The mobile unit could, for example, be a smartphone or smartwatch belonging to the passenger 206. The central processing device could, for example, be a data center belonging to a nutrition counseling service provider belonging to the passenger 206. The device then communicates data wirelessly with the control unit 138 using the communication unit 144.

[0066] The control unit 138 can also output information regarding the classification of the drinking vessel 208 as an orange juice bottle 208 via the output units 108, 110, 112, 116. The input units 117, 126 then register either a confirmation or a correction of this classification. In this embodiment, the passenger 206 confirms the classification of the drinking vessel 208 as an orange juice bottle 208. In another embodiment, the passenger 206 drinks tea from a thermos, but the device has classified the drinking vessel as a thermos containing coffee. The control unit 138 then outputs information regarding the classification of the drinking vessel as a thermos containing coffee via the output units 108, 110, 112, 116. The passenger in vehicle number 206 then notices that the classification of the drinking vessel is incorrect.The input unit 117, 126 is used by the passenger 206 to record a correction to the information regarding the classification as a thermos flask containing coffee. Based on this correction, the control unit 138 then adjusts the classification of the drinking vessel to a thermos flask containing tea.

[0067] The control unit 138 can further adjust the first and / or the second and / or the third limit value depending on the determined ambient temperature. Alternatively or additionally, the control unit 138 can also adjust the first and / or the second and / or the third limit value depending on the passenger's gender, age, a different average waking time, weight, individual settings and / or fluids 210 and / or solid food 232 already consumed.

[0068] Fig. Figure 3 shows a second image 300 taken using the interior camera 134, depicting a second consumption situation.

[0069] The second consumption situation takes place 30 minutes after the one in Fig. The first consumption situation depicted in section 2 takes place, i.e., 60 minutes after the journey begins. After the Fig. In the drinking process shown in Figure 2, the passenger 206 put the orange juice bottle 208 down again, so that the orange juice bottle 208 is now back in the holder 228 on the passenger 206's side. The orange juice bottle 208 now contains less orange juice 210 than in Figure 2. Fig. Figure 2 shows the weight of the orange juice bottle 208, which is detected by the weight sensor 234 of the holder 228. The holder 230 on the driver's side 104 is empty because the driver 104 has removed the croissant 232 from the holder 230 and is holding it. The driver 104 is eating the croissant 232, so his mouth 214 is closed and his cheeks 224 are bulging outwards. His swallowing muscles and larynx 220 are moving. The weight of the croissant 232 was detected by the weight sensor 236 of the holder 230 on the driver's side 104 before the croissant 232 was removed, including during the initial consumption. The driver 104 has not consumed anything since the start of the journey, and the temperature measuring unit 150 determines the ambient temperature to be 20°C.

[0070] The interior camera 134 generates image data corresponding to image 300, which is processed by the control unit 138. Based on the image data, the control unit 138 detects the croissant 232 and the eating action of the driver 104, who is eating the croissant 232.

[0071] The eating process is detected, for example, by the control unit 138 processing the image data using a computer vision method. This allows specific arrangements of body parts to be assigned to the action "eating process." For example, the position of the driver's larynx 220 during swallowing and the shape of the mouth 214 and cheeks 224 during chewing can be assigned to the action "eating process." After eating, the driver 104 places the croissant 232 back in the holder 230 on the driver's side. Placing the croissant 232 is in Fig. 3 not shown.

[0072] The control unit 138 then determines, based on the image data and / or based on the weight information obtained using the weight sensor 236 in the holder 230 on the side of the driver 104, a quantity of food that corresponds to the quantity of croissant 232 consumed by the driver 104 during the eating process.

[0073] For this purpose, the control unit 138 can determine the number of swallowing movements during the eating process based on the image data. Furthermore, the control unit 138 can determine the weight of the solid food 232 before and after eating based on the weight information. Based on the determined number of swallowing movements during eating and / or based on the determined weight of the solid food 232 before and after eating, the control unit 138 can then determine the amount eaten. In this embodiment, the control unit 138 determines the amount eaten to be 20 g.

[0074] Furthermore, based on the image data and database data, the control unit 138 classifies the detected solid food item 232 as a croissant 232. Additionally, based on this classification as a croissant 232 and data from the database and / or the additional database, the control unit 138 determines at least one ingredient and the quantity of that ingredient in the croissant 232. In this embodiment as well, the at least one ingredient is vitamin C. The database and / or the additional database contains data on the vitamin C content in a specific quantity of the croissant 232. Thus, in this embodiment, the control unit 138 determines, for example, that 0.316 mg of vitamin C are contained in 100 g of croissant 232. Based on the data from the database and / or the further database, the control unit 138 can then determine the vitamin C content of 0.063 mg in the 20 g of croissant 232 consumed by the driver 104.

[0075] In the same manner as described for determining the minimum ingredient and the quantity of the ingredient, the control unit 138 can alternatively or additionally determine the calorific value of the croissant 232. Furthermore, the control unit can also determine the calorific value of 102 kcal for the 20 g of croissant 232 consumed by the driver 104 in the same manner.

[0076] In another embodiment, the detected solid food is a ready-made salad, and the database and / or the additional database also contain data on the external appearance, calorific value, and ingredients, such as vitamin C, of ​​solid foods in commercial packaging, such as the ready-made salad. Thus, the design of a ready-made salad package can be associated with a brand, allowing the calorific value and vitamin C content of the salad in the commercial packaging of that brand to be determined based on the data in the database and / or the additional database. The solid food in the commercial packaging could also be a sandwich or a confectionery item.

[0077] Furthermore, the control unit 138, as in conjunction with Fig. As described in section 2, the system compares the drinking volume per minute with the first limit value and, if the volume falls below the first limit value, issues a drinking recommendation to the driver 104. Since the driver 104 has not drunk anything since the start of the journey, the drinking volume per unit of time for the driver 104 is 0 ml / min. The control unit 138 compares the drinking volume per unit of time (0 ml / min) with the first limit value (2.1 ml / min) and issues the drinking recommendation to the driver 104, as the first limit value has been undercut.

[0078] Control unit 138 can continue to add the vitamin C content related to the amount of food consumed to the total vitamin C intake of the vehicle occupant since the start of the journey. Since the driver 104 has not consumed anything since the start of the journey, the vitamin C content related to the amount of food consumed corresponds to the total vitamin C intake. Control unit 138 can, as in conjunction with Fig. As described in section 2, the control unit compares the vitamin C intake per minute with the second limit value and, if the second limit value is not reached, issues the vitamin C intake recommendation to the driver 104. With an intake of 0.063 mg and a driving time of 60 minutes until the second consumption event, the driver's vitamin C intake per minute corresponds to 1.05 µg / min. The control unit 138 compares the intake per unit of time of 1.05 µg / min with the second limit value of 0.10 mg / min and issues the vitamin C intake recommendation to the driver 104, since the second limit value is not reached.

[0079] In the same manner as described for determining the total vitamin C intake, the control unit 138 can alternatively or additionally add the calorific value related to the food quantity to the total calorie intake of the vehicle occupant since the start of the journey. Furthermore, the control unit can also determine a calorie intake per minute of the driver 104 of 1.7 kcal / min in the same way. The control unit 138 compares the calorie intake per unit of time of 1.7 kcal / min with the third limit value of 2.1 kcal / min and outputs the calorie intake recommendation to the driver 104, since the third limit value is not exceeded.

[0080] The control unit 138 can also output information regarding the classification of the solid food 232 as a croissant 232 to the output unit 108, 110, 112, 116. The input unit 117, 126 then records confirmation or correction of this classification. In this embodiment, the driver 104 confirms the classification of the solid food 232 as a croissant 232. In another embodiment, the driver 104 eats a chocolate croissant, but the device has classified the solid food 232 as a regular croissant. The control unit 138 then outputs information regarding the classification of the solid food 232 as a croissant to the output unit 108, 110, 112, 116. The driver 104 then notices that the classification of the solid foodstuff 232 is incorrect.The driver 104 uses input unit 117, 126 to enter a correction to the information regarding the classification as a croissant. Based on this correction, control unit 138 then adjusts the classification of the solid food item to a chocolate croissant.

[0081] As in connection with Fig. As described in section 2, the control unit 138 can further adjust the first and / or the second and / or the third limit value depending on the determined ambient temperature and the other factors already mentioned in the context.

[0082] Fig. Figure 4 shows a procedure for monitoring the fluid intake of a vehicle occupant 104, 206.

[0083] The process starts in step S100. Subsequently, in step S102, an interior camera 134 captures at least one image of an area of ​​the vehicle interior encompassing at least part of a vehicle seat 106, 202, and corresponding image data is generated. In step S104, the control unit 138 processes the image data and, based on this, detects a drinking vessel 208 and a drinking action by the vehicle occupant 104, 206. In step S106, the control unit 138 then determines, based on the image data and / or weight information related to the weight of the drinking vessel 208, a drinking volume that corresponds to the volume of liquid 210 consumed by the vehicle occupant 104, 208 during the drinking action. The process then ends in step S108.

[0084] In the based on the Fig. 1, Fig. 2, Fig. 3 to Fig. In the embodiments described in Figure 4, at least the interior camera 134 in combination with the control unit 138 forms a device for monitoring the fluid intake of a vehicle occupant 104, 206. The control unit 138 is also referred to as the processing unit. Further embodiments described in Figure 4 are described in Figure 4. Fig. 1, Fig. 2, Fig. 3 to Fig. The four elements and features shown and mentioned in the preceding description can be part of the device. Likewise, process steps described with reference to the device can be part of the claimed method. 100 Cockpit 102 vehicles 104 drivers 106 Driver's seat 108 Central Information Display 110 Head Up Display 112 Graphic instrument cluster 114 Dashboard 116 speakers 117 Microphone 118 Steering wheel 120 Control element 122 Gear selector 124 Pedals 126 Input unit 128 Windscreen 132 Interior rearview mirror 134 Interior camera 138 Control unit 140 Data output 142 Data input 144 Communication module 150 temperature measuring units 200 images 202 Passenger seat 206 female passenger 208 Water bottle 210 liquid 212, 214 Mouth 216 Label 218, 220 Larynx 222, 224 Wangen 226 Center console 228, 230 bracket 232 Solid food 234, 236 Weight sensor 300 images S100 to S108 Procedure steps QUOTES INCLUDED IN THE DESCRIPTION

[0000] This list of documents cited by the applicant was automatically generated and is included solely for the reader's convenience. The list is not part of the German patent or utility model application. The DPMA accepts no liability for any errors or omissions. Cited patent literature

[0000] DE 10 2014 203 204 A1

[0003] DE 10 2022 125 658 A1

[0004] JP 2010-186251 A

[0005]

Claims

[1] Device for monitoring the fluid intake of a vehicle occupant (104, 206), comprising at least one interior camera (134) of the vehicle (102) configured to capture at least one image of an area of ​​the vehicle interior encompassing at least part of a vehicle seat (106, 202) and to generate image data corresponding to the image, comprising a processing unit (138) configured to process the image data and, based on the image data, to detect a drinking vessel (208) and to detect a drinking process by the vehicle occupant (104, 206) from the drinking vessel (208), and wherein the processing unit (138) is configured to determine a drinking volume based on the image data and / or based on weight information obtained using a weight sensor relating to the weight of the drinking vessel (208). [2] Device according to claim 1, wherein the processing unit (138) is configured to determine, starting from the image data, at least one of the following parameters - a drinking vessel volume and a change in the angle of inclination of the drinking vessel (208) during the drinking process, and / or - a number of swallowing movements of the vehicle occupant (104, 206) during the drinking process, and / or - the duration of the drinking vessel (208) at the mouth (212, 214) of the vehicle occupant (104, 206) during the drinking process and / or - to determine the drinking vessel volume and a fill level of the drinking vessel (208) before and a fill level of the drinking vessel (208) after the drinking process, and / or, based on the weight information, at least the following parameters - to determine the weight of the drinking vessel (208) before and after the drinking process, and to determine the drinking volume based on at least one of the parameters determined from the image data and / or the weight information. [3] Device according to claim 1 or 2, wherein the device comprises a communication unit (144), wherein the communication unit (144) is designed to communicate data with a database, wherein the processing unit (138) is configured to classify the detected drinking vessel (208) based on the image data and data stored in the database, and based on a classification of the drinking vessel (208) and data from the database and / or another database, to determine at least one ingredient and the quantity of the ingredient of the liquid (210) contained in the drinking vessel (208) and / or wherein the processing unit (138) is configured to classify the detected drinking vessel (208) based on the image data and data stored in the database, and to determine a calorific value of the liquid (210) contained in the drinking vessel (208) based on a classification of the drinking vessel (208) and data from the database and / or the further database, and wherein the processing unit (138) is configured to determine a quantity of the ingested ingredient related to the drinking volume and / or a calorific value related to the drinking volume. [4] Device according to one of claims 1 to 3, wherein the processing unit (138) is configured to add the drinking volume to a total drinking volume of the vehicle occupant (104, 206) since the start of the journey, and wherein the processing unit (138) is configured to determine a drinking volume per unit of time based on the total drinking volume and a driving time since the start of the journey and to compare this drinking volume per unit of time with a preset stored first limit value, and with an output unit (108, 110, 112, 116) which is designed to output information to the vehicle occupants (104, 206), wherein the processing unit (138) is designed to output a drinking recommendation to the vehicle occupants (104, 206) when the first limit value is undershot using the output unit (108, 110, 112, 116). [5] Device according to claim 3 or 4, wherein the processing unit (138) is configured to add the amount of the ingested ingredient related to the drinking volume to a total amount of the ingredient ingested by the vehicle occupant (104, 206) since the start of the journey, and wherein the processing unit (138) is configured to determine an ingredient intake quantity per unit of time based on the total intake quantity of the ingredient and a journey time since the start of the journey, and to compare this ingredient intake quantity per unit of time with a preset stored second limit value, and with an output unit (108, 110, 112, 116) which is designed to output information to the vehicle occupants (104, 206), wherein the processing unit (138) is designed to output an ingredient intake recommendation to the vehicle occupants (104, 206) using the output unit (108, 110, 112, 116) when the second limit value is undershot. [6] Device according to one of claims 3 to 5, wherein the processing unit (138) is configured to add the calorific value related to the drinking volume to a total calorie intake of the vehicle occupant (104, 206) since the start of the journey, wherein the processing unit (138) is configured to determine a calorie intake per unit of time based on the total calorie intake and a driving time since the start of the journey, and to compare this calorie intake per unit of time with a preset stored third limit value, and with an output unit (108, 110, 112, 116) which is configured to output information to the vehicle occupants (104, 206), wherein the processing unit (138) is configured to output a calorie intake recommendation to the vehicle occupants (104, 206) when the third limit value is undershot using the output unit (108, 110, 112, 116). [7] Device according to one of the preceding claims, wherein the processing unit (138) is arranged in a mobile unit or a central processing unit, and wherein the device communicates data wirelessly with the processing unit (138) using the communication unit (144). [8] Device according to one of claims 2 to 7, wherein the processing unit (138) is configured to detect a solid food item (232) and to detect an eating process of the vehicle occupant (104, 206) based on the image data, wherein the processing unit (138) is configured to determine, based on the image data and / or based on weight information obtained using the weight sensor and / or another weight sensor, a quantity of food consumed that corresponds to the quantity of solid food (232) consumed during the eating process by the vehicle occupant (104, 206), and wherein the processing unit (138) is configured to classify the detected solid food (232) based on the image data and data from the database, and based on a classification of the solid food (232) and data from the database and / or the further database, to determine at least one ingredient and the quantity of the ingredient in the solid food (232), and / or wherein the processing unit (138) is configured to classify the detected solid food (232) based on the image data and data from the database, and to determine a calorific value of the solid food (232) based on the classification of the solid food (232) and data from the database and / or the further database, and wherein the processing unit (138) is configured to determine a quantity of the ingested ingredient related to the amount of food consumed and / or a calorific value related to the amount of food consumed. [9] Device according to claim 8, wherein the processing unit (138) is configured to determine, based on the image data, a number of swallowing movements during the eating process, and / or, based on the weight information, to determine a weight of the solid food (232) before and a weight of the solid food (232) after the eating process, and wherein the processing unit (138) is further configured to determine the amount eaten based on the determined number of swallowing movements during the eating process and / or based on the determined weight of the solid food (232) before and after the eating process. [10] Device according to claim 8 or 9, wherein the processing unit (138) is configured to add the determined amount of the ingested ingredient related to the amount of food consumed to the total amount of the ingredient ingested by the vehicle occupant (104, 206) since the start of the journey. [11] Device according to one of claims 8 to 10, wherein the processing unit (138) is configured to add the calorific value related to the amount of food consumed to the total calorie intake of the vehicle occupant (104, 206) since the start of the journey. [12] Device according to one of claims 3 to 11, wherein the processing unit (138) is configured to output information on the classification of the drinking vessel (208) to the output unit (108, 110, 112, 116), and wherein the processing unit (138) is further configured to receive confirmation of the information or a correction of the information relating to the classification of the drinking vessel (208) from an input unit (117, 126), and wherein the processing unit (138) is configured to adapt the classification of the drinking vessel (208) based on the correction of the information to the classification of the drinking vessel (208), and / or wherein the processing unit (138) is configured to output information on the classification of the solid foodstuff (232) to the output unit (108, 110, 112, 116), and wherein the processing unit (138) is further configured to receive confirmation or correction of the information relating to the classification of the solid foodstuff (232) from the input unit (117, 126), and wherein the processing unit (138) is designed to adapt the classification of the solid food (232) based on the correction of the information to the classification of the solid food (232). [13] Device according to one of the preceding claims, wherein the drinking vessel (208) contains a liquid (210) and the fill level of the drinking vessel (208) is determined using an IR camera unit, and / or wherein the drinking vessel volume is determined using a TOF camera unit. [14] Device according to one of claims 4 to 13, comprising a temperature measuring unit configured to determine the ambient temperature of the vehicle occupant (104, 206), and wherein the processing unit (138) is configured to adjust the first and / or the second and / or the third limit value depending on the determined ambient temperature. [15] Method for monitoring the fluid intake of a vehicle occupant (104, 206), in which at least one image is captured using an interior camera (134) depicting an area of ​​the vehicle interior encompassing at least part of a vehicle seat (106, 202) and corresponding image data is generated from the image, in which the image data is processed using a processing unit (138) and a drinking vessel (208) is detected based on the image data and a drinking process of the vehicle occupant (104, 206) is detected, and in which, using the processing unit (138), a drinking volume corresponding to the volume of the liquid (210) consumed during the drinking process by the vehicle occupant (104, 206) is determined based on the image data and / or weight information relating to the weight of the drinking vessel.

Images