Method for controlling a function of a vehicle using the occupant profiles and their relationship to one another

Abstract

A computer-implemented method (100) for controlling a function of a vehicle comprises a step of receiving (102) sensor data from at least one sensor (104) of the vehicle. The at least one occupant of the vehicle is identified on the basis of the received sensor data and assigned a position (122) inside the vehicle. A further step of the method involves obtaining (128) a setting (130) that is associated with the at least one occupant and related to the function of the vehicle. In addition, the position (122) and the setting (130) are stored in a database (136) as an occupant profile (138, 138a, 138b, 138c) such that it is possible to assign a position (122) and a setting (130) to the at least one occupant. Relationships are determined (144) between the occupants if a plurality of occupant profiles (138, 138a, 138b, 138c) are stored in the database (136), and the function of the vehicle is executed using the occupant profiles (138, 138a, 138b, 138c) and their relationship (144) to one another.

Description

[0001] Method for controlling a function of a vehicle using the occupant profiles and their relationship to one another

[0002] Technical area

[0003] The present invention relates to the field of vehicle control. In particular, the present invention relates to a computer-implemented method for controlling a function of a vehicle, as well as a corresponding data processing device, a corresponding computer program product, a corresponding computer-readable storage medium, and a corresponding data carrier signal.

[0004] Technical background

[0005] Today, vehicle passengers can be identified through methods such as seat occupancy checks, seatbelt status, or key fob detection. This is sufficient for use cases such as seat belt reminders or automatically loading the profile of the driver and other vehicle key holders.

[0006] However, current systems are unable to meet the needs of multiple passengers simultaneously seeking a collaborative, personalized user experience. For example, if the system is limited to global logins to media accounts such as streaming services, it's impossible to allow each passenger to use their own account with personalized playlists from the seat they currently occupy. Furthermore, these systems do not take into account the preferences of all passengers or their relationships with each other.

[0007] Disclosure of the invention

[0008] It is therefore the object of the invention to provide a computer-implemented method for controlling a function of a vehicle that eliminates the aforementioned disadvantages. Furthermore, it is the object of the invention to provide a corresponding data processing device, a computer program product, a computer-readable storage medium, and a data carrier signal.

[0009] According to a first aspect of the invention, a computer-implemented method for controlling a function of a vehicle comprises a receiving step in which sensor data is received from at least one sensor of the vehicle.

[0010] The sensors can be, for example, an interior camera, a microphone, a pressure sensor of a seat occupancy system, or a smart device scanner (typically a device for detecting and locating a mobile, internet-connected device). The sensor data corresponding to these sensors can then include, for example, image data, audio data, a binary electronic signal indicating seat occupancy, and a bidirectional data connection to the smart device.

[0011] According to the first aspect of the invention, the computer-implemented method for controlling a function of a vehicle comprises an identification step in which at least one occupant of the vehicle is identified on the basis of the received sensor data.

[0012] For example, image data can be analyzed using artificial intelligence in the form of facial recognition to identify the people in the image. Similarly, audio data can be used to identify the people in the vehicle. If personalized data such as the occupant's weight or smart device ID is stored in a database, identification can also be performed using sensor data from the seat occupancy system or the smart device scanner by accessing this database.

[0013] According to the first aspect of the invention, the computer-implemented method for controlling a function of a vehicle comprises an assignment step in which a position within the vehicle is assigned to each identified occupant based on the received sensor data. Such an assignment is very simple in the case of image data, because the information on the occupant's position is directly accessible in the image data. If one wishes to determine the position of an occupant using audio data, this can be done, for example, using the volume information in the audio data. By identifying the occupants and locating their positions within the vehicle, certain knowledge about all occupants of the vehicle is gained over time.

[0014] According to the first aspect of the invention, the computer-implemented method for controlling a function of a vehicle comprises a step in which a setting associated with the function of the vehicle is obtained from the at least one occupant.

[0015] The setting can be entered directly by the user via an input device such as an infotainment system or received from a smart device. These settings can include, for example, music ratings, seat height adjustments, temperature settings, or even security settings such as access permissions for data on a smart device.

[0016] According to the first aspect of the invention, the computer-implemented method for controlling a function of a vehicle comprises a storage step in which the position and setting are stored in a database as an occupant profile in such a way that an assignment of position and setting to the at least one occupant is possible.

[0017] This can be done, for example, using a data structure in which an identifier is assigned to an occupant. In addition to the identifier, the data structure also contains the position and the setting. This makes it possible to record the position over time and save multiple settings for various functions in the occupant profile. If an occupant is detected whose occupant profile is already stored in the database, the current position and new settings can be added to the occupant profile, or existing settings can be modified.

[0018] In this way, it is possible to create a database with occupant profiles of different occupants. These occupants can be people who know each other, for example, if the vehicle is only used within a family. However, the vehicle can also be a rental vehicle or a vehicle shared between many people. The database can, for example, be decentralized to form a central collection point for occupant profiles. This is advantageous because the occupant profiles of occupants who have never previously traveled together in a vehicle, but in another vehicle in which the procedure is carried out, are immediately available without the need to re-enter the occupant settings.

[0019] According to the first aspect of the invention, the computer-implemented method for controlling a function of a vehicle comprises a step in which the relationships between occupants are determined if several occupant profiles are stored in the database.

[0020] If the current occupants of a vehicle have a close relationship with one another, for example because they are all members of the same family, this can be stored in the occupant profiles. If the occupants are strangers to each other, for example because they only happen to take the same taxi, this is also stored in the occupant profiles. The relationships can be determined, for example, via a phone book on a smart device in which a relationship is already stored. Alternatively, a relationship can also be determined, for example, via user input on an infotainment system and stored in the data structure of the occupant profile as a binary value (trusted / untrusted). Of course, a value that maps the relationship between occupants does not have to be binary, but can represent a whole spectrum between the previously mentioned values.According to the first aspect of the invention, the computer-implemented method for controlling a function of a vehicle comprises a step in which the function is executed using the occupant profiles and their relationship to one another.

[0021] In other words, a vehicle function retrieves the occupant profiles stored in the database before being executed and executes the function based on the contents of the occupant profiles. If the current occupants of a vehicle have a close relationship with each other, a function that displays images stored on one occupant's smart device on a screen in front of another occupant can be permitted. However, if the occupants are strangers, the display is blocked.

[0022] Furthermore, messages could only be read aloud if all vehicle occupants trust each other. If an unknown occupant (or an occupant deemed untrustworthy) is in the vehicle, only a notification, for example, via a screen, would be displayed indicating that a new message has arrived. Personal, particularly health-related, alerts, such as high blood sugar or heart rate, could also be controlled based on the relationships among the occupants.

[0023] If the occupants are already registered in the database, they can be served in a personalized manner. All unregistered occupants can be recorded in the database using this process, and their occupant profile can then be updated over time. Identification can be done visually, for example, using camera-based facial recognition or by analyzing the voice of a speaking occupant. Furthermore, an occupant can be identified using their mobile device or vehicle key fob.

[0024] A vehicle's audio system can calculate the position of occupants by analyzing microphone energy levels or, in the case of an installed microphone array, by calculating the direction of arrival information. Furthermore, seat occupancy status and connected mobile devices can be used to identify and locate occupants, for example, when no other options such as a camera or microphone are available.

[0025] According to an advantageous embodiment of the computer-implemented method for controlling a function of a vehicle, the function of the vehicle relates to one of the following functions: data exchange, media playback, music playback, video playback, access authorizations, user account management, statistical evaluation, control of the interior lighting, seat adjustment, temperature adjustment or occupant warning.

[0026] According to an advantageous embodiment of the computer-implemented method for controlling a function of a vehicle, the at least one sensor is configured to determine at least one of the following measured variables: electromagnetic radiation, sound, pressure, or an electrical signal. Some examples of suitable sensors are a camera, microphone, smart device scanner, push buttons, levers, or switches.

[0027] According to an advantageous embodiment of the computer-implemented method for controlling a function of a vehicle, the setting relates to one of the following settings: privacy, relationship, music rating, video rating, film rating, media rating, brightness setting for a light in the vehicle interior, user name and password, temperature or seat setting.

[0028] According to an advantageous embodiment of the computer-implemented method for controlling a function of a vehicle, the relationship is determined by taking into account the occupants' telephone book. This is because the telephone books of modern smart devices already contain relationship information that can be used immediately. According to an advantageous embodiment of the computer-implemented method for controlling a function of a vehicle, the vehicle's function corresponds to the provision of a music playlist that corresponds to the musical tastes of all occupants.

[0029] According to an advantageous embodiment of the computer-implemented method for controlling a function of a vehicle, the function of the vehicle corresponds to a registration for a media account. This allows a specific user to access their personal media, such as music, videos, etc., from their seat using the devices available there, such as displays and the like.

[0030] According to an advantageous embodiment of the computer-implemented method for controlling a function of a vehicle, the function of the vehicle corresponds to a limitation of the vehicle's speed or power. Thus, a lower speed limit can be specified for inexperienced drivers than for experienced drivers.

[0031] According to a second aspect of the invention, a computer program product comprises instructions which, when the program is executed by a computer, cause the computer to execute the computer-implemented method according to the first aspect of the invention or one of the advantageous embodiments of the first aspect.

[0032] According to a third aspect of the invention, a computer-readable storage medium comprises instructions which, when executed by a computer, cause the computer to carry out the method according to the first aspect of the invention or one of the advantageous embodiments of the first aspect.

[0033] According to a fourth aspect of the invention, a data carrier signal transmits the computer program product of the second aspect of the invention.

[0034] According to a fifth aspect, a data processing device for controlling a function of a vehicle is designed to carry out the method according to the first aspect of the invention or one of the advantageous embodiments of the first aspect.

[0035] The data processing device comprises a processor configured to execute the steps of the method according to the first aspect of the invention or one of the advantageous embodiments of the first aspect. Furthermore, the data processing device comprises a computer-readable storage medium according to the third aspect of the invention, which is communicatively connected to the processor. Furthermore, the data processing device comprises at least one sensor for generating sensor data and an input device for entering a setting relating to a function of the vehicle, which are communicatively connected to the processor.

[0036] The various aspects of the invention overcome the limitations of prior art systems. If the passengers sitting together in the vehicle are known, the inventive method can offer personalization to each passenger. If the relationship between the passengers is also known, the method can, for example, mix preferences and playlists to provide a pleasant experience for everyone traveling together. By taking into account the relationships between the passengers, which are stored in passenger profiles, it can be checked whether the passengers trust each other or whether they want to mix or share their preferences when they are not alone in the vehicle.

[0037] For example, you can set the automatic selection of the privacy mode, load a preferred music source or playlist, create a playlist based on all occupant tastes, set a driving mode (e.g., Sport, Eco, or Comfort), or log in to a personal account. For example, the right-hand seat will log in to the exact media account that the occupant sitting there also has an account with. Depending on the detected occupants sitting together in the vehicle, where each person is located, and their relationship to one another, a personalized or shared user experience can be offered. Furthermore, the database offers the VehicleZ fleet operator the ability to automatically create logbooks or vehicle usage statistics, including information on who traveled together and who was the driver.This will help better understand how vehicles are used in the field (how many passengers, age, which functions are used by which target group, etc.). Decisions can then be made based on this data, for example regarding marketing. Occupants themselves can also use this data, for example to relive their personal musical highlights of the past year, their personal fuel consumption, or the distance traveled over a specific time interval (e.g., week or month). The data can also be shared on an online service such as a social network. Furthermore, in the event of an emergency, emergency services could automatically be provided with further information about the occupants.

[0038] Summary of the characters

[0039] The invention is explained in more detail below using exemplary embodiments and figures. The figures show:

[0040] Figure 1 : A flowchart of the computer-implemented method for

[0041] Control of a function of a vehicle;

[0042] Figure 2: A schematic view of the data processing device for

[0043] Control of a function of a vehicle; and

[0044] Figure 3: An embodiment of the computer-implemented method for controlling a function of a vehicle.

[0045] Detailed description of the characters

[0046] Figure 1 shows a flowchart of the computer-implemented method 100 for controlling a function of a vehicle. In a first step of the computer-implemented method 100 for controlling a function of a vehicle, receiving step 102, sensor data is received from at least one sensor 104 (see Figs. 2 and 3) of the vehicle. The sensor 104 can be, for example, a microphone 106 (see Fig. 3), an interior camera 108 (see Fig. 3), or a smart device scanner 110 (typically a device for detecting and locating a mobile device connected to the Internet, see Fig. 3). The sensor data corresponding to these sensors 104 are then, for example, image data 112 (see Fig. 3), sound data 114 (see Fig. 3), and a bidirectional data connection to the smart device.

[0047] In the next step, identification step 116, the occupants are identified based on the received sensor data. Identification can be performed visually, for example, using camera-based facial recognition 118 (see Fig. 3) or by analyzing the voice of a speaking occupant based on voice biometrics. Furthermore, an occupant can be identified using their mobile device or the vehicle key.

[0048] In a further step, assignment step 120, a position 122 (see Fig. 3) within the vehicle is assigned to each identified occupant based on the received sensor data. For example, a vehicle audio system can calculate the occupant's position 122 by analyzing the microphone energy levels 174 (see Fig. 3) or, in the case of an installed microphone array, by calculating the direction of arrival information. Furthermore, the weight 124 of a seat occupant 126 and connected mobile devices can be used to identify and locate occupants, for example, when no other options such as interior camera 108 or microphone 106 are available.

[0049] To ensure correct position assignment and identification of the occupants, the receiving step 102 of the method 100 is carried out continuously. This is visually represented in Fig. 1 by the circular arrow at the receiving step 102. In a further step, receiving step 128, settings 130 relating to the function of the vehicle are received from at least one occupant. The setting 130 can, for example, be entered directly by the occupant via an input device 132 (Figs. 2 and 3) such as an infotainment system or can be received from a smart device, for example. The settings 130 can, for example, be the rating of pieces of music, a seat setting (e.g., the seat height or the orientation of the backrest), a temperature setting, or even security settings relating to access authorizations for data on a smart device.

[0050] In a storage step 134, the determined position 122 and received settings 130 are stored in a database 136 (see Fig. 3) as an occupant profile 138 (see Fig. 3) such that an assignment of positions 122 and settings 130 to the at least one occupant is possible. For example, it is possible to store the occupant profile 138 as a data structure in the database 136, wherein the data fields position 122, settings 130, and an identifier 140 (see Fig. 3) are provided.

[0051] In a determination step 142, if multiple occupant profiles 138 are stored in the database 136, relationships 144 (see Fig. 3) between the occupants are determined. For example, family members maintain a close relationship 144 with each other, while strangers in a taxi typically do not maintain a close relationship 144 with each other. The relationship 144 can, for example, be entered manually via an input device 132 of the vehicle, such as an infotainment system. Alternatively, it is also possible, if available, to determine this information via a phone book 148 (see Fig. 3) of the occupants' smart devices.

[0052] In an execution step 150, the vehicle's function is executed, taking into account the occupant profiles 138 and their relationships 144 to one another. The relationship 144 between different occupants can, if necessary, be stored in the occupant profiles 138 of the occupants involved.

[0053] The database 136 now contains information about the occupants, their position 122 in the car, their preferences or settings 130 regarding certain vehicle functions, and their relationship 144 to one another. Using this information, it is now possible to provide personalized offers, such as automatic login to a media account at the correct location in the vehicle. If a particular occupant sits in the front passenger seat, they can use their own music playlists 152 (see Fig. 3) right there. If they sit in a rear seat on the next trip, the method 100 enables them to access this content there as well.

[0054] Knowing the relationship 144 between the occupants makes it possible to take preferences into account and therefore mix playlists 152 to provide a pleasant experience for all occupants traveling together. By considering the relationships 144 between the occupants stored in occupant profiles 138, it is possible to check whether the occupants trust each other and select a policy for data exchange between the occupants accordingly.

[0055] Figure 2 shows a schematic view of the device 154 for data processing for controlling a function of a vehicle.

[0056] The data processing device 154 has a processor 156, which is communicatively connected to a computer-readable storage medium 158, at least one sensor 104, an input device 132, and a control unit 160. Commands are stored on the computer-readable storage medium 158, which allow the processor 156 to execute the steps of the method 100 from Fig. 1. The sensor data from the sensors 104 are available as input for this method 100. The input device 132 serves to input settings 130 relating to the vehicle function affected by the method 100. For example, the input device 132 can correspond to an infotainment system, via which settings 130 relating to the seat height, the relationship 144 of other occupants, or musical taste can be entered.

[0057] The control unit 160 serves to execute the vehicle function affected by the method 100. This function may, for example, consist of creating a music playlist 152 that takes into account the musical tastes of all occupants. The control unit 160 may be a separate device. However, it is also possible for the control unit 160 to be part of another device. For example, it is possible for the input device 132 and the control unit 160 to be configured as part of an infotainment system.

[0058] Figure 3 shows an embodiment of the computer-implemented method 100 for controlling a function of a vehicle.

[0059] In the example of Fig. 3, a plurality of sensors 104 are provided to serve as the basis for the method 100 of Fig. 1. Specifically, a microphone 106, an interior camera 108, a smart device scanner 110, a seat occupancy system 126, and a seat belt status system 164 are provided.

[0060] Each of the sensors 104 generates its own sensor data. The microphone 106 generates audio data 114, the interior camera 108 generates image data 112, the smart device scanner 110 provides access to a phone book 148, an assignment to an occupant, and a smartphone's media library 166. The seat occupancy system 126 provides the weight 124 resting on the seats, and the seat belt status system 164 provides a status 170 regarding which seat belts are fastened.

[0061] The audio data 114 from microphone 106 can be used to identify an occupant using speaker recognition 172 based on voice biometrics. Furthermore, the audio data 114 and microphone energy levels 174 can be used to locate the occupants.

[0062] Similarly, occupants can be identified and located in the vehicle from the image data 112 of the interior camera 108 using known image processing methods such as facial recognition 118 and object detection 178.

[0063] The sensor data is then used to generate an occupant profile 138. This includes a unique identifier 140, a position 122 within the vehicle, preferences 130 regarding musical taste, and information about the relationship 144 to other occupant profiles 138a, 138b, 138c. Preferably, the identifier 140 is created using occupant-specific information. For example, the identifier 140 is created based on key features in facial recognition 118 or speaker recognition 172.

[0064] In the example of Fig. 3, the information about the relationship 144 is read from the phone book 148 and the setting 130 regarding music taste is read from the media library 166 of the smartphone using the smart device scanner 110. The position 122 within the vehicle is determined by comparing the various position information from the sensors 104.

[0065] In the example of Fig. 3, it is also possible to define the relationship 144 with other occupants via an input device 132. This can be done, for example, via a corresponding menu of an infotainment system. Similarly, it is possible to specify the setting 130 relating to the vehicle's function—in the example of Fig. 3, information about musical taste. An infotainment system can also provide a corresponding menu for this purpose, for example.

[0066] The interest profile 138 is stored in a database 136, which already contains other occupant profiles 138a, 138b, 138c. If a new setting 130 has been added to an already saved occupant profile 138, 138a, 138b, 138c, or an existing setting 130 has been changed, the occupant profile 138, 138a, 138b, 138c can be easily updated in the database 136. Since the identifier 140 was created with occupant-specific information, this prevents multiple occupant profiles 138, 138a, 138b, 138c from being created and saved for one occupant. Instead, an already stored occupant profile 138, 138a, 138b, 138c of a known occupant can be modified if the occupant is recognized.

[0067] A control unit 160, which is configured to create a music playlist 152 in such a way that the musical tastes of all occupants are taken into account, can access the database 136 containing the occupant profiles 138, 138a, 138b, 138c. The control unit 160 can be identical to the input device 132, but in the example of Fig. 3, it is a separate device. It creates a music playlist 152 using the information stored in the occupant profiles 138, 138a, 138b, 138c in the database 136.

[0068] An alternative function of the vehicle could be to display a slideshow of images from the smartphones on the screens in the vehicle. To do this, the control unit 160 analyzes the relationships 144 between the occupants and only displays images if a close relationship 144 exists. In other words, a trust level serves as the basis for privacy policy.

[0069] List of reference symbols

[0070] 100 procedures

[0071] 102 Reception step

[0072] 104 sensors

[0073] 106 Microphone

[0074] 108 Interior camera

[0075] 110 smart device scanners

[0076] 112 image data

[0077] 114 audio files

[0078] 116 Identification step

[0079] 118 Facial recognition

[0080] 120 Assignment step Position Weight Seat occupancy system Maintenance step Settings Input device Storage step Database Occupant profile a, 138b, 138c Other occupant profiles Identifier Determination step Relationships Input device Phone book Execution step Music playlist Device Processor Storage medium Control unit Seat belt status system Media library Status

[0081] Speaker recognition Microphone energy levels Object detection

Claims

Patent claims 1 . Computer-implemented method (100) for controlling a function of a vehicle, comprising the steps of: a) receiving (102) sensor data from at least one sensor (104) of the vehicle, b) identifying (116) at least one occupant of the vehicle based on the received sensor data, c) assigning (120) a position (122) within the vehicle to each identified occupant based on the received sensor data, d) obtaining (128) a setting (130) associated with the at least one occupant with respect to the function of the vehicle, e) storing (134) the position (122) and setting (130) in a database (136) as an occupant profile (138, 138a, 138b, 138c) such that an assignment of position (122) and setting (130) to the at least one occupant is possible, f) determining (142), if several Inmate profiles (138) are stored in the database (136), of relationships (144) between inmates,and g) execution (150) of the function using the occupant profiles (138), 138a, 138b, 138c) and their relationship (144) to one another., 2. Computer-implemented method according to claim 1, characterized in that the function of the vehicle relates to one of the following functions: data exchange, media playback, music playback, video playback, access authorizations, user account management, statistical analysis, control of the interior lighting, seat adjustment, temperature adjustment or occupant warning.

3. Computer-implemented method according to claim 1 or 2, characterized in that the at least one sensor (104) is designed to determine at least one of the following measured variables: electromagnetic radiation, sound, pressure, an electrical signal.

4. Computer-implemented method according to one of the preceding claims, characterized in that the setting (130) relates to one of the following settings: privacy, relationship, music rating, video rating, movie rating, media rating, brightness setting for a light in the vehicle interior, user name and password, temperature or seat setting.

5. Computer-implemented method according to one of the preceding claims, characterized in that the relationship (144) is determined taking into account a telephone book (148) of the occupants.

6. Computer-implemented method according to one of the preceding claims, characterized in that the function of the vehicle corresponds to providing a music playlist (152) corresponding to the musical tastes of all occupants.

7. Computer-implemented method according to one of the preceding claims, characterized in that the function of the vehicle corresponds to a registration to a media account.

8. Computer-implemented method according to one of the preceding claims, characterized in that the function of the vehicle corresponds to a limitation of the speed that can be driven by the vehicle or its power.

9. A computer program product comprising instructions which, when executed by a computer, cause the computer to carry out the computer-implemented method according to any one of the preceding claims.

10. A computer-readable storage medium (158) comprising instructions which, when executed by a computer, cause the computer to carry out the method according to one of claims 1 to 8.

11. A data carrier signal transmitting the computer program product according to claim 9.

12. A data processing device (154) for controlling a function of a vehicle, designed to carry out a computer-implemented method (100) according to one of claims 1 to 8, wherein the data processing device (154) comprises: a) a processor (156) configured to carry out the steps of the computer-implemented method (100) according to one of claims 1 to 8, b) a computer-readable storage medium (158) according to claim 10, which is communicatively connected to the processor (156), c) at least one sensor (104) for generating sensor data, which is communicatively connected to the processor (156), d) an input device (132) for inputting a setting (130) relating to a function of the vehicle, which is communicatively connected to the processor (156), and e) a control device (160) for executing the function.

13. A vehicle comprising a device (154) for data processing for controlling a function of the vehicle according to claim 12.

Images