Method and communication unit for augmenting communication

Abstract

Communication unit (151) for providing communication (200) with a second communication unit (152); wherein the communication unit (151) is configured, - to determine a communication message (201) generated by a user of the communication unit (151) for communication (200) and to send it to the second communication unit (152); - To determine user sensor data in relation to the user's identity; - to determine, based on user sensor data, which user from a group of different users of the communication unit (151) generated the communication message (201); - automatically determine context information (202) relating to the communication message (201) that indicates the identity of the user who generated the communication message (201); and - to send the context information (202) as part of the communication (200) to the second communication unit (152).

Description

[0001] The invention relates to a method and a corresponding communication unit that make it possible to increase the quality of communication between communication partners.

[0002] A vehicle can enable one or more users to communicate with one or more communication partners via a communication platform, such as a text messaging chat platform or an audio communication platform, while traveling. The one or more communication partners may be located at least partially outside the vehicle.

[0003] The user, especially the driver, of a vehicle is typically occupied with other tasks while driving, particularly with the driving task of controlling the vehicle longitudinally and / or laterally, so the user usually participates in communication with only limited attention. Furthermore, it can happen that, for example when using a hands-free system, several occupants of the vehicle simultaneously participate in a communication with a partner outside the vehicle. This can be perceived as confusing and / or unsatisfactory by the communication partner.

[0004] US 2017 / 0308267 A1 describes a method for the automatic, context-based determination of emotion icons. DE 102018215626 A1 describes a method for reproducing the emotion of a driver. US 2016 / 0308794 A1 describes a method for determining a suggested response message.

[0005] This document addresses the technical task of efficiently and reliably increasing the quality of communication between communication partners, especially when a vehicle user is involved.

[0006] The problem is solved by each of the independent claims. Advantageous embodiments are described, inter alia, in the dependent claims. It should be noted that additional features of a claim dependent on an independent claim, without the features of the independent claim itself or only in combination with a subset of the features of the independent claim, can constitute a separate invention independent of the combination of all features of the independent claim, which can be made the subject of an independent claim, a divisional application, or a subsequent application. This applies equally to technical teachings described in the description, which can constitute an invention independent of the features of the independent claims.

[0007] According to one aspect, a communication unit is described as enabling communication with a second communication unit. The communication unit may include a user interface that allows a user to generate a communication message, in particular a sequence of communication messages. For this purpose, the user interface may include controls (e.g., a physical or virtual keyboard) to generate a text message. Alternatively or additionally, the user interface may include at least a microphone to generate an audio message.

[0008] The communication unit is configured to detect a communication message generated by a user of the communication unit and send it (e.g., via a wireless communication link) to the second communication unit. Specifically, an audio message or a text message can be detected and sent.

[0009] The communication unit is further configured to automatically determine contextual information related to the communication message. This contextual information can be determined independently of the user or any user of the communication unit. In particular, the contextual information for the communication message can be determined without any input from the user or any user of the communication unit (beyond the communication message itself).

[0010] The communication unit is designed to collect user sensor data relating to the user, particularly regarding the user's identity and / or emotions. This user sensor data can include, for example, image and / or audio data. Contextual information can then be precisely determined based on this user sensor data.

[0011] The communication unit is further configured to determine, based on user sensor data, particularly voice and / or facial recognition, which user from a group of different users of the communication unit generated the communication message. Contextual information is then determined that reveals the identity (e.g., the name) of the user who generated the communication message.

[0012] Additionally, the communication unit can be configured to determine content data relating to the content and / or semantics of the communication message. For this purpose, a semantic analysis of the communication message can be performed (e.g., using artificial intelligence). The contextual information can then be precisely determined based on the content data. For example, additional information that supplements the content of the communication message can be provided as contextual information. This additional information can be determined (as described below) based on device sensor data.

[0013] Alternatively or additionally, the communication unit can be configured to acquire device sensor data relating to a device (e.g., a vehicle or a mobile user device) in which the communication unit is operated. The device sensor data can, for example, display: the device's speed; the planned route to a destination (e.g., planned on the device's navigation system); and / or the device's temperature and / or the temperature in its environment. Alternatively or additionally, the device sensor data can display: the (predicted) arrival time, departure time, and / or departure location; the traffic situation; the weather conditions, visibility, coefficient of friction, and / or road surface condition; the vehicle's status (e.g., convertible top open, light sensor, remaining range, fuel consumption, and / or driving mode (Sport, Comfort, etc.)); and the entertainment status (radio station, playlist name, track / artist, etc.).) and / or the situation of actions (eating, singing, playing, etc.); a cognitive demand on the driver (e.g., the presence of a complex traffic situation); and / or external information, e.g., regarding forecasts concerning the length or duration of a traffic jam, a construction site, etc.

[0014] The contextual information can then be determined precisely based on the device sensor data (e.g. to display the speed of movement, the route, the destination, the time of arrival at the destination, and / or the temperature).

[0015] Contextual information can thus be automatically generated for a communication, especially for a specific communication message, to enhance the communication. This contextual information can then be sent to the second communication unit as part of the communication. Furthermore, the contextual information can be displayed along with the communication message via a user interface of the second communication unit (which may be, for example, part of a vehicle or a mobile device). This improves the convenience and quality of the communication.

[0016] The communication unit can be configured to identify contextual information as supplementary information to the specific communication message. For example, a code can be sent to the second communication unit indicating that the contextual information is supplementary (not explicitly generated by a user). Based on this code, the second communication unit can then verify whether a received message is a communication message (entered by a user) or contextual information related to a communication message. The second communication unit can then explicitly identify the contextual information, allowing the user of the second communication unit to distinguish between a communication message and contextual information.

[0017] The communication unit can thus be configured to send the context information along with a code that allows the second communication unit to distinguish a received communication message from the received context information. This further improves the quality and convenience of communication.

[0018] The communication unit can be configured to automatically detect a pictogram, particularly an emoticon, as contextual information and insert it into the communication (e.g., directly following the communication message). The pictogram can depend on the user's emotion and / or the content of the communication message. This can significantly enhance the quality of communication.

[0019] The communication unit can be configured to detect and send text and / or a pictogram as a communication message and / or contextual information. Alternatively or additionally, the communication unit can be configured to enable text-based communication. In particular, the communication unit can be configured to detect an audio signal from the user and, based on this signal, generate a text-based communication message and send it to the second communication unit. This allows for speech-to-text translation, enabling particularly convenient communication (especially for a user in a vehicle).

[0020] The communication unit can be configured to automatically insert contextual information into communications, at least via a user-configurable setting. For example, a user can configure this setting through the communication unit's user interface. This setting can allow, restrict, or completely disable the automatic insertion of contextual information. Depending on the communication unit's setting, contextual information can then be determined and inserted into a specific communication message, or the determination and transmission of contextual information can be prevented. This further enhances user convenience.

[0021] According to another aspect, a further (second) communication unit is described for establishing communication with a first communication unit. The second communication unit can be designed to be complementary to the first communication unit. In particular, the second communication unit can act as a receiver and the first communication unit as a sender within the context of communication. A device typically comprises a combination of the first communication unit (as sender) and the second communication unit (as receiver).

[0022] The (second) communication unit is configured to receive a communication message from the first communication unit. This communication message may have been generated by a user of the first communication unit.

[0023] The (second) communication unit is further configured to automatically receive contextual information relating to the communication message, generated by the first communication unit, along with a code. This code can be designed in such a way that it allows the (second) communication unit to recognize that the contextual information was automatically generated by the first communication unit (and not by a user of the first communication unit).

[0024] Furthermore, the (second) communication unit is configured to output the communication message, along with the contextual information, to a user of the (second) communication unit via a user interface (e.g., by displaying it on a screen or playing it back through a speaker). The output can be presented in such a way that it is clear to the user of the (second) communication unit that the communication message was generated by a user of the first communication unit and that the contextual information was automatically inserted by the first communication unit. This can be achieved, for example, by using different colors and / or different fonts for the communication message and the contextual information.

[0025] The combined output of communication messages with contextual information can increase the quality and convenience of communication.

[0026] According to another aspect, a (road) motor vehicle (in particular a passenger car or a truck or a bus or a motorcycle) is described that includes at least one of the communication units described in this document.

[0027] According to another aspect, a mobile user device (e.g. a smartphone or a tablet PC) is described that includes at least one of the communication units described in this document.

[0028] According to another aspect, a method for establishing communication with a second communication unit is described. This method includes recognizing a communication message (e.g., a text message) generated by a user at a first communication unit. Furthermore, the method includes sending the communication message to the second communication unit (e.g., via a wireless communication link). Additionally, the method includes automatically recognizing contextual information related to the communication message and sending this contextual information to the second communication unit as part of the communication process.

[0029] According to another aspect, a further (complementary) procedure for establishing communication with a first communication unit is described. This procedure involves receiving a communication message from the first communication unit, where the communication message was generated by a user of the first communication unit. Furthermore, the procedure involves receiving context information automatically generated by the first communication unit regarding the communication message, along with a code. This code enables the system to recognize that the context information was automatically generated by the first communication unit.

[0030] The procedure further includes outputting the communication message along with the context information via a user interface, so that it is recognizable that the communication message was generated by a user of the first communication unit, and that the context information was automatically inserted by the first communication unit.

[0031] Another aspect described is a software (SW) program. The SW program can be configured to run on a processor (e.g., on a vehicle's control unit) and thereby execute one of the procedures described in this document.

[0032] According to another aspect, a storage medium is described. The storage medium can contain a software program that is configured to run on a processor and thereby execute one of the procedures described in this document.

[0033] It should be noted that the methods, devices, and systems described in this document can be used both alone and in combination with other methods, devices, and systems described in this document. Furthermore, any aspect of the methods, devices, and systems described in this document can be combined with one another in a variety of ways. In particular, the features of the claims can be combined with one another in a variety of ways.

[0034] The invention will now be described in more detail using exemplary embodiments. Fig. 1 an exemplary communication system with a vehicle and a vehicle-external communication device; Fig. 2. An exemplary text communication; and Fig. 3a and Fig. 3b each a flowchart of an exemplary procedure for supporting, in particular augmenting, communication.

[0035] As stated at the outset, this document deals with improving the quality of communication between communication partners, particularly when a vehicle user is involved. In this context, it shows Fig. 1 An exemplary communication system 150 with several communication units 151, 152, one of which is located in a vehicle 100 and the other in a mobile (external to the vehicle) communication device 110. The first communication unit 151 includes, for example, a user interface 102 of the vehicle 100, which has, for example, one or more microphones for receiving audio signals and / or one or more control elements for receiving text messages. Furthermore, the user interface 102 can be configured to output an audio signal from another communication partner (e.g., via a loudspeaker) and / or to output a text message from another communication partner (e.g., via a screen). In general, the user interface 102 can be configured to receive a communication message from a first communication partner and / or to output a communication message from a second communication partner.

[0036] The communication messages can be sent and / or received via a transmit / receive module 103 of the vehicle 100, e.g. via a (wireless) communication connection 113, such as WLAN, Bluetooth, 3G, 4G or 5G. The process of acquiring and sending or receiving and outputting communication messages can be controlled by a (control) device 101 of the vehicle 100.

[0037] The vehicle 100 may further include a user sensor 104, which is configured to acquire user sensor data relating to the user of the vehicle 100. The user sensor 104 may, for example, include a camera directed at the user. The (control) device 101 may, for example, be configured to determine the user's identity based on the user sensor data from the user sensor 104.

[0038] In particular, the identity of the person who generated a specific communication message can be determined based on the user sensor data from user sensor 104.

[0039] The vehicle 100 is an example of a first communication unit 151 in a general communication system 150. The mobile communication device 110, with a user interface 112 (which, for example, has a touch-sensitive screen) and a user sensor 114 (for example, a selfie camera), is an example of a second communication unit 152 in the communication system 150.

[0040] Fig. Figure 2 shows an exemplary text communication 200 (as an example of general (text or audio) communication). The communication 200 comprises a sequence of first communication messages 201, which are captured at the first communication unit 151 and sent to the second communication unit 152, and a sequence of second communication messages 211, which are captured at the second communication unit 152 and sent to the first communication unit 151.

[0041] The communication system 150, in particular at least one communication unit 151, 152, can be configured to augment or supplement a communication 200 with additional contextual information 202. In particular, contextual information 202 can be determined and provided for a specific communication message 201.

[0042] For example, based on the user sensor data from user sensor 104, it can be determined which user from a group of different users generated (e.g., spoke or wrote) a specific communication message 201. The user's identity can then be automatically provided as context information 202 along with the communication message 201 within the communication 200.

[0043] Alternatively or additionally, based on the user sensor data from user sensor 104 and / or based on the content of a communication message 201 (in particular based on the semantics of a communication message 201), an emotion of the user who generated the communication message 201 can be determined. This emotion can then be incorporated into the communication 200, for example, by means of an emoticon or additional text (i.e., context information 202).

[0044] Further examples of context information 202 are, • the value of the movement speed of the device 100, 110 in which a communication unit 151, 152 is located (e.g., when the movement speed is mentioned in a communication message 201, for example, within a traffic jam); and / or • the arrival time at a destination and / or the estimated travel time of a route planned on a navigation system, for example when the current route is mentioned in a communication message 201.

[0045] A communication unit 151, 152 can be configured to automatically add a variety of different pieces of information as context information 202 to a communication message 201. For example, the user's identity, an emoticon, further information relating to the content of the communication unit 201, etc., can be added as context information 202.

[0046] A communication unit 151, 152 can allow a user to define one or more settings regarding automatically generated context information 202. For example, the level of detail of automatically generated context information 202 can be set. The context information 202 can then be generated based on the one or more settings and inserted into a communication 200.

[0047] When multiple people communicate using the same device 151, 152, it is usually not possible to assign a message 201 to a specific person. The measures described in this document enable a communication partner to assign a message 201 to a person by automatically adding context information 202.

[0048] A chat client or telephone conference client 151, 152 may, in addition to input options for text and / or speech, include a further component 104 that is trained to recognize the person currently speaking and / or texting within a group or room, and especially within a vehicle 100, by means of speaker recognition and / or image recognition. This information can be automatically provided as context information 202 for a message 201. Additionally, messages 201 may be enhanced with emoticons, e.g., based on the emotions of communication partners, especially in combination with the emotions of the other communication partner and / or in combination with the context from the vehicle 100, the environment, and / or the communication history 200 (especially the chat).

[0049] Person A might be in a chat with another person B outside of vehicle 100. If another person C inside vehicle 100 speaks into the chat, and this person is not the original chat partner A, the one or more messages 201 from person C can be flagged. For example, context information 202 can be displayed whenever a message 201 originates from person C.

[0050] A communication unit 151, 152 can be configured to perform automatic speech-to-text and / or automatic text-to-speech transformation. For example, an audio message can be automatically converted into a text message 201, which is then sent to the communication partner. Conversely, a received text message 201 can be automatically converted into an audio message and output as an audio message. This enables convenient and safe text communication 200, particularly in a vehicle 100.

[0051] Alternatively or additionally, one or more contextual parameters (e.g., temperature, speed, estimated time of arrival, etc.) can be implicitly and / or proactively (e.g., textually or as a pictogram) inserted into a conversation 200 as contextual information 202. A communication unit 151, 152 can automatically recognize that contextual information 202 should be inserted for a specific topic and / or problem within the communication 200. For this purpose, the content of one or more messages 201 can be analyzed. The one or more messages 201 can then be augmented with contextual information 202. The contextual information 202 can include, for example, text (e.g., template-based and / or generated using artificial intelligence) and / or icons. The contextual information 202 can be provided as a separate contribution within a communication 200.

[0052] It may be possible to enable a user to suppress the insertion of context information 202 generally or selectively (e.g. for one or more topics and / or contexts).

[0053] Fig. Figure 3a shows a flowchart of an exemplary (possibly computer-implemented) procedure 300 for providing communication 200, in particular text communication or text chat, with a second communication unit 152. The procedure 300 can be executed by a first communication unit 151. The first communication unit 151 can be part of a vehicle 100.

[0054] The procedure 300 comprises determining 301 a communication message 201 generated by a user at a first communication unit 151 for communication 200. The communication message 201 may, for example, have been generated by the user via a user interface 102 of the first communication unit 151 (e.g., spoken or texted).

[0055] Furthermore, the procedure 300 includes sending 302 the communication message 201 to the second communication unit 152. The communication message 201 can then be output via a user interface 112 of the second communication unit 152.

[0056] The procedure 300 further includes the automatic determination 303 of context information 202 relating to the communication message 201. This automatic determination 303 can be performed without any user intervention at the first communication unit 151. The context information 202 can be determined, for example, based on user sensor data relating to the user and / or based on device sensor data relating to the device 100, 120 in which the first communication unit 151 is operated. Alternatively or additionally, the context information 202 can be determined automatically based on a semantic analysis of the communication message 201.

[0057] Furthermore, the procedure includes sending context information 202 as part of communication 200 to the second communication unit 152. Context information 202 can be sent in the same format as the original communication message 201 (e.g., as a text message or an audio message). Context information 202 can indicate (e.g., through a code) that it was automatically generated by the first communication unit 151 (and not explicitly by a user at the first communication unit 151). Context information 202 can then be output via the user interface 112 of the second communication unit 152 (e.g., directly following communication message 201). It can be clearly indicated that this is context information 202 (and not a communication message 201).

[0058] Fig.Figure 3b shows a flowchart of a complementary (possibly computer-implemented) procedure 310 for providing communication 200 with a first communication unit 151. The procedure 310 can be executed by a second communication unit 152.

[0059] The procedure 310 comprises receiving 311 a communication message (in particular a text message) 201 for communication 200 from the first communication unit 151. The communication message 201 may have been generated (e.g., spoken or written) by a user of the first communication unit 151. The communication message 201 may be received via a (possibly wireless) communication link 113.

[0060] The procedure 310 further comprises receiving 312 context information 202 automatically generated by the first communication unit 151 in relation to the communication message 201. The context information 202 can be received together with a code, the code making it possible to recognize that the context information 202 was automatically generated by the first communication unit 151 (and not by a user of the first communication unit 151).

[0061] Furthermore, the procedure 310 includes the output 313 of the communication message 201 together with the context information 202 via a user interface 112 (e.g., directly one after the other). The communication message 201 and the context information 202 can be output in such a way (e.g., by using different colors) that it is recognizable to a user (of the second communication unit 152) that the communication message 201 was generated by a user of the first communication unit 151, and that the context information 202 was automatically inserted by the first communication unit 151.

[0062] The measures described in this document can increase the comfort and quality of (text) communication 200 (especially for a vehicle user 100).

[0063] The present invention is not limited to the embodiments shown. In particular, it should be noted that the description and the figures are intended only to illustrate the principle of the proposed methods, devices, and systems by way of example.

Claims

[1] Communication unit (151) for providing communication (200) with a second communication unit (152); wherein the communication unit (151) is configured, - to determine a communication message (201) generated by a user of the communication unit (151) for communication (200) and to send it to the second communication unit (152); - To determine user sensor data in relation to the user's identity; - to determine, based on user sensor data, which user from a group of different users of the communication unit (151) generated the communication message (201); - automatically determine context information (202) relating to the communication message (201) that indicates the identity of the user who generated the communication message (201); and - to send the context information (202) as part of the communication (200) to the second communication unit (152). [2] Communication unit (151) according to claim 1, wherein the communication unit (151) is configured to determine, by means of speech and / or facial recognition, which user from the group of different users of the communication unit (151) generated the communication message (201). [3] Communication unit (151) according to one of the preceding claims, wherein the communication unit (151) is configured, - To determine content data relating to the content and / or semantics of the communication message (201); and - to determine the context information (202) also on the basis of the content data. [4] Communication unit (151) according to one of the preceding claims, wherein the communication unit (151) is configured, - To determine device sensor data relating to a device (100, 110) in which the communication unit (151) is operated; and - to determine the context information (202) also on the basis of the device sensor data. [5] Communication unit (151) according to claim 4, wherein the device displays sensor data: - a movement speed of the device (100, 110); - a planned route to a destination of the device (100, 110); - a temperature of the device (100, 110) and / or a temperature in the environment of the device (100, 110); - an arrival time, a departure time and / or a departure location; - a traffic situation; - a weather situation, visibility, road friction coefficient and / or road surface condition; - a state of a vehicle (100) in which the communication unit (151) is arranged; - a situation in the vehicle (100) in which the communication unit (151) is located; - a request to a driver of the vehicle (100) in which the communication unit (151) is located; and / or - a prediction regarding a situation outside the vehicle (100) in which the communication unit (151) is located. [6] Communication unit (151) according to one of the preceding claims, wherein the communication unit (151) is configured to automatically determine a pictogram, in particular an emoticon, as context information (202) and insert it into the communication (200). [7] Communication unit (151) according to claim 6 with reference to claim 3, wherein the pictogram depends on an emotion of the user and / or on the content of the communication message (201). [8] Communication unit (151) according to any one of claims 1 to 5, wherein the communication unit (151) is configured, - to identify and send text and / or a pictogram as a communication message (201) and / or as contextual information (202); and / or - to enable text communication (200). [9] Communication unit (151) according to one of the preceding claims, wherein the communication unit (151) is configured, - to detect an audio signal from the user; and - to generate a text communication message (201) based on the audio signal and send it to the second communication unit (152). [10] Communication unit (151) according to one of the preceding claims, wherein the communication unit (151) is configured, - to determine at least one user-configurable setting of the communication unit (151) for the automatic insertion of context information (202) into communications (200); and - depending on the setting of the communication unit (151) to determine the context information (202) or to prevent the determination and transmission of the context information (202). [11] Communication unit (151) according to one of the preceding claims, wherein the communication unit (151) is configured to send the context information (202) together with a code that enables the second communication unit (152) to distinguish a received communication message (201) from received context information (202). [12] Communication unit (152) for providing communication (200) with a first communication unit (151); wherein the communication unit (152) is set up, - to receive a communication message (201) for communication (200) from the first communication unit (151); wherein the communication message (201) was generated by a user of the first communication unit (151); - to receive context information (202) automatically generated by the first communication unit (151) with respect to the communication message (201) together with a code; wherein the code of the communication unit (152) makes it possible to recognize that the context information (202) was automatically generated by the first communication unit (151); wherein the context information (202) with respect to the communication message (201) indicates an identity of the user of the first communication unit (151) from a group of distinct users of the first communication unit (151) who generated the communication message (201); and - to output the communication message (201) together with the context information (202) via a user interface (112) to a user of the communication unit (152) so that it is recognizable to the user that - the communication message (201) was generated by a user of the first communication unit (151); and - the context information (202) was automatically inserted by the first communication unit (151). [13] Motor vehicle (100) comprising at least one communication unit (151, 152) according to any one of the preceding claims. [14] Method (300) for providing communication (200) with a second communication unit (152); wherein the method (300) comprises, - Determine (301) a communication message (201) generated by a user at a first communication unit (151) for communication (200); - Sending (302) the communication message (201) to the second communication unit (152); - Determining user sensor data in relation to a user's identity; - Determine, based on user sensor data, which user from a group of different users of the first communication unit (151) generated the communication message (201); - automatic determination (303) of contextual information (202) relating to the communication message (201) that indicates the identity of the user who generated the communication message (201); and - Sending (304) the context information (202) as part of the communication (200) to the second communication unit (152). [15] Method (310) for providing communication (200) with a first communication unit (151); wherein the method (310) comprises, - Receiving (311) a communication message (201) for communication (200) from the first communication unit (151); wherein the communication message (201) was generated by a user of the first communication unit (151); - Receiving (312) context information (202) automatically generated by the first communication unit (151) with respect to the communication message (201) together with a code; wherein the code makes it possible to recognize that the context information (202) was automatically generated by the first communication unit (151); wherein the context information (202) with respect to the communication message (201) indicates an identity of the user of the first communication unit (151) from a group of distinct users of the first communication unit (151) who generated the communication message (201); and - Output (313) of the communication message (201) together with the context information (202) via a user interface (112), so that it is recognizable that - the communication message (201) was generated by a user of the first communication unit (151); and - the context information (202) was automatically inserted by the first communication unit (151).

Images