Communication assistance for vehicles

Abstract

Various aspects of the present invention relate to control systems, vehicle systems, methods, and computer-readable instructions for an assistance system for controlling a vehicle. The control system for the vehicle includes one or more processors configured to: receive an assistance request signal indicating an assistance request event, the assistance request signal being configured to initiate a communication connection between the vehicle and a third party; upon receiving the assistance request signal, send a signal to a display screen of the vehicle, wherein the signal causes the display screen to display a visual indicator indicating the initiation of the communication connection and the availability of transmitting a response to the third party by providing non-audio message user input; receive non-audio message user input indicating a message to be provided to the third party in response to the display of the visual indicator; and transmit a message corresponding to the non-audio message user input to the third party.

Description

Technical Field

[0001] This disclosure relates to a control system for an assistance system for controlling a vehicle. Aspects of the invention relate to a control system for an assistance system for controlling a vehicle, a vehicle system, a vehicle, a computer-implemented method, and computer-readable instructions for performing the method. Background Technology

[0002] Means of communication between vehicle occupants and receivers outside the vehicle are known. For example, a driver may be able to make a telephone call to a party outside the vehicle, such as an assistance service (e.g., a breakdown assistance service or emergency service). Such audio communication may not be accessible to all vehicle users, such as those who are deaf or have hearing impairments. Furthermore, such users with hearing difficulties may not be aware that an audio call is taking place because they may not be able to hear what the other party is saying.

[0003] The purpose of this invention is to address one or more of the disadvantages associated with the prior art. Summary of the Invention

[0004] The present invention provides a control system for an assistance system for controlling a vehicle. The aspects of the invention relate to a control system for an assistance system for controlling a vehicle, a vehicle system, a vehicle, and a computer-implemented method, as claimed in the appended claims. The terms "vehicle user" and "occupant" are used interchangeably to refer to either the driver of the vehicle or a passenger in the vehicle.

[0005] According to one aspect of the present invention, a control system for controlling an assistance system of a vehicle is provided, the control system comprising one or more processors configured to: receive an assistance request signal indicating an assistance request event, the assistance request signal being configured to initiate a communication connection between the vehicle and a third party; upon receiving the assistance request signal, send a signal to a display screen of the vehicle, wherein the signal causes the display screen to display a visual indicator indicating the initiation of the communication connection; receive user input indicating a message to be provided to a third party in response to the display of the visual indicator; and transmit the message to the third party.

[0006] In some examples, the assistance system may include an eCall system. The control system may be part of or complementary to the eCall system. In some cases, one or more processors may be configured to: receive an assistance request signal indicating an event of an assistance request from the emergency call system; and, based on the received assistance request signal from the emergency call system, send a signal to the vehicle's display screen. In some examples, visual indicators may include visual indications that the eCall system has been activated.

[0007] The control system includes one or more controllers, which collectively include: at least one electronic processor having an electrical input for receiving input signals; and at least one memory device electrically coupled to the at least one electronic processor and having instructions stored therein; wherein the at least one electronic processor is configured to access the at least one memory device and execute the instructions on the at least one memory device to: receive an assistance request signal as an input signal, the assistance request signal indicating an assistance request event, the assistance request signal being configured to initiate a communication connection between the vehicle and a third party; upon receiving the assistance request signal, send a signal to the vehicle's display screen, wherein the signal causes the display screen to display a visual indicator indicating the initiation of the communication connection; receive user input as additional user input, the user input indicating a message to be provided to a third party in response to the display of the visual indicator; and transmit the message to the third party.

[0008] Advantageously, when a communication connection is established between the vehicle and a third party regarding assistance needed by the vehicle, a visual indicator is provided to the user to indicate that communication is in progress. The user can respond to the third party by providing a message in response to seeing the visual indicator displayed. If the user has a hearing impairment, or if the user is panicked or shocked due to a situation requiring assistance, such as a breakdown or car accident, the visual indicator helps the user understand that a call is in progress. In response, the user can then provide user input indicating a message to be sent back to the caller and send that message.

[0009] According to one aspect of the invention, a control system for controlling an assistance system of a vehicle is provided, the control system comprising one or more processors configured to: receive an assistance request signal indicating an assistance request event, the assistance request signal being configured to initiate a communication connection between the vehicle and a third party; upon receiving the assistance request signal, send a signal to a display screen of the vehicle, wherein the signal causes the display screen to display a visual indicator indicating the initiation of the communication connection and the availability of transmitting a response to the third party by providing non-audio message user input; receive non-audio message user input indicating a message to be provided to the third party in response to the display of the visual indicator; and transmit a message corresponding to the non-audio message user input to the third party.

[0010] The control system includes one or more controllers, which collectively include: at least one electronic processor having an electrical input for receiving input signals; and at least one memory device electrically coupled to the at least one electronic processor and having instructions stored therein; wherein the at least one electronic processor is configured to access the at least one memory device and execute the instructions on the at least one memory device to: receive an assistance request signal as an input signal, the assistance request signal indicating an assistance request event, the assistance request signal being configured to initiate a communication connection between the vehicle and a third party; upon receiving the assistance request signal, send a signal to the vehicle's display screen, wherein the signal causes the display screen to display a visual indicator, the visual indicator indicating the initiation of the communication connection and the availability of transmitting a response to the third party by providing non-audio message user input; receive non-audio message user input as additional user input, the non-audio message user input indicating a message to be provided to the third party in response to the display of the visual indicator; and transmit the message corresponding to the non-audio message user input to the third party.

[0011] This system can advantageously complement audio call systems that enable drivers and third parties to speak to each other in the event of an assistance request, such as a collision or breakdown. For example, deaf or hard-of-hearing drivers, or those who are shocked or panicked due to an accident or breakdown, may not be aware that an audio call is in progress. Visual cues of the ongoing communication are advantageous, allowing users to respond to messages from third parties. Given the ability to provide non-audio message input in response to cues of an ongoing audio communication with a third party, users are better able to respond and respond appropriately. Users have greater control over which third-party assistance is provided. For example, instead of the third party failing to receive a voice response from a user who cannot speak or hear (which could serve as a default action call to be deployed to the vehicle's emergency services), the third-party call handler can receive a response from a driver who is visually cued to be in progress and can provide a non-audio response, enabling the provision of appropriate assistance rather than default assistance.

[0012] In some examples, the communication connection can be an audio communication connection. That is, the vehicle can initiate an audio communication connection with a third party, such as a telephone call or an internet-based audio call. Advantageously, the control system can operate using existing audio communication technologies, and in some examples, non-audio inputs for transmission during a call can be converted into audio, and / or non-audio communication can be used in conjunction with audio communication to supplement audio communication on the audio communication channel.

[0013] The assistance request signal may include one or more of the following: a fault assistance request signal generated in response to a user of the vehicle providing input to the vehicle's fault assistance input device; an emergency assistance request signal generated in response to a user of the vehicle providing input to the vehicle's emergency assistance input device; a fault assistance request signal generated in response to a vehicle fault module determining that a vehicle fault event has occurred; and an accident request signal generated in response to one or more sensors of the vehicle detecting a collision event.

[0014] Advantageously, the system can be used to assist drivers in situations such as: a driver making a breakdown call from the vehicle, for example, via a hardwired breakdown call button; a driver making an emergency assistance call from the vehicle, for example, via a hardwired emergency call button; automatically issuing a breakdown assistance call when sensor data (e.g., collision sensors present in the vehicle's bumpers, wheels, etc.) indicates a breakdown event; or automatically issuing an emergency assistance call in the event of a collision. Existing breakdown and emergency call systems can be advantageously supplemented by: visually acknowledging to the user that a call is in progress, and having the ability to respond in a non-audio manner during the call to communicate clearly and effectively with third parties. The third party that transmits a message to the user in response to non-audio message input can be an assistance service associated with the vehicle manufacturer, a breakdown assistance service, or an emergency service provider such as police, ambulance service, or fire service.

[0015] Non-audio message user input indicating a message to be provided to a third party may include one or more of the following: audio-activated message input, configured to provide a message to a third party indicating that the message sender has a hearing impairment and to activate the vehicle's audio receiver, which is configured to receive and provide a voice audio message from the message sender for transmission to the third party after activation and after providing the message indicating that the message sender has a hearing impairment; free text message input, wherein the vehicle includes a free text input device configured to receive a text message input by the user; and pre-written message selection input, wherein the display screen is configured to present one or more pre-written messages for the user to select.

[0016] Advantageously, once a user is visually prompted that a call is in progress and a response is available, there are different possible ways in which the user can respond to a third party. The user can select an option to provide input to a third party by pressing a button or touching an area of ​​the touchscreen, indicating that the vehicle cannot properly hear the audio output. The third party can then understand how the user is reacting during the call and that the user may be communicating through methods other than simply speaking and listening. The user may be able to speak to the third party, but the third party has been informed that the user cannot hear the voice response and may instead be receiving comments from the third party in an alternative manner, for example, displayed after text-to-speech conversion. The user may be able to input free-text messages, for example, via a keyboard such as a virtual keyboard displayed on the vehicle's screen, thus advantageously enabling the user to respond to a third party in an unrestricted manner without speaking. The user may be able to select pre-written messages presented on the vehicle's screen, thus advantageously enabling the user to quickly and easily select a response to a third-party caller without speaking.

[0017] If the non-audio message user input includes a selection of one or more pre-written messages, the one or more pre-written messages displayed for selection can be one or more of the following: text pre-written messages; image-based graphic pre-written messages; and image-based sign language pre-written messages. Advantageously, the pre-written messages that the user can select to communicate with a third party can have different forms, which can be set as user preferences in the driver's vehicle preference settings. For example, some users may be most accustomed to reading text messages to understand the content to be sent to a third party, while another user may be most accustomed to graphic messages such as symbols, and a user whose first language is sign language may be most accustomed to selecting pre-written messages represented in sign language.

[0018] If the non-audio message user input includes the selection of one or more pre-written messages, the content of one or more pre-written messages may depend on one or more of the following: the type of the third party with which it initiated the communication connection is a fault service operator or an emergency service operator; and one or more sensor types, wherein the assistance request signal is initiated by one or more sensors of the vehicle having said one or more sensor types detecting a collision event.

[0019] Advantageously, if the messages presented for selection are pre-written messages, they can be selected and presented based on the circumstances of the third-party call. For example, if the call is with a third-party fault service provider, the pre-written message may include a reference number to the driver's vehicle fault insurance policy, or it may include options related to the nature of the fault, such as a suspected fault or the area where the vehicle has stopped. As another example, if the call is with emergency services, the pre-written message may include health or communication data, such as the user's hearing level or other health information, and may include information about the number of passengers and any suspected injuries or the severity of injuries. As yet another example, if the pre-written message depends on the type of sensor that detected the collision event, the pre-written message may indicate that a frontal collision occurred if the sensors at the front of the vehicle were activated; or that a tire blowout occurred, as detected by sensors at the rear tires.

[0020] If the non-audio message user input includes a selection of one or more pre-written messages, the control system can be configured to: receive the non-audio message user input indicating a message to be provided to a third party in response to a user selection made by a displayed visual indicator; and transmit the message corresponding to the selected pre-written text message to the third party. Advantageously, the displayed visual indicator can act as an input object with which the user can interact to transmit the corresponding message to the third party. Such an arrangement can provide a simple and clear user interface for use by users who may be in emergency or stressful situations, and it is beneficial to find simple methods of information presentation and easy interaction with assistance.

[0021] Pre-written messages can be displayed as image-based messages, and the control system can be configured to convert the image-based messages into text messages before transmitting them to a third party. Advantageously, users can select images representing their desired response to a third party, and these images can be converted into text messages for the third-party operator (and possibly into voice messages via text-to-speech conversion for subsequent conversion). For example, if a user is deaf, they may find it easiest to select an image representing a sign language representation of the message they wish to convey, and if the call handler cannot understand sign language, the sign language representation can be converted into text and / or voice for the call handler.

[0022] Visual indicators may include one or more of the following: a text message describing the event that led to the initiation of the communication connection; and an image indicating the event that led to the initiation of the communication connection. Advantageously, the visual indicator presented to the user may be text- or image-based (e.g., a sign language representation of a message) indicating that a call has been made to a third party.

[0023] In some examples, non-audio message user input includes a selectable hearing impairment or deafness option on the display of a visual indicator. That is, in response to an assistance request event, the selectable hearing impairment or deafness mode can be accessible on the display. Allowing the user to optionally select hearing impairment or deafness support can reduce the computational burden when the hearing impairment or deafness option is not needed. In some examples, in response to selecting a hearing impairment or deafness option, a signal can be sent to the vehicle's display screen to cause the display of a visual indicator. The signal to the display screen can cause an additional visual indicator indicating hearing impairment or deafness to be displayed for the vehicle user. Advantageously, this allows emergency or breakdown personnel arriving at the vehicle to determine that the occupant is deaf or has a hearing impairment.

[0024] One or more processors may also be configured to collectively receive user information signals indicating that the vehicle user has a hearing impairment or deafness. This user information may be received from the optional options above and / or user profile information stored in the vehicle. In some examples, the message corresponding to a non-audio message user input may include information that the vehicle user has a hearing impairment or deafness. This information may be the complete non-audio message user input or it may be additional information accompanying the non-audio message user input. In some cases, this information may be automatically included in the message.

[0025] Initiating a communication connection between a vehicle and a third party may include establishing a two-way audio call between the vehicle and the third party. Advantageously, visual indications and reception of user responses may occur concurrently with the ongoing audio call to complement existing audio call configurations used for reporting faults or emergencies.

[0026] Messages transmitted to third parties may include: instructions that no assistance from the third party is to be provided; and requests to terminate established two-way audio calls. Advantageously, when making an audio call to a third party, the user can clearly indicate that no assistance is actually needed. For example, the system may be unintentionally activated or activated during a demonstration of vehicle features in a showroom, or sensors may record an incident even if the vehicle is drivable and the occupants are uninjured. In these cases, an emergency response to the vehicle may not be necessary, but dispatching an emergency responder may be the default action if the third party does not receive a voice response from the driver during the ongoing audio call. Therefore, advantageously, the user can prevent the dispatch of emergency or breakdown services when they are not actually needed and can communicate with the third party to explain this situation.

[0027] The control system can be configured to: receive message signaling from a third party, the message signaling being configured to provide audio output from the vehicle's audio output device; convert the message signaling from being configured to provide audio output to being configured to provide visual output from the vehicle's display device; and cause the display device to output visual output corresponding to the received message signaling. Advantageously, the control system is capable of receiving audio input from a third-party caller and converting that audio input into a visual representation for the user. For example, if the user is deaf or has a hearing impairment, the user can still understand what the third party is saying by receiving messages as written text messages or messages in video messages displayed in sign language.

[0028] Upon receiving an assistance request signal, the control system can be configured to automatically transmit one or more of the following to a third party: the vehicle's current location; identification data of the registered vehicle owner; and health data of one or more occupants of the vehicle. This information can be provided automatically to help the third party understand the reason for the call and provide appropriate assistance without requiring the vehicle occupants to provide such information.

[0029] In addition to displaying visual indicators on the vehicle's screens, the control system can also be configured to output one or more of the following: visual warning indicators; and tactile warning indicators. Such additional indicators can help draw the user's attention to the situation and the ongoing call.

[0030] In one aspect, a vehicle system is provided, comprising: a control system disclosed herein; a display device; and a communication module configured to facilitate communication between the vehicle and a third party. Advantageously, the control system can be configured for use with the display device in the vehicle, such as an infotainment system or a central display device, and is configured for use with the communication module to manage the sending and receiving of audio and other messages with the third party.

[0031] In one aspect, a vehicle is provided that includes any system or control system disclosed herein.

[0032] In one aspect, a computer-implemented method for controlling an assistance system for a vehicle is provided, the method comprising: receiving an assistance request signal indicating an assistance request event, the assistance request signal being configured to initiate a communication connection between the vehicle and a third party; sending a signal to a display screen of the vehicle upon receiving the assistance request signal, wherein the signal causes the display screen to display a visual indicator indicating the initiation of the communication connection and the availability of transmitting a response to the third party by providing a non-audio message input; receiving a non-audio message input indicating a message to be provided to the third party in response to the display of the visual indicator; and transmitting the message to the third party.

[0033] In one aspect, a computer-readable instruction is provided that, when executed by a computer, is configured to perform a method according to any of the said methods.

[0034] Within the scope of this application, it is expressly intended that all aspects, embodiments, examples, and alternatives set forth in the foregoing paragraphs, in the claims, and / or in the following description and drawings, as well as particularly their various features, may be adopted independently or in any combination. That is, all embodiments and / or features of any embodiment may be combined in any manner and / or combination, unless such features are incompatible. The applicant reserves the right to amend any originally filed claim or accordingly file any new claim, including the right to modify any originally filed claim to be subordinate to any other claim and / or incorporated into any other claim, even though it was not initially claimed in this manner. Attached Figure Description

[0035] One or more embodiments of the invention will now be described by way of example only with reference to the accompanying drawings, in which:

[0036] Figure 1 A vehicle according to an embodiment of the present invention is shown;

[0037] Figure 2 A series of steps are illustrated in which a control system according to an embodiment of the present invention is configured to perform;

[0038] Figure 3 A control system including a processor according to an embodiment of the present invention is shown;

[0039] Figure 4 A series of steps are illustrated in which a control system according to an embodiment of the present invention is configured to be implemented;

[0040] Figures 5A to 5C A schematic example of a visual indicator and a manner for providing non-audio message input to a user, according to an embodiment of the present invention, is shown;

[0041] Figure 6 An example system according to an embodiment of the present invention is shown; and

[0042] Figure 7 An example method according to an embodiment of the present invention is shown. Detailed Implementation

[0043] In the UK, one in five people have hearing loss or total hearing loss. Currently, more than 1.5 billion people, nearly 20% of the global population, have some degree of hearing loss. Of these, 430 million may have disabling hearing loss. It is projected that by 2050, more than 700 million people worldwide may have disabling hearing loss.

[0044] To date, assistance features such as using a vehicle's emergency call function to call emergency or breakdown services from inside the vehicle rely on two-way verbal communication between the vehicle occupants and the assistance service operator. Verbal conversation may not be helpful if the vehicle occupants have hearing impairments or deafness, especially in emergency situations where they may be stressed, panicked, or disoriented. The same applies if the verbal conversation is not conducted in their first language or is not a type of communication they can easily engage in.

[0045] In vehicles equipped with emergency call functionality, occupants can manually activate the feature and contact emergency services or breakdown assistance, or contact emergency services in response to triggers such as airbag deployment, typically initiating audio / voice communication with emergency services. By default, emergency services may be dispatched to the vehicle if the emergency call handler cannot hear a verbal response from the occupant. However, deaf or hard-of-hearing individuals may not hear the call handler and therefore may not realize they need to respond. This unnecessarily leads to emergency service dispatch when this is not an appropriate course of action. Current solutions for deaf or hard-of-hearing vehicle occupants involve using their own mobile phones to call for assistance instead of the vehicle's built-in emergency call feature. This has the disadvantage that other features of the vehicle's emergency call feature may be inaccessible to the user.

[0046] The examples disclosed herein can use signals sent in response to activating an emergency call feature or triggering an event such as airbag deployment as a signal to provide users with additional communication capabilities to communicate with third parties, such as emergency services. This additional communication capability can be indicated to the user by displaying an indicator on the vehicle's display screen, informing vehicle occupants that a call to a third-party service has been initiated. The examples also provide methods for vehicle occupants to communicate with third parties that complement simply providing audio input via speech, making the in-vehicle emergency call system more accessible and usable to a wider portion of the community.

[0047] In practice, the examples disclosed herein can be provided in a vehicle by downloading a computer program application (“app”) into the vehicle’s control system. The app can operate in a background mode, monitor triggering events that fully load the app, and provide communication capabilities and visual indications of ongoing calls to third parties. Advantageously, no changes are required to the vehicle’s existing emergency call infrastructure, and the examples disclosed herein can be provided as additional or supplementary functionality to improve accessibility to the vehicle’s emergency call features.

[0048] This article refers to the appendix Figure 1 A vehicle 10 according to an embodiment of the present invention is described. The vehicle 10 includes a vehicle system (V2X) for controlling the vehicle, as disclosed herein. Figure 6 (as shown in the diagram) or control system ( Figure 2 and Figure 3 (As shown in the diagram). Vehicle 10 also includes an assistance system. The assistance system can be considered to include call functionality that allows the occupants of the vehicle to make and maintain audio calls with third-party assistance services.

[0049] Figure 2 A series of steps are illustrated in which a control system 100 according to an embodiment of the present invention is configured to perform. The control system 100 is used to control an assistance system of a vehicle 10. The control system 100 includes one or more processors configured to collectively receive an assistance request signal 210 indicating an assistance request event 202. The assistance request signal 210 is configured to initiate a communication connection between the vehicle 10 and a third party. Upon receiving the assistance request signal 210, the control system 100 is configured to send a signal 212 to a display screen of the vehicle 204, wherein the signal 212 causes the display screen to display a visual indicator. The visual indicator indicates the initiation of the communication connection. The control system 100 is configured to: receive user input 214 indicating a message to be provided to a third party in response to the display of the visual indicator 206; and transmit a message 216 to the third party 208.

[0050] Therefore, existing fault and emergency call systems can be advantageously supplemented by the following: users are visually alerted that a call is in progress and have the ability to respond in a non-audio manner during the call to communicate clearly and effectively with third parties. The third party that transmits message 216 to the user in response to non-audio message user input 214 can be an assistance service or emergency service provider associated with the vehicle manufacturer or fault recovery service, such as police, ambulance services, or fire services.

[0051] The assistance request signal 210 can be received in response to an event that triggers a call to a third party. For example, the assistance request signal 210 may include a fault assistance request signal generated in response to a user of the vehicle providing input to the vehicle's fault assistance input device (e.g., a dedicated "fault call" button). The assistance request signal may also include a fault assistance request signal generated in response to a vehicle fault event being determined by the vehicle's fault module. Advantageously, the control system 100 can be used to assist the driver when the driver makes a fault call from the vehicle, for example via a hardwired fault call button, or the fault assistance call may be automatically issued upon receipt of sensor data indicating that a fault event has occurred (e.g., collision sensors present in the vehicle's bumpers, wheels, etc.).

[0052] The assistance request signal 210 may include an emergency assistance request signal generated in response to input provided by a user of the vehicle to the vehicle's emergency assistance input device (e.g., a dedicated "emergency call" button). The assistance request signal 210 may also include an accident request signal generated in response to one or more sensors of the vehicle 10 detecting a collision event. Advantageously, the control system 100 may be used to assist the driver in situations where the driver makes an emergency assistance call from the vehicle, for example, via a hardwired emergency call button, or automatically initiates an emergency assistance call upon collision detection.

[0053] Figure 3 A control system 100 including a processor 120 is shown according to an embodiment of the present invention. The control system 100 may include one or more controllers 110 (in this example, one controller 110 is shown).

[0054] The controller 110 includes a processing device 120 and a memory device 1130. The processing device 120 may be one or more electronic processing devices 120 operably executing computer-readable instructions. The memory device 130 may be one or more memory devices 130. The memory device 130 is electrically coupled to the processing device 120. The memory device 130 is configured to store instructions, and the processing device 120 is configured to access the memory device 130 and execute the instructions stored on the memory device 130.

[0055] The control system 100 is configured to receive an assistance request signal as an input signal 165, for example, from an ongoing assistance request input and / or an ongoing assistance event detected by one or more sensors 160. The assistance request signal may originate from an eCall or bCall system. The assistance request signal indicates an assistance request event. The assistance request signal is configured to initiate a communication connection between the vehicle and a third party. Upon receiving the assistance request signal, the control system 100 may then output a control signal 155 to control the information displayed on the vehicle's display screen 170 (as a receiving module). The control signal 155 causes the display screen 170 to display a visual indicator. The visual indicator indicates the initiation of the communication connection and the availability of transmitting a response to a third party by providing a non-audio message user input. The control system 100 is configured to receive the non-audio message user input 165 as additional user input. The non-audio message user input 165 indicates a message to be provided to a third party in response to the display of the visual indicator. The control system 100 is configured to transmit the message 155 corresponding to the non-audio message user input to the third party 170, which is acting as a receiving module. In some examples, message 155 may include information indicating that audio message transmission is unavailable, for use in informing a third party 170. For example, message 155 may provide information to a third party, such as an alarm indicating that a vehicle user is deaf or has a hearing impairment.

[0056] Controller 110 includes an input device 140 and an output device 150. Input device 140 may include an electrical input 140 of controller 110. Output device 150 may include an electrical output 150 of controller 110. Input 140 is configured to receive an assistance request signal 165, which is an electrical signal indicating an assistance request event. Output device 150 is configured to output a control signal 155 to control information displayed on a display screen. Input device 140 is configured to receive a non-audio message user input 165 indicating a message to be provided to a third party in response to the display of a visual indicator. Output device 150 is configured to output a message 155 corresponding to the non-audio message user input to a third party 170. Multiple input devices 140 may be present for receiving input signals 165 of corresponding types, and similarly, multiple output devices 150 may be present for outputting different types of output signals 155.

[0057] In some examples, the input device 140 of controller 110 may be configured to receive information signals in addition to the assistance request signal 165. The information signals may include information about one or more vehicle users from a user profile. For example, the information signals may include information that one or more vehicle users are deaf or have hearing impairments. A user may provide information about their hearing impairment or deafness to the vehicle's onboard computer, and this information may be stored in a user profile.

[0058] Controller 110 can be configured to control output device 150 to include user profile information in addition to or in combination with message 155 corresponding to non-audio message user input to third party 170. Although for clarity, information signal and assistance request signal 165 are described as separate signals, assistance request signal 165 may include information signal in some examples. That is, assistance request signal 165 may include information from the user profile. In some examples, each driver of the vehicle may have an associated user profile that is loaded when the driver is identified as riding in or driving the vehicle.

[0059] Figure 4 A series of steps are illustrated in which a control system according to an embodiment of the invention is configured to perform. Step 402 represents setting up the control system by installing, for example, an application to run on the vehicle's onboard computer. The application provides functionality to at least a portion of the onboard computer, which is configured to perform the functions disclosed herein. The ability to install an application to provide functionality is advantageous because the user can choose to install the application if they wish to use the functionality, and choose not to install the application, or choose to put the application into a sleep mode or similar mode to prevent the functionality from becoming available.

[0060] Step 404 indicates whether the application can be activated or not. In some examples, each driver in the vehicle may have an associated user profile that is loaded when the driver is identified as a passenger or driver of the vehicle, and each user profile may contain information indicating whether the application / feature is activated. For example, a deaf driver may want to activate the feature, while a driver with normal hearing may not want to activate the application (and free up computing resources for other activities). In some examples, the driver may provide the vehicle's onboard computer with information that they have a hearing impairment or deafness, and thus the application's functionality, as well as one or more other deaf / hearing-impaired assistance features, can be automatically activated.

[0061] Step 406 indicates that the app is operating in background mode, "listening" to indicate an event that a help call has been made. That is, the app can be considered to be in standby mode, where, upon receiving a signal indicating that a help call has been made, the app enables additional functions as discussed herein. Step 408 indicates that a help / emergency event signal has been triggered inside the vehicle, and step 410 indicates that the event signal is transmitted to the app to indicate that a help call will be made to a third party due to a collision or other help / emergency event.

[0062] For example, in step 406, the application may operate in a mode in which it receives data indicating that a malfunction or emergency button has been pressed; that sensors have recorded data indicating a collision or medical emergency; or similar data. In some examples, the application may be linked to medical monitoring functions in the vehicle, so that, for example, if the driver is detected as unresponsive or unable to operate the vehicle safely (e.g., they have had a stroke, are asleep, are having a seizure, etc.), the vehicle may autonomously navigate to a safe stopping position and stop its movement; in step 408, the vehicle may generate an emergency signal; and in step 410, the application may receive an emergency signal indicating the need for assistance and operate in conjunction with an automatically generated audio call to emergency services, as disclosed herein.

[0063] Step 412 indicates that an audio call to a third party has occurred due to an assistance / emergency event that has been detected. In this step, data from the vehicle and / or its sensors may also be shared with the third party to help the call recipient understand what triggered the ongoing audio call. In some examples, such a call may be made through an eCall (emergency call) system or a bCall (breakdown call) system.

[0064] Step 414 indicates that the application then operates as disclosed to present a visual indicator on the vehicle's display screen, indicating the triggering of an audio call, and that a response can be provided to a third party on the call by providing non-audio message user input. Step 416 indicates that a user-selectable option is displayed on the display screen, which the user can interact with to provide a non-audio response. Figures 5A to 5C The image shows an example of content that can be displayed on a screen. Such examples could include the availability of pre-prepared messages to be sent to third parties, such as “Need emergency medical assistance,” “I have been in a collision but do not need emergency services,” or “Please end the call, no assistance needed (test call).”

[0065] Figures 5A to 5C A schematic example of a visual indicator 502 displayed on a vehicle display screen 500 according to an embodiment of the present invention is shown, as well as a method for providing non-audio message input to a user.

[0066] Figures 5A to 5C Visual indicator 502 is shown, indicating that an audio call to a third party is being initiated. Figure 5CAn example of a visual indicator 502b is shown, which indicates the reason for making a call to a third party, and is discussed below. While an example of a telephone symbol 502 is shown, the visual indicator 502 can be any display used to visually indicate to vehicle occupants that an audio call has been initiated. For example, the visual indicator can include various symbols (e.g., exclamation marks, connection symbols, or other warning symbols), flashing indicators on a display screen, flashing indicators elsewhere in the vehicle (e.g., flashing or illuminated LEDs on the dashboard to steering wheel), or displays of objects configured to allow message input (e.g., a virtual keypad 506, preset messages 508a to 508b, or other means).

[0067] As in Figure 5A In this example, non-audio message user input indicating a message to be provided to a third party may include audio-activated message input, which is configured to provide a message to the third party indicating that the message sender has a hearing impairment or deafness but can speak and intends to provide an audio response to the third party. In this example, this input could be provided by a user interacting with call symbol 504 to indicate to the third party that they can provide audio input (and therefore the third party can hear the user speaking) but the user has a hearing impairment or deafness and cannot hear audio from the third party.

[0068] Non-audio message user input can also activate the vehicle's audio receiver (e.g., a microphone). The audio receiver is configured to, upon activation and after providing a message to a third party indicating that the message sender has a hearing impairment, receive and provide a voice-audio message from the message sender for transmission to the third party. This example is useful when a third party understands that the vehicle occupant providing the voice response cannot hear and is understanding what the third-party caller is saying through some form of voice-to-non-audio conversion, such as via voice-to-text or voice-to-sign language conversion. In some examples, non-audio message user input can also activate voice-to-text, voice-to-image, or voice-to-sign language functions, such that the voice / audio input from the third party is converted into text or another visual communication representation from the voice for display to the vehicle occupant.

[0069] As in Figure 5B In this context, non-audio message user input can include free text message input. As shown, in such an example, the vehicle includes a free text input device configured to receive text messages input by the user. Figure 5BA display screen with a virtual keyboard 506 is shown, through which a user can input free text messages and transmit them to a third party (in written / text form, or in audio form via text-to-speech conversion of a written message). The keyboard 506 or text input device is not necessarily a virtual keyboard 506, nor does it have to be part of the display visual indicator 502 on the same display screen—in other examples, the keyboard could be a physical keyboard in a vehicle, or it could be a virtual keyboard on a display screen different from the display screen showing the visual indicator 502. In such examples, the user can type any response they desire. In some examples, predictive text functionality may be present to help the user provide a response easily or quickly; for example, a user who initially types “I don’t need…” may be presented with a predicted alternative text input such as “any medical assistance” or “police response,” which the user can select by touching a prediction test on display screen 500 or by any other suitable method known in the field of predictive text.

[0070] like Figure 5C As shown, non-audio message user input may include selection input for pre-written messages, wherein the display is configured to present one or more pre-written messages 508a, 508b, 508c for the user to select. In this example, the pre-written message sent to a third party may be a written form of a message received by the third party (e.g., an icon 508a conveying "an ambulance is needed"). Multiple possible pre-written messages may be displayed for the user to select one or more for sending to a third party. In some examples, a "pre-written message" icon or object may be presented for selection, and once selected, the available pre-written message is displayed to the user as a text representation of the pre-written message for selection.

[0071] Pre-written messages can be represented as symbols 508b that can be selected by the user, where the symbol indicates a message to be sent to a third party (e.g., a "thumbs up" symbol to convey the message "we're okay" to a third party, or a "police car" symbol to indicate that a police response is needed).

[0072] Pre-written messages can be graphically represented as “sign language images” 508c, such as images or animations of people communicating in British Sign Language (BSL), American Sign Language (ASL), or other acceptable sign language 508c that the user can choose, and the corresponding message is then sent to a third party (by providing a response associated with the selected image 508c in written text or audio form). For example, a specific sign language (e.g., British BSL, American ASL) can be preset by the user or can be automatically determined based on the user profile.

[0073] In summary, if the non-audio message user input includes a selection of one or more pre-written messages 508a to 508c, the one or more pre-written messages 508a to 508c displayed for selection can be one or more of the following: a text pre-written message 508a; an image-based graphic pre-written message 508b; and an image-based sign language pre-written message 508c. Advantageously, the pre-written messages 508a to 508c that the user can select to communicate with a third party can have different forms, which can be set as user preference in the driver's vehicle preference settings. For example, some users may be most accustomed to reading text messages to understand the content to be sent to a third party, while another user may be most accustomed to graphic messages such as symbols, and a user whose first language is sign language may be most accustomed to selecting a pre-written message represented as sign language.

[0074] If the non-audio message user input includes a selection of one or more pre-written messages 508a to 508c, the content of one or more pre-written messages may depend on one or more of the following: the type of the third party with which it initiated the communication connection is a fault service operator or an emergency service operator; and one or more sensor types, wherein the assistance request signal is initiated by one or more sensors of the vehicle having one or more sensor types detecting a collision event. Advantageously, if the message presented for selection is a pre-written message, it can be selected and presented according to the circumstances of the third-party call. For example, if the call is with a fault service third party, the pre-written message may include a reference number to the driver's fault insurance policy, or may include a selection of options related to the nature of the fault (e.g., suspected fault, sensor activation, or the location where the vehicle has stopped). As another example, if the call is with emergency services, the pre-written message may include health or communication data, such as the user's hearing level or other health information, and may include information about the number of passengers and any suspected injuries or the severity of injuries. As another example, if the pre-written message depends on the sensor type of the sensor that detected the collision event, the pre-written message could indicate that a forward collision occurred if a sensor at the front of the vehicle was activated, or that a tire blowout occurred on a rear tire, as detected by a sensor at the rear tire.

[0075] If the non-audio message user input includes a selection of one or more pre-written messages 508a to 508c, the control system can be configured to: receive the non-audio message user input indicating a message to be provided to a third party in response to a user selection made by a displayed visual indicator; and transmit the message corresponding to the selected pre-written text message to the third party. Advantageously, the displayed visual indicator can act as an input object that the user can interact with to transmit the corresponding message to the third party. Such an arrangement provides a simple and clear user interface for use by a user who may be in an urgent or stressful situation and finds simple information presentation and easy-to-use assistance methods beneficial.

[0076] Pre-written messages can be displayed as image-based messages 508b, 508c, and the control system can be configured to convert the image-based messages into text messages before transmitting them to a third party. Advantageously, users can select an image representing their desired response to a third party, and this image can be converted into a text message for the third-party operator (and possibly into a voice message via text-to-speech conversion for subsequent conversion). For example, if a user is deaf, they may find it easiest to select an image representing a sign language representation of the message they wish to convey, and if the call handler cannot understand sign language, the sign language representation can be converted into text and / or voice for the call handler.

[0077] In some examples, such as Figure 5C The visual indicator 502b shown may include one or more of the following: a text message describing the event that caused the communication connection to be initiated (the example statement shown is “!999 call in progress!”); and an image indicating the event that caused the communication connection to be initiated. Advantageously, the visual indicator 502b presented to the user may be text- or image-based (e.g., sign language representation of a message) indicating that a call has been made to a third party and may indicate some information about the reason or nature of the call. In some examples, the display of objects 504, 506, 508a to 508c themselves may serve as visual indicators 502, 502b that the user can interact with to provide a response. An example may be the display of one or more buttons, which respectively state, for example, “Need 999 help – accident,” “No medical help needed but troubleshooting assistance needed,” or “Error alert, no assistance needed.”

[0078] Advantageously, once the user is informed by the display of visual cues 502, 502b that a call is in progress and a response is available, there are different possible ways 504, 506, 508a to 508c in which the user can provide a response to a third party.

[0079] Initiating a communication connection between a vehicle and a third party may include establishing a two-way audio call between the vehicle and the third party. Advantageously, visual indications 502, 502b and reception of user responses may be performed in conjunction with the ongoing audio call to complement existing audio call configurations used for reporting faults or emergencies.

[0080] In some examples, the message sent to the third party may include: an indication that no assistance from the third party is to be provided; and a request to terminate an established two-way audio call. When an audio call is made to a third party but the user does not actually need assistance from the third party, the user can clearly indicate that assistance is not actually needed. For example, the system may be activated unintentionally or during a demonstration of vehicle features in a showroom, or sensors may record an incident even if the vehicle is drivable and the occupants are uninjured. In these cases, although an emergency response to the vehicle may not be necessary, dispatching an emergency responder may be the default action if the third party does not receive a voice response from the driver during the ongoing audio call. Therefore, advantageously, the user can recognize from a visual indication that an assistance call has been initiated, and the user can prevent the dispatch of emergency or breakdown services by communicating with the third party to explain the situation when emergency or breakdown services are not actually needed.

[0081] In some examples, the control system 100 can be configured to receive message signaling from a third party configured to provide audio output from the vehicle's audio output device; convert the message signaling configured to provide audio output into visual output configured to provide visual output from the vehicle's display device; and cause the display device to output visual output corresponding to the received message signaling. In other words, the control system 100 may have or be able to access speech-to-text or speech-to-sign language conversion to convert third-party voice messages into visual representations. Advantageously, the control system is capable of receiving audio input from a third-party caller and converting that audio input into a visual representation for the user; for example, if the user is deaf or has a hearing impairment, the user can still understand what the third party is saying by receiving messages as written text messages or messages in video messages displayed in sign language.

[0082] Upon receiving an assistance request signal, the control system 100 can be configured to automatically transmit one or more of the following to a third party: the vehicle's current location; identification data of the registered vehicle owner; and health data of one or more occupants of the vehicle. Such information can be provided automatically without requiring the vehicle occupants to provide it, in order to help the third party understand the reason for the call and provide appropriate assistance.

[0083] In addition to displaying visual indicators 502 on the vehicle's display screen, the control system 100 may also be configured to output one or more of the following: visual warning indicators (e.g., flashing images, symbols, or lights); and tactile warning indicators (e.g., vibrations of the steering wheel, seat, gearshift, or other objects in contact with the vehicle occupants). Such additional indicators can help draw the user's attention to the situation and the ongoing call.

[0084] Figure 6 An example vehicle system 600 according to the examples disclosed herein is shown. The vehicle system 600 includes a control system 100, a display device 500, and a communication module 650, both of which are connected to the control system 100. The communication module 650 is configured to facilitate communication between the vehicle and a third party 660. Advantageously, the control system 100 may be configured for use with the display device 500 in the vehicle, such as an infotainment device or a central display, and is configured for use with the communication module 650 to manage the sending and receiving of audio and other messages with the third party.

[0085] Figure 7 An example method 700 according to the examples disclosed herein is shown. Method 700 is for controlling vehicle 10 (e.g., Figure 1 The method 700 is a computer-implemented method for the assistance system of the vehicle 10 shown. Figure 2 and Figure 3 The control system 100 shown is or Figure 6 The vehicle system 600 shown is executed. Specifically, refer to... Figure 3 The memory 130 may include computer-readable instructions that, when executed by the processor 120, perform a method 700 according to an embodiment of the present invention. The method includes: receiving an assistance request signal 702 indicating an assistance request event, the assistance request signal being configured to initiate a communication connection between a vehicle and a third party; sending a signal 704 to a display screen of the vehicle upon receiving the assistance request signal, wherein the signal causes the display screen to display a visual indicator, and the visual indicator indicating the initiation of the communication connection and the availability of transmitting a response to the third party by providing a non-audio message input; receiving a non-audio message input 706 indicating a message to be provided to the third party in response to the display of the visual indicator; and transmitting the message to the third party 708. Computer-readable instructions are provided that, when executed by a computer, are configured to perform actions such as... Figure 7 The method shown herein or other methods disclosed herein. For example, instructions may be stored on memory 130 and executed by processor 120, such as... Figure 3 As shown in the image.

[0086] It should be understood that various changes and modifications can be made to this invention without departing from the scope of this application.

[0087] This invention may be further defined in the following numbered clauses:

[0088] 1. A control system for an assistance system for controlling a vehicle, the control system comprising one or more processors, said one or more processors being configured to:

[0089] Receive an assistance request signal indicating an assistance request event, the assistance request signal being configured to initiate a communication connection between the vehicle and a third party;

[0090] Upon receiving the assistance request signal, a signal is sent to the vehicle's display screen, wherein the signal causes the display screen to show a visual indicator indicating the initiation of the communication connection and the availability of transmitting a response to the third party by providing non-audio message user input;

[0091] Receiving a non-audio message user input in response to the display of the visual indicator to provide a message to the third party; and

[0092] The message corresponding to the non-audio message user input will be transmitted to the third party.

[0093] 2. The control system according to Clause 1, wherein the assistance request signal includes one or more of the following:

[0094] A fault assistance request signal generated in response to input provided by a user of the vehicle to the vehicle's fault assistance input device, or a fault assistance request signal generated in response to a vehicle fault event determined by the vehicle's fault module; and

[0095] An emergency assistance request signal is generated in response to an input provided by a user of the vehicle to the vehicle's emergency assistance input device, or in response to one or more sensors of the vehicle detecting a collision event.

[0096] 3. A control system according to Clause 1 or 2, wherein the non-audio message user input instructing the message to be provided to the third party includes one or more of the following:

[0097] An audio activation message input provides the third party with a message indicating that the message sender has a hearing impairment, and activates the vehicle's audio receiving device, which is configured to receive and provide a voice audio message from the message sender for transmission to the third party after activation and after providing the message indicating that the message sender has a hearing impairment.

[0098] Free-text message input, wherein the vehicle includes a free-text input device configured to receive text messages input by a user for transmission to the third party; and

[0099] Pre-written message input, wherein the display screen is configured to present one or more pre-written messages for user selection and transmission to the third party.

[0100] 4. The control system according to Clause 3, wherein the non-audio message user input includes the pre-written message input, and wherein the one or more pre-written messages are displayed as one or more of the following for selection:

[0101] Text pre-written messages;

[0102] Image-based pre-written messages; and

[0103] Image-based sign language pre-written messages.

[0104] 5. The control system according to Clause 3 or Clause 4, wherein the non-audio message user input includes the pre-written message input, and wherein the content of the one or more pre-written messages depends on one or more of the following:

[0105] Is the type of the third party that initiated the communication connection a fault service provider or an emergency service provider; and / or

[0106] The assistance request signal is initiated when one or more sensors of the vehicle detect a collision event using one or more sensor types.

[0107] 6. The control system according to any one of clauses 3 to 5, wherein the non-audio message user input includes the pre-written message input, and wherein the control system is configured to:

[0108] The receiving instruction is a non-audio message input by the user in response to a user selection made by the displayed visual indicator to be provided to the third party; and

[0109] The message corresponding to the selected pre-written text message is transmitted to the third party.

[0110] 7. The control system according to Clause 6, wherein the pre-written message is displayed as an image-based message, and the control system is configured to convert the message from an image-based message into a text message before transmitting the message to the third party.

[0111] 8. The control system according to any of the preceding clauses, wherein the visual indicator includes one or more of the following:

[0112] A text message describing the event that caused the initiation of the communication connection; and

[0113] An image indicating the event that caused the initiation of the communication connection.

[0114] 9. The control system according to any of the preceding clauses, wherein initiating a communication connection between the vehicle and a third party includes establishing a two-way audio call between the vehicle and the third party.

[0115] 10. A control system according to any of the preceding clauses, wherein the messages transmitted to the third party include:

[0116] There were no instructions to provide assistance from the aforementioned third party; and

[0117] Request to terminate an established two-way audio call.

[0118] 11. A control system according to any of the preceding clauses, wherein the control system is configured to:

[0119] Receive message signaling from the third party, the message signaling being configured to provide audio output from the vehicle's audio output device;

[0120] The message signaling is converted from being configured to provide audio output to being configured to provide visual output from the vehicle's display device; and

[0121] The display device outputs a visual output corresponding to the received message signaling.

[0122] 12. A vehicle system comprising:

[0123] Control system as described in any of the foregoing clauses;

[0124] The display device; and

[0125] A communication module configured to facilitate communication between the vehicle and the third party.

[0126] 13. A vehicle comprising the system described in Clause 12 or the control system described in Clauses 1 to 11.

[0127] 14. A computer-implemented method for controlling a vehicle assistance system, the method comprising:

[0128] Receive an assistance request signal indicating an assistance request event, the assistance request signal being configured to initiate a communication connection between the vehicle and a third party;

[0129] Upon receiving the assistance request signal, a signal is sent to the vehicle's display screen, wherein the signal causes the display screen to show a visual indicator indicating the initiation of the communication connection and the availability of transmitting a response to the third party by providing non-audio message input;

[0130] Receiving a non-audio message input indicating a message to be provided to the third party in response to the display of the visual indicator; and

[0131] The message is then transmitted to the third party.

[0132] 15. A computer-readable instruction that, when executed by one or more processors, causes the one or more processors to perform the method described in accordance with clause 14.

Claims

1. A control system for an assistance system for controlling a vehicle, the control system comprising one or more processors, said one or more processors being configured to: Receive an assistance request signal indicating an assistance request event, the assistance request signal being configured to initiate a communication connection between the vehicle and a third party; Upon receiving the assistance request signal, a signal is sent to the vehicle's display screen, wherein... The signal causes the display screen to show a visual indicator that indicates the initiation of the communication connection and the availability of transmitting a response to the third party by providing non-audio message user input; Receiving a non-audio message user input in response to the display of the visual indicator to provide a message to the third party; and The message corresponding to the non-audio message user input will be transmitted to the third party.

2. The control system according to claim 1, wherein, The one or more processors are collectively configured to receive an assistance request signal from the vehicle emergency call system indicating the assistance request event; And, based on the assistance request signal received from the emergency call system, send the signal to the vehicle's display screen.

3. The control system according to claim 1 or 2, wherein, The assistance request signal includes one or more of the following: An emergency assistance request signal is generated in response to an input provided by a user of the vehicle to the vehicle's emergency assistance input device, or in response to one or more sensors of the vehicle detecting a collision event.

4. The control system according to any of the preceding claims, wherein, The non-audio message user input that instructs the user to provide a message to the third party includes one or more of the following: An audio activation message input is provided to the third party, indicating that the message sender has a hearing impairment, and the audio receiving device of the vehicle is activated. The audio receiving device is configured to receive a voice audio message from the message sender and provide the voice audio message for transmission to the third party after activation and after providing the message indicating that the message sender has a hearing impairment. Free-text message input, wherein the vehicle includes a free-text input device configured to receive text messages input by a user for transmission to the third party; and Pre-written message input, wherein the display screen is configured to present one or more pre-written messages for user selection and transmission to the third party, optionally wherein the one or more pre-written messages are displayed for selection as one or more of the following: Text pre-written messages; Image-based pre-written messages; and Image-based sign language pre-written messages.

5. The control system according to claim 4, wherein, The non-audio message user input includes the pre-written message input, and wherein the content of the one or more pre-written messages depends on one or more of the following: Is the type of the third party that initiated the communication connection a fault service provider or an emergency service provider; and / or The assistance request signal is initiated when one or more sensors of the vehicle detect a collision event using one or more sensor types.

6. The control system according to any of the preceding claims, wherein, The non-audio message user input includes a hearing impairment or deafness option selectable on the display of the visual indicator.

7. The control system according to any of the preceding claims, wherein, The one or more processors are also collectively configured to: receive a user information signal indicating that the vehicle user has hearing impairment or deafness, wherein the message corresponding to the non-audio message user input includes the information that the vehicle user has hearing impairment or deafness.

8. The control system according to claim 7, wherein, Information about the vehicle user having hearing impairment or deafness is stored in the vehicle's user profile.

9. The control system according to any one of claims 6 to 8, wherein, The signal to the display screen causes visual indicators for other vehicle users who have hearing impairments or deafness.

10. The control system according to any of the preceding claims, wherein, The message transmitted to the third party includes: There were no instructions to provide assistance from the aforementioned third party; and Request to terminate an established two-way audio call.

11. The control system according to any of the preceding claims, wherein, The control system is configured to: Receive message signaling from the third party, the message signaling being configured to provide audio output from the vehicle's audio output device; The message signaling is changed from being configured to provide audio output to being configured to provide visual output from the vehicle's display device; as well as The display device outputs a visual output corresponding to the received message signaling.

12. A vehicle system comprising: The control system according to any of the preceding claims The display device; as well as A communication module configured to facilitate communication between the vehicle and the third party.

13. A vehicle comprising the system according to claim 12 or the control system according to claims 1 to 11.

14. A computer-implemented method for controlling a vehicle assistance system, the method comprising: Receive an assistance request signal indicating an assistance request event, the assistance request signal being configured to initiate a communication connection between the vehicle and a third party; Upon receiving the assistance request signal, a signal is sent to the vehicle's display screen, wherein the signal causes the display screen to show a visual indicator indicating the initiation of the communication connection and the availability of transmitting a response to the third party by providing non-audio message input; Receiving a non-audio message input indicating a message to be provided to the third party in response to the display of the visual indicator; and The message is then transmitted to the third party.

15. A computer-readable instruction that, when executed by one or more processors, causes the one or more processors to perform the method according to claim 14.

Images