Contextual external display

Abstract

A system for providing messages on an external display of a vehicle includes: a plurality of in-vehicle sensors adapted to collect data relating to an environment surrounding the vehicle and objects within the environment surrounding the vehicle; a system controller in communication with the plurality of in-vehicle sensors and adapted to receive data from the plurality of in-vehicle sensors relating to an environment around the vehicle and objects within the environment around the vehicle and identify current contextual characteristics of the vehicle that need to be interpreted to a third party within the environment around the vehicle; and an external display in communication with the system controller, the system controller further adapted to display, via the external display, an interpretation message relating to a current contextual feature of the vehicle to a third party within the environment around the vehicle.

Description

Technical Field

[0001] The present disclosure relates to a system and method for providing information about a vehicle's condition or action to an entity external to the vehicle. Background Art

[0002] Vehicles are equipped with many sensors to monitor contextual characteristics of the vehicle and the environment around the vehicle. Current systems do not use such data to determine when interpretations of contextual characteristics of the vehicle can be adapted to entities external to the vehicle and provide interpretation messages to such entities via external human-machine interfaces.

[0003] Therefore, while current systems and methods achieve their intended purposes, a need exists for a new and improved system and method for providing an interpretive message to an entity external to a vehicle to provide such entity with information regarding contextual characteristics of the vehicle. Summary of the invention

[0004] According to several aspects of the present disclosure, a method for providing a message on an external display of a vehicle includes: using multiple on-board sensors to collect data related to the environment around the vehicle and objects within the environment around the vehicle, using a system controller that communicates with the multiple on-board sensors to receive the data related to the environment around the vehicle and objects within the environment around the vehicle, using the system controller to identify current situational features of the vehicle that need to be explained to a third party within the environment around the vehicle, and using the external display to display an explanation message related to the current situational features of the vehicle to the third party within the environment around the vehicle.

[0005] According to another aspect, identifying, with the system controller, current contextual features of the vehicle requiring explanation to third parties within the environment surrounding the vehicle further comprises receiving manually created interpretation information from an occupant of the vehicle via a human machine interface (HMI).

[0006] According to another aspect, using a system controller to identify current situational features of the vehicle that need to be explained to third parties within the environment surrounding the vehicle also includes: using a computer vision algorithm to identify actions taken by third parties within the environment surrounding the vehicle, and using a machine learning algorithm to determine that the identified actions taken by third parties within the environment surrounding the vehicle indicate that current situational features of the vehicle need to be explained to such third parties within the environment surrounding the vehicle.

[0007] According to another aspect, using a system controller to identify current situational features of the vehicle that need to be explained to third parties within the environment surrounding the vehicle also includes: using a computer vision algorithm to identify the location and activities of the vehicle, and using a machine learning algorithm to determine that the location and activities of the vehicle need to be explained to third parties within the environment surrounding the vehicle.

[0008] According to another aspect, identifying, with the system controller, current contextual features of the vehicle that require explanation to a third party within the environment surrounding the vehicle further includes: utilizing the system controller in communication with at least one physiological sensor associated with an occupant within the vehicle, receiving data related to a medical condition of the occupant, and identifying, with the system controller, a medical emergency.

[0009] According to another aspect, identifying, using a system controller, current situational features of the vehicle that require explanation to a third party within the environment surrounding the vehicle further includes: utilizing a computer vision algorithm and a machine learning algorithm within the system controller that communicate with a database adapted to store data of past events to analyze real-time data collected by a plurality of on-board sensors, utilizing a machine learning algorithm to calculate a probability that current situational features of the vehicle require explanation to a third party within the environment surrounding the vehicle, and determining that an explanation is required when the calculated probability exceeds a preset threshold.

[0010] According to another aspect, displaying an explanatory message related to current situational features of the vehicle to a third party within the environment surrounding the vehicle using an external display also includes one of: automatically displaying the explanatory message using the external display, or prompting the vehicle occupant with the explanatory message via a human-machine interface (HMI), and displaying the explanatory message using the external display after receiving a confirmation from the vehicle occupant via the HMI.

[0011] According to another aspect, the method further includes determining content of the explanation message using a machine learning algorithm and a graphical image generator after identifying, using a system controller, current contextual features of the vehicle that require explanation to a third party within an environment surrounding the vehicle.

[0012] According to another aspect, determining the content of the explanatory message using the machine learning algorithm and the graphical image generator further includes calculating, using the machine learning algorithm, an estimated time that current contextual features of the vehicle will persist and including the estimated time in the explanatory message.

[0013] According to several aspects of the present disclosure, a system for providing a message on an external display of a vehicle includes: a plurality of on-board sensors adapted to collect data related to an environment surrounding the vehicle and objects within the environment surrounding the vehicle; a system controller communicating with the plurality of on-board sensors and adapted to receive data related to the environment surrounding the vehicle and objects within the environment surrounding the vehicle from the plurality of on-board sensors and identifying current situational features of the vehicle that need to be explained to a third party within the environment surrounding the vehicle; and an external display communicating with the system controller, the system controller further adapted to display an explanation message related to the current situational features of the vehicle to a third party within the environment surrounding the vehicle via the external display.

[0014] According to another aspect, the system further includes a human machine interface (HMI), wherein upon identifying a current contextual feature of the vehicle that requires explanation to a third party within the environment surrounding the vehicle, the system controller is further adapted to receive a manually created explanation message from an occupant of the vehicle via the HMI.

[0015] According to another aspect, the system further comprises a computer vision algorithm within the system controller and a machine learning algorithm in communication with the system controller, wherein, when identifying current situational features of the vehicle that require explanation to third parties in the environment surrounding the vehicle, the system controller is further adapted to identify, using the computer vision algorithm, actions taken by third parties within the environment surrounding the vehicle, and to determine, using the machine learning algorithm, that the identified actions taken by third parties within the environment surrounding the vehicle indicate that current situational features of the vehicle require explanation to such third parties within the environment surrounding the vehicle.

[0016] According to another aspect, the system further comprises a computer vision algorithm within the system controller and a machine learning algorithm in communication with the system controller, wherein, when identifying current situational features of the vehicle that require explanation to a third party within the environment surrounding the vehicle, the system controller is further adapted to identify the location and activity of the vehicle using the computer vision algorithm, and to determine that the location and activity of the vehicle requires explanation to the third party within the environment surrounding the vehicle using the machine learning algorithm.

[0017] According to another aspect, the system further includes at least one physiological sensor associated with an occupant within the vehicle and in communication with the system controller, wherein, upon identifying a current situational feature of the vehicle that requires explanation to a third party within the environment surrounding the vehicle, the system controller is further adapted to receive data related to a medical condition of the occupant from the at least one physiological sensor associated with the occupant within the vehicle and identify a medical emergency.

[0018] According to another aspect, the system further comprises a computer vision algorithm within a system controller and a machine learning algorithm in communication with the system controller and with a database adapted to store data of past events, wherein, when identifying current situational features of the vehicle that require explanation to a third party within the environment surrounding the vehicle, the system controller is further adapted to analyze real-time data collected by a plurality of on-board sensors using the computer vision algorithm, calculate the probability that the current situational features of the vehicle require explanation to a third party within the environment surrounding the vehicle using the machine learning algorithm using the data of past events, and determine that an explanation is required when the calculated probability exceeds a preset threshold.

[0019] According to another aspect, the system further includes a human-machine interface (HMI), wherein, when an explanatory message related to a current situational feature of the vehicle is displayed to a third party within the environment surrounding the vehicle using the external display, the system controller is further adapted to one of: automatically display the explanatory information using the external display, or prompt the explanatory message to an occupant of the vehicle via the HMI, and display the explanatory message using the external display upon receiving a confirmation from the occupant of the vehicle via the HMI.

[0020] According to another aspect, after identifying current contextual features of the vehicle that require explanation to third parties within the environment surrounding the vehicle, the system controller is further adapted to utilize a machine learning algorithm and a graphical image generator to determine content of the explanation message.

[0021] According to another aspect, when determining the content of the interpretive message using the machine learning algorithm and the graphic image generator, the system controller is further adapted to calculate, using the machine learning algorithm, an estimated time that the current contextual features of the vehicle will persist and include the estimated time in the interpretive message.

[0022] Further areas of applicability will become apparent from the description provided herein.It should be understood that the description and specific examples are intended for purposes of illustration only and are not intended to limit the scope of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The drawings described herein are for illustration purposes only and are not intended to limit the scope of the present disclosure in any way.

[0024] Figure 1 is a schematic diagram of a vehicle having a system for providing a hazardous object warning to an occupant before the occupant exits the vehicle according to an exemplary embodiment;

[0025] Figure 2 is a schematic diagram of a system according to an exemplary embodiment;

[0026] Figure 3 is a schematic diagram of a vehicle moving slowly due to the presence of a slow-moving vehicle ahead;

[0027] Figure 4 It is a schematic picture of a vehicle parked at a gas pump;

[0028] Figure 5 It is a schematic picture of a vehicle parked at a city mailbox;

[0029] Figure 6 yes Figure 5 a schematic rear view of the vehicle shown;

[0030] Figure 7 is a schematic picture of a vehicle parked on the side of a highway displaying an explanatory message that the vehicle has a flat tire;

[0031] Figure 8 is a schematic image of a vehicle parked on the side of a highway displaying an explanatory message requesting emergency assistance; and

[0032] Fig. 9 is a flow chart showing a method according to an exemplary embodiment.

[0033] The drawings are not necessarily drawn to scale, and some features may be exaggerated or minimized to show details of particular components. In some cases, well-known components, systems, materials, or methods are not described in detail to avoid obscuring the present disclosure. Therefore, the specific structural and functional details disclosed herein should not be interpreted as limiting, but merely as a basis for the claims and as a representative basis for teaching those skilled in the art to adopt the present disclosure in different ways. DETAILED DESCRIPTION

[0034] The following description is merely exemplary in nature and is not intended to limit the present disclosure, application or use. In addition, it is not intended to be constrained by any express or implied theory presented in the aforementioned technical field, background technology, content of the invention or the following specific embodiments. It should be understood that throughout the drawings, the corresponding reference numerals represent similar or corresponding parts and features. As used herein, the term "module" refers to any hardware, software, firmware, electronic control component, processing logic and / or processor device, individually or in any combination, including but not limited to: application specific integrated circuits (ASICs), electronic circuits, processors (shared, dedicated or groups) and memories that execute one or more software or firmware programs, combinational logic circuits and / or other suitable components that provide the described functions. Although the drawings shown herein depict examples with certain element arrangements, additional intermediate elements, devices, features or components may be present in actual embodiments. It should also be understood that the drawings are merely illustrative and may not be drawn to scale.

[0035] As used herein, the term "vehicle" is not limited to automobiles. Although this article mainly describes the present technology in conjunction with automobiles, the present technology is not limited to automobiles. These concepts can be used in a variety of applications, such as aircraft, ships, other vehicles, and consumer electronic components.

[0036] According to an exemplary embodiment, Figure 1A vehicle 10 is shown with an associated system 11 for providing an explanatory message on an external display of the vehicle 10 in accordance with various embodiments. Generally speaking, the system 11 works in conjunction with other systems within the vehicle 10 to display an explanatory message that provides information to third parties outside the vehicle, informing such third parties of current contextual features of the vehicle 10. The vehicle 10 generally includes a chassis 12, a body 14, front wheels 16, and rear wheels 18. The body 14 is disposed on the chassis 12 and generally surrounds components of the vehicle 10. The body 14 and the chassis 12 may together form a frame. The front wheels 16 and the rear wheels 18 are each rotatably connected to the chassis 12 near respective corners of the body 14.

[0037] In various embodiments, the vehicle 10 is an autonomous vehicle and the system 11 is included in the autonomous vehicle 10. The autonomous vehicle 10 is, for example, a vehicle 10 that is automatically controlled to transport passengers from one location to another. Although the vehicle 10 is depicted as a passenger car in the illustrated embodiment, it should be understood that any other vehicle may also be used, including motorcycles, trucks, sport utility vehicles (SUVs), recreational vehicles (RVs), etc. In an exemplary embodiment, the vehicle 10 is equipped with a so-called level four or level five automation system. The level four system represents "high automation" and refers to the driving mode-specific performance of the autonomous driving system in all aspects of dynamic driving tasks, even if the human driver does not respond appropriately to the intervention request. The level five system represents "full automation" and refers to the full-time performance of the autonomous driving system in all aspects of dynamic driving tasks under all road and environmental conditions that the human driver can manage. The novel aspects of the present disclosure are also applicable to non-autonomous driving vehicles.

[0038] As shown, the vehicle 10 generally includes a propulsion system 20, a transmission system 22, a steering system 24, a braking system 26, a sensor system 28, an actuator system 30, at least one data storage device 32, a vehicle controller 34, and a wireless communication module 36. In an embodiment where the vehicle 10 is an electric vehicle, the transmission system 22 may not be present. In various embodiments, the propulsion system 20 may include an internal combustion engine, an electric motor such as a traction motor, and / or a fuel cell propulsion system. The transmission system 22 is configured to transmit power from the propulsion system 20 to the front wheels 16 and rear wheels 18 of the vehicle according to a selectable speed ratio. According to various embodiments, the transmission system 22 may include a stepped automatic transmission, a continuously variable transmission, or other suitable transmissions. The braking system 26 is configured to provide braking torque to the front wheels 16 and rear wheels 18 of the vehicle. In various embodiments, the braking system 26 may include friction brakes, wire control brakes, regenerative braking systems (such as motors), and / or other suitable braking systems. The steering system 24 affects the position of the front wheels 16 and rear wheels 18. Although depicted as including a steering wheel for illustrative purposes, in some embodiments contemplated within the scope of the present disclosure, the steering system 24 may not include a steering wheel.

[0039] The sensor system 28 includes one or more sensing devices 40a-40n that sense observable conditions of the external environment and / or the internal environment of the autonomous vehicle 10. The sensing devices 40a-40n may include, but are not limited to, radar, lidar, global positioning system, optical camera, thermal imager, ultrasonic sensor, and / or other sensors. The camera may include two or more digital cameras spaced at a selected distance from each other, wherein the two or more digital cameras are used to obtain stereo images of the surrounding environment in order to obtain a three-dimensional image or map. The plurality of sensing devices 40a-40n are used to determine information about the environment around the vehicle 10. In an exemplary embodiment, the plurality of sensing devices 40a-40n include at least one of a motor speed sensor, a motor torque sensor, an electric drive motor voltage and / or current sensor, an accelerator pedal position sensor, a coolant temperature sensor, a cooling fan speed sensor, and a transmission oil temperature sensor. In another exemplary embodiment, the plurality of sensing devices 40a-40n also include sensors for determining information about the environment around the vehicle 10, such as an ambient air temperature sensor, an atmospheric pressure sensor, and / or a camera and / or video camera positioned to observe the environment in front of the vehicle 10. In another exemplary embodiment, at least one of the plurality of sensing devices 40 a - 40 n is capable of measuring distances in the environment surrounding the vehicle 10 .

[0040] In a non-limiting example where the plurality of sensing devices 40a-40n include cameras, the plurality of sensing devices 40a-40n use an image processing algorithm to measure distances, the image processing algorithm being configured to process images from the cameras and determine distances between objects. In another non-limiting example, the plurality of vehicle sensors 40a-40n include a stereo camera with distance measurement capabilities. In one example, at least one of the plurality of sensing devices 40a-40n is fixed to the interior of the vehicle 10, for example, in the roof lining of the vehicle 10, with a view through the windshield of the vehicle 10. In another example, at least one of the plurality of sensing devices 40a-40n is a camera fixed to the exterior of the vehicle 10 (e.g., on the roof of the vehicle 10), with a view of the environment surrounding the vehicle 10 and adapted to collect information (images) related to the environment outside the vehicle 10. It should be understood that various other types of sensing devices, such as LiDAR sensors, ultrasonic ranging sensors, radar sensors, and / or time-of-flight sensors are within the scope of the present disclosure. The actuator system 30 includes one or more actuator devices 42 a - 42 n that control one or more vehicle 10 features, such as, but not limited to, the propulsion system 20 , the transmission system 22 , the steering system 24 , and the braking system 26 .

[0041] The vehicle controller 34 includes at least one processor 44 and a computer readable storage device or medium 46. The at least one data processor 44 can be any custom or commercial processor, a central processing unit (CPU), a graphics processing unit (GPU), an auxiliary processor of several processors associated with the vehicle controller 34, a semiconductor-based microprocessor (in the form of a microchip or chipset), a macroprocessor, any combination thereof, or any device generally used to execute instructions. The computer readable storage device or medium 46 can include volatile storage devices, and non-volatile storage devices, such as read-only memory (ROM), random access memory (RAM), and keep alive memory (KAM). KAM is a persistent or non-volatile memory that can be used to store various operating variables when the at least one data processor 44 is powered off. The computer readable storage device or medium 46 can be implemented using any of a variety of known memory devices, such as PROM (programmable read-only memory), EPROM (electrical PROM), EEPROM (electrically erasable PROM), flash memory, or any other electrical, magnetic, optical, or combination memory device capable of storing data used by the controller 34 in controlling the vehicle 10, some of which represents executable instructions.

[0042] The instructions may include one or more separate programs, each of which includes an ordered list of executable instructions for implementing logical functions. When executed by at least one processor 44, the instructions receive and process signals from the sensor system 28, execute logic, calculations, methods and / or algorithms for automatically controlling components of the vehicle 10, and generate control signals to the actuator system 30 to automatically control components of the vehicle 10 based on the logic, calculations, methods and / or algorithms. Although Figure 1 Only one controller 34 is shown, and embodiments of the vehicle 10 may include any number of controllers 34 that communicate via any suitable communication medium or combination of communication mediums and collaborate to process sensor signals, execute logic, calculations, methods and / or algorithms, and generate control signals 10 that automatically control features of the autonomous vehicle 10.

[0043] In various embodiments, one or more instructions of the vehicle controller 34 are included in the trajectory planning system and, when executed by at least one data processor 44, generate trajectory outputs that solve the kinematic and dynamic constraints of the environment. For example, the instructions receive process sensor and map data as input. The instructions implement a graph-based approach with a customized cost function to handle different road scenarios of cities and highways.

[0044] The wireless communication module 36 is configured to wirelessly transmit information to and from other remote entities 48, such as, but not limited to, other vehicles ("V2V" communication), infrastructure ("V2I" communication), remote systems, remote servers, cloud computers, and / or personal devices. In an exemplary embodiment, the communication system 36 is a wireless communication system configured to communicate via a wireless local area network (WLAN) using the IEEE 802.11 standard, or by using cellular data communications. However, additional or alternative communication methods such as dedicated short-range communication (DSRC) channels are also considered to be within the scope of the present disclosure. DSRC channels refer to one-way or two-way short-range to medium-range wireless communication channels designed specifically for automotive use and a set of corresponding protocols and standards.

[0045] The vehicle controller 34 is a non-universal electronic control device having a pre-programmed digital computer or processor, memory or non-transitory computer-readable medium, and a transceiver [or input / output port] for storing data such as control logic, software applications, instructions, computer code, data, lookup tables, etc. Computer-readable media include any type of media that can be accessed by a computer, such as read-only memory (ROM), random access memory (RAM), hard drive, compact disk (CD), digital video disk (DVD), or any other type of memory. "Non-transitory" computer-readable media excludes wired, wireless, optical or other communication links that transmit temporary electrical or other signals. Non-transitory computer-readable media include media that can permanently store data and media that can store data and then rewrite the data, such as rewritable optical disks or erasable memory devices. Computer code includes any type of program code, including source code, object code, and executable code.

[0046] refer to Figure 2 , shows a schematic diagram of the system 11. The system 11 includes a system controller 34A in communication with a plurality of sensing devices (on-board sensors) 40a-40n. The system controller 34A may be the vehicle controller 34, or the system controller 34A may be a separate controller in communication with the vehicle controller 34. In addition to the plurality of on-board sensors 40a-40n, the system controller 34A also communicates with an external display 50.

[0047] In an exemplary embodiment, the plurality of onboard sensors 40a-40n are adapted to collect data related to the environment around the vehicle 10 and objects within the environment around the vehicle 10. The plurality of onboard sensors 40a-40n detect objects near the vehicle 10 and third-party entities such as other vehicles and people. The data collected by the plurality of onboard sensors 40a-40n is received by the system controller 34A, wherein the system controller 34A is adapted to identify current contextual features of the vehicle 10 that need to be explained to third parties within the environment around the vehicle 10, and to display explanation messages 52 related to the current contextual features of the vehicle 10 to the third parties within the environment around the vehicle 10 via the external display 50.

[0048] Situational features that require explanation to third parties in the environment surrounding the vehicle 10 include situations where the vehicle 10 is located in a certain position or is operating in a manner that may be unusual, or may pose an obstacle to third-party entities in the environment surrounding the vehicle 10. Such situational features are prompts that the system controller 34A needs to interpret the message 52. For example, referring to Figure 3, the situational feature of vehicle 10 is that vehicle 10 is moving slower than the posted speed limit because vehicle 10 is following a slow-moving farm tractor 54. A third-party vehicle 56 is traveling behind vehicle 10, and therefore, it may be helpful to provide explanation message 52 to third-party vehicle 56 to let third-party vehicle 56 know why vehicle 10 is traveling at the posted speed limit. Figure 3 As shown, the external display 50 displays an explanatory message 52 “Slow Moving Vehicle Ahead” to provide information to the third party vehicle 56 as to why the vehicle 10 is moving below the posted speed limit.

[0049] refer to Figure 4 , the vehicle 10 is parked at a gas pump 58. The contextual feature of the vehicle 10 is that the vehicle 10 is parked at the gas pump 58, preventing other vehicles from using the gas pump 58. The external display 50A, 50B displays an explanatory message 52 "Wait 5 minutes" to provide a third-party vehicle with an estimate of how long such third-party vehicle needs to wait before using the gas pump 58.

[0050] The external display 50, 50A, 50B may be any type of display suitable for displaying external messages. In an exemplary embodiment, Figure 3 The external display 50 and Figure 4 The illustrated external display 50A is an interior / exterior display system adapted to illuminate an explanatory message 52 within a window surface 60 of the vehicle 10. The interior / exterior display system may use any suitable transparent display technology to display information onto the window surface 60 of the vehicle 10 for viewing by a third-party entity outside the vehicle 10. In an exemplary embodiment, the interior / exterior display system of the display 50, 50A uses an excitation light projected onto the window surface 60 to illuminate a transparent phosphor within a substrate embedded on the window surface 60. The transparent phosphor is a luminescent particle that fluoresces in response to being excited by the excitation light. This type of interior / exterior display system is discussed in more detail in U.S. Application No.: 17 / 749,464, filed on May 20, 2022, entitled “HYBRID AUGMENTED REALITY HEAD-UP DISPLAY FOR CREATING AN EDGE-TO-EDGE AUGMENTED REALITY VIEW,” the entire contents of which are incorporated herein by reference.

[0051] In another exemplary embodiment, Figure 4 The illustrated external display 50B is any suitable known display screen mounted on or within an exterior surface 62 of the vehicle 10. Those skilled in the art will appreciate that the external display 50 may be any known technology suitable for providing external messaging from the vehicle 10.

[0052] refer to Figure 5 , vehicle 10 is a mail truck parked in front of a city mailbox 64, where a mail carrier 66 will exit vehicle 10 to pick up mail that has been deposited in city mailbox 64. The situational feature of vehicle 10 is that vehicle 10 is parked in front of city mailbox 64, potentially obstructing traffic flow along the street. External display 50 provides an explanatory message 52 "Mail pickup 2 minutes" to provide information to third-party vehicles explaining why vehicle 10 is parked at that location and the estimated time that vehicle 10 will be parked there. This will allow third-party vehicles to understand why vehicle 10 is parked at that location and provide third-party vehicles with information about how long vehicle 10 will be parked at that location, thereby allowing third-party vehicles to choose to wait patiently, or decide to bypass vehicle 10. As Figure 5 As shown, the external display 50 can be positioned on a side of the vehicle 10, or with reference to Figure 6 , the external display 50 can be positioned at the rear of the vehicle 10. The external display 50 can use any window surface 60 of the vehicle 10, or can be mounted in or on any exterior surface of the vehicle 10.

[0053] refer to Figure 7 , the vehicle 10 is located in a lane of a highway due to a flat tire of the vehicle 10. The situational feature of the vehicle 10 is that the vehicle 10 may partially or completely block traffic in the lane of the highway. The external display 50 provides an explanatory message 52 "Flat Tire" to provide information to third-party vehicles explaining why the vehicle 10 is parked in this position.

[0054] In an exemplary embodiment, the system 11 also includes a human machine interface (HMI) 68, wherein the system controller 34A is further adapted to receive an explanation message 52 manually created from an occupant of the vehicle 10 via the HMI 68 when identifying a current contextual feature of the vehicle 10 that requires explanation to a third party within the environment surrounding the vehicle 10. For example, referring again to Figure 5 , when the postman 66 parks the vehicle 10 (mail truck) in front of the city mailbox 64, the postman 66 knows that it takes two minutes to get off the vehicle 10, collect the mail from the city mailbox 64, return to the vehicle 10 and continue. Therefore, the postman 66 manually enters the explanation message 52 "Mail pickup 2 minutes" using the HMI and instructs the system 11 to display the explanation message 52 on the external display 50.

[0055] Similarly, refer again to Figure 7 , the driver of vehicle 10 manually enters the explanation message 52 "Flat Tire" using HMI 68. In this way, when the vehicle is unable to travel along the road, the occupants in vehicle 10 can manually enter an explanation message to provide any information they want to the third party vehicle. Figure 8, an occupant of the vehicle 10 manually enters the interpretation message 52 “Emergency Help Needed” using the HMI 68 to notify third parties that the vehicle requires assistance due to a medical or other emergency.

[0056] In another exemplary embodiment, the system 11 further includes a computer vision algorithm 70 within the system controller 34A and a machine learning algorithm 72 in communication with the system controller 34A, wherein, when identifying current contextual features of the vehicle 10 that require explanation to third parties within the environment surrounding the vehicle 10, the system controller 34A is further adapted to utilize the computer vision algorithm 70 to identify actions taken by third parties within the environment surrounding the vehicle 10, and utilize the machine learning algorithm 72 to determine that the identified actions taken by third parties within the environment surrounding the vehicle 10 indicate that current contextual features of the vehicle 10 require explanation to such third parties within the environment surrounding the vehicle 10.

[0057] Reference again Figure 3 , the system controller 34A identifies situational features, such as the vehicle 10 traveling slower than the posted speed limit, which may indicate a need for an explanation message 52. Additionally, the system controller 34A identifies a third-party vehicle 56 trailing the vehicle 10 using computer vision algorithms 70 and image recognition techniques applied to images of the environment outside the vehicle 10 captured by a camera (at least one of the plurality of onboard sensors 40a-40n). The system controller 34A identifies the trailing of the third-party vehicle 56 using machine learning algorithms 72, particularly in conjunction with data that the vehicle 10 is moving slower than the posted speed limit, as an indication that an explanation message 52 may be needed. Thus, the system controller 34A identifies actions taken by a third-party entity (e.g., the trailing of the third-party vehicle 56) as a query from a third party requesting an explanation message 52, and automatically generates and displays the explanation message 52. Such actions may include, but are not limited to, trailing of a following third-party vehicle, active driving (steering, accelerating / decelerating) of a third-party vehicle, honking of a third-party vehicle, gestures or audible expressions made by passengers or bystanders of the third-party vehicle, and the like. In an exemplary embodiment, the system controller 34A automatically generates the explanation message 52 , prompts the occupant of the vehicle 10 with the explanation message 52 via the HMI 68 , and displays the explanation message 52 using the external display 50 upon receiving a confirmation from the occupant of the vehicle 10 via the HMI 68 .

[0058] In another exemplary embodiment, the system 11 uses a computer vision algorithm 70 within the system controller 34A and a machine learning algorithm 72 in communication with the system controller 34A to identify the location and activity of the vehicle using the computer vision algorithm 70 and to determine, using the machine learning algorithm 72, that the location and activity of the vehicle 10 needs to be explained to a third party within the environment surrounding the vehicle 10 .

[0059] Reference again Figure 5 , the system controller 34A identifies situational features that may indicate a need for an explanation message 52, such as the vehicle 10 being parked along a street. In addition, the system controller 34A uses the computer vision algorithm 70 and image recognition techniques applied to images of the environment outside the vehicle 10 to identify objects within the environment surrounding the vehicle 10, such as a city mailbox 64, street signs, buildings, or other landmarks. The system controller 34A uses the machine learning algorithm 72 to identify the city mailbox 64 and other objects, and together with the data that the vehicle is stopped, determines that an explanation message 52 may be needed. Based on past events stored in a database 74 that is in communication with the system controller 34A, the machine learning algorithm 70 identifies the location where the vehicle 10 is parked, and predicts activities and estimates the time of the activities. Thus, the system controller 34A uses the computer vision algorithm 70 and the machine learning algorithm 72 to identify the location where the vehicle 10 is parked and predict activities and times, and automatically generates and displays the explanation message 52. In an exemplary embodiment, the system controller 34A automatically generates the explanation message 52 , prompts the occupant of the vehicle 10 with the explanation message 52 via the HMI 68 , and displays the explanation message 52 with the external display 50 upon receiving a confirmation from the occupant of the vehicle 10 via the HMI 68 .

[0060] In another exemplary embodiment, the system 11 includes at least one physiological sensor 76 associated with an occupant within the vehicle 10 and in communication with the system controller 34A, wherein the system controller 34A is further adapted to receive data related to a medical condition of the occupant from the at least one physiological sensor 76 and identify a medical emergency when identifying current situational features of the vehicle 10 that require explanation to a third party within the environment surrounding the vehicle 10.

[0061] At least one physiological sensor 76 monitors physiological signals, such as brain waves and heart rate, pulse rate, and breathing rate signals. Although an electroencephalogram (EEG) that monitors brain waves can provide the best physiological signal for assessing the occupant's state of attention, such data is difficult to obtain while driving. Another biosignal is an electrocardiogram (ECG) signal with heart rate and heart rate variability information. Other methods of collecting physiological data include collecting biosignals such as breathing (via sensors located in the seats and / or seat belts), grip strength and photoplethysmography (PPG) (via sensors on the steering wheel), and muscle current (electromyography (EMG)). These biosignals can also be received from wearable devices worn by the occupants, such as smart watches. Various techniques are used to extract meaningful features from the physiological sensor 76 readings, including time series analysis, frequency domain analysis, and spatiotemporal patterns. Machine learning algorithms 72 are used to classify possible medical emergencies for the occupants. Therefore, if Figure 8As shown, the system controller 34A uses data from a plurality of onboard sensors 40a-40n to identify that the vehicle is parked in or near a highway lane and uses a computer vision algorithm 70 and a machine learning algorithm 72 and machine learning techniques to identify physiological signals from at least one physiological sensor 76 that indicate a medical emergency, wherein the system controller determines that the vehicle 10 is stopped because the driver has a medical emergency and automatically generates and displays an explanatory message, such as Figure 8 shown.

[0062] In an exemplary embodiment, the system controller 34A uses the computer vision algorithm 70 and the machine learning algorithm 72 and data related to past events stored in the database 76 to analyze the real-time data collected by the plurality of vehicle-mounted sensors 40a-40n, uses the machine learning algorithm 72 to utilize the data of past events from the database 74 to calculate the probability that the current situational features of the vehicle 10 require explanation to a third party within the environment surrounding the vehicle 10, and when the calculated probability exceeds a preset threshold, determines that the message 52 needs to be explained.

[0063] A database 74 in communication with the system controller 34A is adapted to store data relating to past events in which explanatory messages were manually or automatically generated and displayed, as well as environmental aspects (weather, temperature, precipitation), actions of third-party entities (tailgating, active driving, gestures by an occupant of the vehicle or bystander), and vehicle aspects (position, parked, stopped, engine off) during such past events.

[0064] The machine learning algorithm 72 is adapted to predict the probability that current situational features require interpretation of the message 52 based on real-time data collected by multiple on-board sensors 40a-40n, analysis of captured images using computer vision algorithms 70 and image recognition techniques, and data 74 of past events in a database.

[0065] Various techniques are used to extract meaningful features from sensor readings and data, including time series analysis, frequency domain analysis, and spatiotemporal patterns. The machine learning algorithm 72 can be a long short-term memory (LSTM), a gated recurrent unit (GRU), a decision tree, a random forest, a support vector machine (SVM), a neural network (NN), a K-nearest neighbor algorithm (KNN), a gradient boosting, and a recurrent neural network (RNN). The machine learning algorithm 72 is trained using data collected from multiple different vehicles. A diverse dataset is collected from vehicles equipped with sensors such as GPS, accelerometers, cameras, radars, and LIDARs. The data covers a variety of driving scenarios, including urban, highway, and off-road driving. Before feeding the data to the machine learning model 64, a preprocessing step is performed to remove noise. The machine learning algorithm 72 is trained on a labeled data set and evaluated using various performance metrics (e.g., accuracy, precision, recall, F1 score, and confusion matrix). The hyperparameters of the model are adjusted to obtain the best results. The machine learning algorithm 72 is also trained on the training data and will learn to map the input features to the corresponding probabilities.

[0066] Data from events in which the system controller 34A generates an explanation message 52 (automatically or in response to a manual prompt via the HMI 68) is saved to the database 74 and used to continuously update the machine learning algorithm 72. Thus, the system controller 34A uses the machine learning algorithm 72 and machine learning techniques to predict the probability that an explanation message 52 is needed based on analysis of real-time data collected by the plurality of onboard sensors 40a-40n, analysis of captured images using the computer vision algorithm 70 and image recognition techniques, and data from past events from the database 74.

[0067] For example, refer again to Figure 5, when the vehicle 10 (mail truck) is parked in front of the city mailbox 64, the system controller 34A utilizes computer vision algorithms and images collected from multiple onboard sensors 40a-40n to identify the location of the vehicle 10 (parked in front of the city mailbox 64) and the activity (picking up mail). The system controller 34A uses the machine learning algorithm 72 to determine whether an explanation message is needed. For example, based on data from the database 74, the machine learning algorithm 72 identifies that an explanation message is typically needed when the vehicle 10 stops at this location to pick up mail between 3:00 p.m. and 5:00 p.m. during peak traffic hours. However, based on data from the database 74, if the vehicle 10 is parked at this location during other times, an explanation message is typically not needed. Therefore, if the vehicle 10 is parked at the city mailbox 64 outside of peak traffic hours, the system controller 34A does not automatically generate the explanation message 52. If the probabilistic analysis of the real-time data indicates a critical requirement for the explanation message 52, the system controller 34A can prompt the mail carrier 66 via the HMI 68 and let the mail carrier 66 decide whether they want to display the explanation message 52.

[0068] In another example, again referring to Figure 5 , when the vehicle 10 (mail truck) is parked in front of the city mailbox 64, the system controller 34A uses computer vision algorithms and images collected from multiple on-board sensors 40a-40n to identify the location of the vehicle 10 (parked in front of the city mailbox 64) and the activity (picking up the mail). The system controller 34A uses the machine learning algorithm 72 to determine whether the explanation message 52 is needed and the content of the explanation message 52. For example, based on data from the database 74, the machine learning algorithm 72 identifies that when the vehicle 10 is parked at the city mailbox 64 on Monday, the mail carrier 66 takes an average of 5 minutes to get off the vehicle, pick up the mail, and resume the route. This is because every Monday, when the mail delivery system is not running, the mail accumulates in the city mailbox 64 on the weekend. However, on the remaining days of the week, Tuesday through Friday, when the vehicle 10 is parked at the city mailbox 64, the mail carrier 66 takes an average of less than 2 minutes to get off the vehicle, pick up the mail, and resume the route. Therefore, if the vehicle 10 is parked at the city mailbox 64 during peak traffic hours on Monday, an explanation message including an estimated time of 5 minutes will be automatically generated. On any other day, the explanatory message 52 displayed when the vehicle 10 is parked at the city mailbox 64 will include an estimated time of 2 minutes. The estimated time may be affected by other factors, such as the season, where data from the database 74 indicates that during the winter, the mail carrier 66 takes longer to collect mail from the city mailbox 64. Therefore, if the plurality of sensors 40a-40n detect freezing temperatures or the presence of snow on the ground, the machine learning algorithm 72 will adjust the estimated time accordingly.

[0069] In one exemplary embodiment, after identifying current contextual features of the vehicle 10 that require explanation to third parties within the environment surrounding the vehicle 10, the system controller 34A is further adapted to utilize the machine learning algorithm 72 and the graphic image generator 78 to determine the content of the explanation message 52. The graphic image generator 78 processes information from the system controller 34A and the machine learning algorithm 72 and determines an appropriate explanation message 52 to display.

[0070] refer to Fig. 9 , a method 100 for providing a message on an external display 50 of a vehicle 10 includes: starting at box 102, using multiple on-board sensors 40a-40n to collect data related to the environment around the vehicle 10 and objects within the environment around the vehicle 10; moving to box 104, using a system controller 34A in communication with the multiple on-board sensors 40a-40n to receive data related to the environment around the vehicle 10 and objects within the environment around the vehicle 10; moving to box 106, using the system controller 34A to identify current situational features of the vehicle 10 that need to be explained to third parties within the environment around the vehicle 10; and moving to box 108, using the external display 50 to display an explanation message 52 related to the current situational features of the vehicle 10 to third parties within the environment around the vehicle 10.

[0071] In an exemplary embodiment, identifying, using the system controller 34A at box 106, current contextual features of the vehicle 10 that require explanation to third parties within the environment surrounding the vehicle 10 also includes: moving from box 104 to box 110, receiving an explanation message 52 manually created by an occupant of the vehicle 10 via the human-machine interface (HMI) 68.

[0072] In an exemplary embodiment, identifying, using the system controller 34A at box 106, current contextual features of the vehicle 10 that need to be explained to third parties within the environment surrounding the vehicle 10 also includes: moving from box 104 to box 112, using the computer vision algorithm 70 to identify actions taken by third parties within the environment surrounding the vehicle 10; and moving to box 114, using the machine learning algorithm 72 to determine that the identified actions taken by third parties within the environment surrounding the vehicle 10 indicate that current contextual features of the vehicle 10 need to be explained to such third parties within the environment surrounding the vehicle 10.

[0073] In an exemplary embodiment, identifying current contextual features of the vehicle 10 at box 106 using the system controller 34A that require explanation to third parties within the environment surrounding the vehicle 10 also includes: moving from box 104 to box 116, using the computer vision algorithm 70 to identify the location and activity of the vehicle 10; and moving to box 118, using the machine learning algorithm 72 to determine that the vehicle's location and activity need to be explained to third parties within the environment surrounding the vehicle 10.

[0074] In an exemplary embodiment, identifying, at box 106, using the system controller 34A, current contextual features of the vehicle 10 that require explanation to a third party within the environment surrounding the vehicle 10 also includes: moving from box 104 to box 120, using the system controller 34A in communication with at least one physiological sensor 76 associated with an occupant within the vehicle 10 to receive data related to a medical condition of the occupant; and moving to box 122, using the system controller 34A to identify a medical emergency.

[0075] In an exemplary embodiment, identifying, at box 106, current situational features of the vehicle 10 that require explanation to third parties within the environment surrounding the vehicle 10 using the system controller 34A also includes: moving from box 104 to box 124, analyzing real-time data collected by multiple on-board sensors 40a-40n using a computer vision algorithm 70 and a machine learning algorithm 72 within the system controller 34A and in communication with a database 74 adapted to store data of past events; moving to box 126, calculating, using the machine learning algorithm 72, a probability that the current situational features of the vehicle 10 require explanation to third parties within the environment surrounding the vehicle 10; and moving to box 128, determining that an explanation is required when the calculated probability exceeds a preset threshold.

[0076] In an exemplary embodiment, displaying an explanatory message 52 related to current situational features of the vehicle 10 using the external display 50 to a third party within the environment surrounding the vehicle 10 at box 108 also includes one of: moving from box 106 to box 130, automatically displaying the explanatory message 52 using the external display 50, or moving from box 106 to box 132, prompting an occupant of the vehicle 10 with the explanatory message via a human-machine interface (HMI) 68; and moving from box 132 to box 130, displaying the explanatory message 52 using the external display 50 after receiving a confirmation from the occupant of the vehicle 10 via the HMI 68 at box 132.

[0077] In an exemplary embodiment, the method 100 also includes moving to box 134 after using the system controller 34A at box 106 to determine the content of the explanation message 52 using the machine learning algorithm 72 and the graphic image generator 78.

[0078] In another exemplary embodiment, determining the content of the interpretive message 52 using the machine learning algorithm 72 and the graphical image generator 78 at box 134 also includes: calculating an estimated time that the current situation characteristics of the vehicle 10 will last using the machine learning algorithm 72, and including the estimated time in the interpretive message 52.

[0079] The systems and methods of the present disclosure provide the advantage of providing an explanatory message to a third party when the contextual circumstances of the vehicle indicate that an explanatory message is necessary.

[0080] The description of the present disclosure is merely exemplary in nature, and variations that do not depart from the gist of the present disclosure are intended to fall within the scope of the present disclosure. These variations should not be regarded as departing from the spirit and scope of the present disclosure.

Claims

1. A method for providing a message on an external display of a vehicle, comprising: collecting data related to an environment surrounding the vehicle and data related to objects within the environment surrounding the vehicle using a plurality of onboard sensors; receiving, with a system controller in communication with the plurality of onboard sensors, the data relating to the environment surrounding the vehicle and the data relating to the objects within the environment surrounding the vehicle; identifying, with the system controller, current contextual characteristics of the vehicle that need to be explained to a third party within the environment surrounding the vehicle; as well as An explanatory message related to the current situational features of the vehicle is displayed to the third party within the environment around the vehicle using an external display.

2. The method according to claim 1, wherein: Identifying, with the system controller, current contextual features of the vehicle that require explanation to a third party within the environment surrounding the vehicle further comprises receiving, via a human machine interface (HMI), a manually created explanation message from an occupant of the vehicle.

3. The method according to claim 1, wherein: The identifying, using the system controller, current contextual features of the vehicle that need to be explained to a third party within the environment surrounding the vehicle further comprises: utilizing a computer vision algorithm, identifying actions taken by the third party within the environment surrounding the vehicle; and Using a machine learning algorithm, it is determined that the identified action taken by the third party within the environment around the vehicle indicates that the current contextual characteristics of the vehicle need to be explained to the third party within the environment around the vehicle.

4. The method according to claim 1, wherein: The identifying, using the system controller, current contextual features of the vehicle that need to be explained to a third party within the environment surrounding the vehicle further comprises: utilizing computer vision algorithms to identify the location and movement of the vehicle; and Using the machine learning algorithm, it is determined that the location and activity of the vehicle requires explanation to the third party within the environment surrounding the vehicle.

5. The method according to claim 1, wherein: The identifying, using the system controller, current contextual features of the vehicle that need to be explained to a third party within the environment surrounding the vehicle further comprises: receiving data related to a medical condition of an occupant within the vehicle using the system controller in communication with at least one physiological sensor associated with the occupant; and A medical emergency is identified using the system controller.

6. The method according to claim 1, wherein: The identifying, using the system controller, current contextual features of the vehicle that need to be explained to a third party within the environment surrounding the vehicle further comprises: analyzing real-time data collected by the plurality of onboard sensors using a computer vision algorithm and a machine learning algorithm located within the system controller and in communication with a database adapted to store data of past events; Calculating, using the machine learning algorithm, a probability that current contextual features of the vehicle require explanation to a third party within the environment surrounding the vehicle; and When the calculated probability exceeds a preset threshold, it is determined that an explanation is required.

7. The method according to claim 6, wherein: The displaying, using an external display, of an explanatory message related to the current situational features of the vehicle to a third party within the environment surrounding the vehicle further comprises one of the following: automatically displaying the explanation message using the external display; or The explanation message is prompted to an occupant of the vehicle via a human machine interface (HMI), and upon receiving a confirmation from the occupant of the vehicle via the HMI, the explanation message is displayed using the external display.

8. The method according to claim 7, further comprising: After identifying, using the system controller, current contextual features of the vehicle that require explanation to third parties within the environment surrounding the vehicle, the content of the explanation message is determined using the machine learning algorithm and graphic image generator.

9. The method according to claim 8, wherein: Determining the content of the explanation message using the machine learning algorithm and the graphic image generator also includes: using the machine learning algorithm to calculate an estimated time that the current situational characteristics of the vehicle will last and including the estimated time in the explanation message.

10. A system for providing a message on an external display of a vehicle, comprising: a plurality of on-board sensors adapted to collect data relating to an environment surrounding the vehicle and objects within the environment surrounding the vehicle; a system controller in communication with the plurality of onboard sensors and adapted to: receiving data related to the environment around the vehicle and data related to objects within the environment around the vehicle from the plurality of onboard sensors; as well as identifying current contextual characteristics of the vehicle that need to be explained to a third party within the environment around the vehicle; as well as An external display is in communication with the system controller, the system controller being further adapted to display an explanatory message related to the current contextual features of the vehicle to a third party within the environment surrounding the vehicle via the external display.

Images