Present relevant warnings to the vehicle operator

Abstract

Embodiments include systems and methods executed by a processor in a vehicle to present a relevant alert to a vehicle operator, including receiving a vehicle-to-everything (V2X) communication including information regarding a V2X identified threat, determining whether the vehicle operator is not aware of the V2X identified threat, and in response to determining that the vehicle operator is not aware of the V2X identified threat, determining a display location that is most likely to receive the vehicle operator's attention and generating an alert regarding the V2X identified threat on the determined display location. The generated alert regarding the V2X identified threat on the determined display location may exclude alerts regarding any threat conditions that the processor has determined the vehicle operator is aware of.

Description

[Technical field]

[0001] Related Applications This application claims the benefit of priority from U.S. Patent Application No. 17 / 496,691, filed October 7, 2021, the entire contents of which are incorporated herein by reference. [Background technology]

[0002] Intelligent Transport Systems (ITS) aim to provide services related to different transport modes and improve road safety, traffic efficiency, and energy conservation. ITS networks include advanced telematics and hybrid communications, including Internet Protocol (IP) based communications, as well as ad-hoc direct communications between vehicles, infrastructure, and individuals carrying wireless devices configured for such communications. ITS networks also enable more specific types of communications, such as Vehicle-to-Infrastructure (V2I), Vehicle-to-Network (V2N), Vehicle-to-Vehicle (V2V), Vehicle-to-Pedestrian (V2P), Vehicle-to-Device (V2D), and Vehicle-to-Grid (V2g), all collectively referred to as Vehicle-to-Everything (V2X) communications.

[0003] Vehicles equipped with V2X technology receive messages from nearby entities, including roadside units, vehicles, and individuals carrying wireless devices. The V2X-equipped equipment can analyze and use information from the received messages to improve vehicle safety operation. When appropriate, the V2X-equipped equipment can alert the vehicle operator of safety-critical events or conditions that may pose a threat to the vehicle or vehicle occupants. Summary of the Invention

[0004] Various aspects may include a method that may be implemented on a processor of a vehicle and a system for implementing the method for presenting relevant alerts to a vehicle operator in response to the operator's field of view and area of ​​attention.

[0005] Some aspects include receiving a vehicle-to-everything (V2X) communication including information regarding a V2X-identified threat, determining whether a vehicle operator recognizes the V2X-identified threat, and in response to determining that the vehicle operator does not recognize the V2X-identified threat, determining a display location that is most likely to receive the vehicle operator's attention, and generating an alert regarding the V2X-identified threat on the determined display location. Some aspects may include generating a display that excludes an alert regarding any threat condition recognized by the vehicle operator.

[0006] Some aspects may include determining a location of an area outside the vehicle including the V2X-identified threat and determining a direction of a vehicle operator's line of sight, and determining whether the vehicle operator recognizes the V2X-identified threat may include determining that the vehicle operator does not recognize the V2X-identified threat in response to determining that the location of the area outside the vehicle including the V2X-identified threat is not within the vehicle operator's field of view in the determined direction of the vehicle operator's line of sight.

[0007] Some aspects may include determining a location of an area outside the vehicle including the V2X identified threat and determining whether the location of the area outside the vehicle including the V2X identified threat is within a perceived blind spot of the vehicle operator, and determining whether the vehicle operator has recognized the V2X identified threat may include determining that the vehicle operator does not recognize the V2X identified threat in response to determining that the location of the area outside the vehicle including the V2X identified threat is within a perceived blind spot of the vehicle operator.

[0008] Some aspects may include determining a direction of the vehicle operator's gaze, and determining the display locations most likely to receive the vehicle operator's attention may include identifying display locations that are visible within the vehicle operator's field of view in the determined direction of the vehicle operator's gaze.

[0009] In some aspects, determining the display location most likely to receive the vehicle operator's attention may include determining which of the multiple vehicle displays is designated as a preferred display for the V2X-identified threat.

[0010] Some aspects may include determining whether a threat confirmation has been received from a vehicle operator regarding the V2X identified threat, and determining whether the vehicle operator is aware of the V2X identified threat may include determining that the vehicle operator is not aware of the V2X identified threat in response to determining that a threat confirmation has not been received from the vehicle operator regarding the V2X identified threat.

[0011] Some aspects may include receiving V2X occlusion data indicating conditions exist that reduce visibility of the V2X-identified threat to the vehicle operator, and determining whether the vehicle operator is aware of the V2X-identified threat may include determining that the vehicle operator is not aware of the V2X-identified threat in response to receiving the V2X occlusion data.

[0012] Some aspects may include increasing a notification priority of the V2X-identified threat in response to determining that the vehicle operator is not aware of the V2X-identified threat and determining whether the increased notification priority of the V2X-identified threat is a higher priority than other identified threats, and generating an alert regarding the V2X-identified threat on the determined display location may include generating an alert regarding the V2X-identified threat but excluding alerts regarding the other identified threats on the determined display location in response to determining that the increased notification priority of the V2X-identified threat is a higher priority than other identified threats.

[0013] In some aspects, V2X communications including information regarding V2X identified threats may be received from a source remote from the vehicle.

[0014] A further aspect includes a vehicle system including a memory and a processor configured to perform operations of any of the methods summarized above. A further aspect may include a vehicle system having various means for performing functions corresponding to any of the methods summarized above. A further aspect may include a non-transitory processor-readable storage medium having stored thereon processor-executable instructions configured to cause a processor of the vehicle system to perform various operations corresponding to any of the methods summarized above.

[0015] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate exemplary embodiments of the claims and, together with the general description given above and the detailed description below, serve to explain the features of the claims. [Brief description of the drawings]

[0016] [Figure 1] FIG. 1 is a schematic diagram illustrating an example V2X system suitable for implementing various embodiments. [Figure 2A]FIG. 1 is a component block diagram illustrating a vehicle suitable for implementing various embodiments. [Figure 2B] FIG. 1 is a component block diagram illustrating a vehicle suitable for implementing various embodiments. [Figure 2C] FIG. 1 is a component diagram of an exemplary vehicle system suitable for implementing various embodiments. [Figure 2D] FIG. 1 is a conceptual diagram illustrating an example V2X communication protocol stack suitable for implementing various embodiments. [Figure 3A] FIG. 1 is a component block diagram illustrating components of an example vehicle management system suitable for implementing various embodiments. [Figure 3B] FIG. 2 is a component block diagram illustrating components of another example vehicle management system suitable for implementing various embodiments. [Figure 4A] FIG. 2 is a process flow diagram of an exemplary method for presenting relevant alerts to a vehicle operator, according to various embodiments. [Figure 4B] FIG. 1 is a process flow diagram of example operations that may be performed as part of a method for presenting relevant alerts to a vehicle operator, according to various embodiments. [Figure 4C] FIG. 1 is a process flow diagram of example operations that may be performed as part of a method for presenting relevant alerts to a vehicle operator, according to various embodiments. [Figure 4D] FIG. 1 is a process flow diagram of example operations that may be performed as part of a method for presenting relevant alerts to a vehicle operator, according to various embodiments. [Figure 4E] FIG. 1 is a process flow diagram of example operations that may be performed as part of a method for presenting relevant alerts to a vehicle operator, according to various embodiments. [Figure 4F] FIG. 1 is a process flow diagram of example operations that may be performed as part of a method for presenting relevant alerts to a vehicle operator, according to various embodiments. [Figure 4G]FIG. 1 is a process flow diagram of example operations that may be performed as part of a method for presenting relevant alerts to a vehicle operator, according to various embodiments. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0017] Various embodiments are described in detail with reference to the accompanying drawings. Wherever possible, the same reference numbers are used throughout the drawings to refer to the same or like parts. References made to specific examples and implementations are for illustrative purposes only and do not limit the scope of the claims.

[0018] Generally, various embodiments may leverage elements of the V2X warning system, vehicle interior / occupant monitoring system (OMS), and on-board sensors to identify and display relevant alerts to a vehicle operator among multiple vehicle displays at appropriate locations, such as on-board displays that are most likely to receive the attention of the vehicle operator. Various embodiments may calculate obstructed or degraded line of sight, areas of the vehicle operator's perception, and whether a threat is recognized by the operator to present only relevant alerts at locations and / or in a manner that is likely to be noticed by the vehicle operator. Additionally, the relevant alert or alerts presented to the vehicle operator may exclude information or alerts regarding threats already recognized by the operator and / or low priority threats. Additionally, the relevant alert or alerts may be presented only at selected display locations rather than multiple display locations, which may clutter the display and potentially reduce the likelihood that the operator will recognize and evaluate the alert(s).

[0019] The term "system on chip" (SOC) is used herein to refer to a single integrated circuit (IC) chip that includes multiple resources and / or processors integrated on a single substrate. A single SOC may include circuits for digital, analog, mixed signal, and radio frequency functions. A single SOC may also include any number of general purpose and / or special purpose processors (digital signal processors, modem processors, video processors, etc.), memory blocks (e.g., ROM, RAM, Flash, etc.), and resources (e.g., timers, voltage regulators, oscillators, etc.). A SOC may also include software for controlling the integrated resources and processors, as well as for controlling peripheral devices.

[0020] The term "system in package" (SIP) may be used herein to refer to a single module or package that contains multiple resources, computing units, cores, and / or processors on two or more IC chips, substrates, or SOCs. For example, a SIP may include a single substrate on which multiple IC chips or semiconductor dies are stacked in a vertical configuration. Similarly, a SIP may include one or more multi-chip modules (MCMs) on which multiple ICs or semiconductor dies are packaged in a singulated substrate. A SIP may also include multiple independent SOCs packaged in close proximity, coupled to each other via high-speed communication circuits, such as on a single motherboard or within a single wireless device. The proximity of the SOCs facilitates high-speed communication and sharing of memory and resources.

[0021] Terms such as "component," "system," "unit," "module," and the like refer to computer-related entities, such as, but not limited to, hardware, firmware, a combination of hardware and software, software, or software in execution, configured to perform certain operations or functions. For example, a component may be, but is not limited to, a process running on a processor, a processor, an object, an executable, a thread of execution, a program, and / or a computer. By way of example, both an application running on a communications device and the communications device may be referred to as a component. One or more components may reside within a process and / or thread of execution, and a component may be localized on one processor or core and / or distributed among two or more processors or cores. Additionally, these components may execute from various non-transitory computer-readable media having various instructions and / or data structures stored thereon. Components may communicate by way of local and / or remote processes, function or procedure calls, electronic signals, data packets, memory reads / writes, and other computer, processor, and / or process related communication methods known in the art.

[0022] The term "V2X identified threat" refers to an event or condition on or near a vehicle that may cause harm or risk of harm to the vehicle, an occupant of the vehicle, or anything on or within the vehicle. Additionally, harm or risk to the vehicle may include harm or risk to any extension of the vehicle, such as towed vehicles, cargo, and / or associated vehicles in a caravan or operating as a platoon.

[0023] The term "V2X-onboard equipment" refers to equipment onboard a vehicle or device that provides vehicle-to-everything (V2X) functionality. V2X-onboard equipment typically includes a processing system that may include one or more processors, SoCs, and / or SIPs, any of which may include one or more components, systems, units, and / or modules that implement the V2X functionality (collectively referred to herein as "processing system" for brevity). Aspects of the V2X-onboard equipment and functionality may be implemented with hardware components, software components, or a combination of hardware and software components.

[0024] The term "wireless device" is used herein to refer to any one or all of the following: cellular telephones, smartphones, portable computing devices, tablet computers, smartbooks, wireless email receivers, multimedia Internet-enabled cellular telephones, wearable devices including smart watches, smart clothing, smart glasses, smart wristbands, smart jewelry (e.g., smart rings, smart bracelets, etc.), and similar electronic devices that contain memory, wireless communication components, and a programmable processor.

[0025] Cooperative-ITS (C-ITS) is under development to improve road safety and pave the way for the realization of fully autonomous driving based on the exchange of information via direct wireless short-range communications dedicated to C-ITS and road transport and traffic telematics (RTTT). Several regions of the world are developing standards for vehicle-based communication systems and functions, including standards developed by the Institute of Electrical and Electronics Engineers (IEEE) and the Society of Automotive Engineers (SAE) for use in North America, and by the European Telecommunications Standards Institute (ETSI) and the European Committee for Standardization (CEN) for use in Europe. Included in such systems is the ability for vehicles to broadcast messages that can be received and processed by other vehicles to improve road safety. Such messages are called Basic Safety Messages (BSM) in North America and Cooperative Awareness Messages (CAM) in Europe (collectively referred to herein as BSM messages for brevity).

[0026] The IEEE 802.11p standard is the basis for Dedicated Short Range Communications (DSRC) and ITS-G5 communication standards. IEEE 1609 is a higher level standard based on IEEE 802.11p. The Cellular Vehicle-to-Everything (C-V2X) standards are competing standards developed under the auspices of the 3rd Generation Partnership Project. These standards serve as the basis for vehicle-to-vehicle and vehicle-to-traffic system wireless communications used to support intelligent highways, autonomous and semi-autonomous vehicles, and improve the overall efficiency and safety of highway transportation systems. Other V2X wireless technologies are also being considered in different regions of the world. The techniques described herein are applicable to any V2X wireless technology.

[0027] The C-V2X protocol specifies two transmission modes that together provide 360° non-line-of-sight awareness and higher levels of predictability for enhanced road safety and autonomous driving. The first transmission mode includes Direct C-V2X, which includes vehicle-to-vehicle (V2V), vehicle-to-infrastructure (V2I), and vehicle-to-pedestrian (V2P) communications and provides extended communication range and reliability in a dedicated Intelligent Transport System (ITS) 5.9 gigahertz (GHz) spectrum that is independent of cellular networks. The second transmission mode includes vehicle-to-network (V2N) communications in mobile broadband systems and technologies, such as third generation wireless mobile communication technologies (3G) (e.g., Global System for Mobile Communications (GSM) Evolution (EDGE) systems, Code Division Multiple Access (CDMA) 2000 systems, etc.), fourth generation wireless mobile communication technologies (4G) (e.g., Long Term Evolution (LTE) systems, LTE-Advanced systems, Mobile Worldwide Interoperability for Microwave Access (Mobile WiMAX) systems, etc.), fifth generation wireless mobile communication technologies (5G NR systems, etc.). Other V2X wireless technologies are also under consideration in different regions of the world. The techniques described herein are applicable to any V2X wireless technology.

[0028] V2X systems and technologies hold great promise for improving traffic flow and vehicle safety by allowing vehicles to share information about their location, speed, heading, braking, and other factors that may be useful to other vehicles for collision avoidance and other safety functions. Vehicles equipped with V2X equipment may transmit their vehicle information frequently (e.g., up to 20 times per second) in small messages such as Basic Safety Messages (BSM) or Cooperative Awareness Messages (CAM). If all V2X-enabled vehicles transmit such BSM / CAM messages, then all receiving vehicles will have the information needed to control their own speed and direction to avoid collisions and efficiently and safely position vehicles relative to each other.

[0029] Current V2X applications use maps, vehicle kinematics, and vehicle heading / tracking to calculate and identify safety-critical events or conditions within identified areas of relevance. Events and conditions outside these areas of relevance may be ignored for the purpose of alerting the operator. Defining areas of relevance may be useful to reduce tracked V2X objects and focus computational power and resources on more appropriate safety-related data. However, such areas of relevance do not take into account the actual line of sight from the operator's seat or the current operator's line of sight direction. Thus, such systems often alert the operator to threats to the vehicle or its occupants that the operator already knows about. Repeated alerts that may be ignored may cause the operator to become desensitized to the alerts, which may reduce the benefit of such alert systems. Also, alert systems that often generate unnecessary alerts may cause user discomfort and / or fatigue.

[0030] Various embodiments include methods and systems configured to execute methods for presenting relevant alerts to a vehicle operator in a manner configured to increase the likelihood that the operator will perceive and act on such alerts. In various embodiments, a vehicle processing system (e.g., V2X on-board equipment) can receive V2X communications including information regarding a V2X-identified threat to the vehicle or vehicle occupants. The received V2X communications including information regarding the V2X-identified threat may be from an on-board system, a V2X infrastructure, another vehicle, another external system, etc. In response to receiving the message, the vehicle processing system may evaluate multiple factors relevant to determining whether the V2X-identified threat poses a true threat to the vehicle or passengers. The vehicle processing system can evaluate information from various vehicle sensors to determine whether the vehicle operator recognized the V2X-identified threat. An operator-recognized threat may not warrant an alert, but the operator may need to be alerted to unrecognized V2X-identified threats. In various embodiments, the vehicle's processing system may evaluate inputs from the vehicle OMS and on-board sensors or equipment settings to identify explicit and / or implicit indications as to whether the operator recognized the V2X-identified threat.

[0031] The vehicle processing system may determine a display location that is most likely to receive the vehicle operator's attention in response to determining that the vehicle operator does not recognize the V2X-identified threat. The vehicle may have a large head-up display capability, such as a head-up display on the side windows and / or multiple displays, such as multiple different displays on the dashboard, that encompass much if not all of the windshield. Additionally, the operator may be wearing a head-mounted display (HMD) that provides an augmented reality display of vehicle information overlaid on the inside and outside view of the vehicle that the user is looking at. To increase the likelihood that the operator will see and perceive a threat warning message or alert, the processing system may determine where the operator is looking and identify a display location within the operator's line of sight that will present a visual warning regarding the V2X-identified threat. For example, if a threat, such as an oncoming vehicle, is approaching from the operator's right side, but the operator is currently gazing to the left side of the vehicle, the vehicle processing system may render a warning message on a vehicle display location (e.g., a portion of or located on the large display) to the operator's left side. In vehicles equipped with a large display (e.g., a HUD that wraps around the windshield), the display location for presenting the alert message determined by the vehicle processing system may be the portion of the display that is aligned with the operator's current line of sight. In such vehicles, the vehicle processing system may move the alert message across the large display as the operator moves his / her eyes and head (i.e., the display location may change in response to the operator's eye / head movements). In vehicles equipped with multiple displays (e.g., a small HUD, a dashboard display, and a side panel display), the determined display location may be the one of the multiple displays that is most closely aligned with the operator's current line of sight.In such vehicles, the vehicle processing system may move the warning message from one display to another as the operator moves his or her eyes and head (i.e., the display selected to present the alert message may change in response to the operator's eye / head movements). In some embodiments suitable for vehicles that include a display dedicated or preferred for displaying V2X-related alerts, determining the display location most likely to receive the vehicle operator's attention may involve selecting one of multiple vehicle displays that has been designated as the preferred display for V2X-identified threats.

[0032] The processing system can then generate an alert for the V2X-identified threat and render the alert at the determined display location to improve the likelihood that the operator will see and act on the alert message. To avoid overloading the operator with information that is less important to the operator, the vehicle processing system can generate an alert display that includes only the most relevant or highest priority information, which may include consideration of whether the operator is aware (or may be aware) of some threats. In some embodiments, the processing system can generate an alert for the V2X-identified threat, which excludes warnings for any threat conditions that the vehicle operator is already aware of or can see. For example, once the vehicle processing system determines that a particular threat has been recognized by the operator, the generated alert message can remove information for the recognized threat. As another example, if there is a threat posed by an object or an approaching vehicle on both the right and left side of the operator, the vehicle processing system can render a warning message at a vehicle display location (e.g., a large display or a portion of a located display) on the operator's left side that may include only the threat to the operator's right side, since the operator can see the threat to the operator's left side. In some embodiments, the processing system may evaluate information related to the operator's attention to determine whether the operator actually recognized a threat in the direction the operator was looking before filtering out the warning message from the warning presented on the determined display location. Additionally, the relevant warning or warnings may be presented only at the selected display location (i.e., where the operator is looking), avoiding repeating the warning or warnings on multiple displays or display locations, which may potentially reduce the likelihood that the operator will recognize and evaluate the warning.

[0033] Various embodiments may improve the safety and effectiveness of a vehicle safety system that presents useful warnings to an operator of a V2X-equipped vehicle at a display location where the operator is viewing, and avoids rendering unnecessary warnings that may lead to operator discomfort or unnecessary warning fatigue. Various embodiments thus improve the safety and effectiveness of a vehicle safety system by increasing the likelihood that an operator will perceive and respond to a threat warning.

[0034] 1 illustrates an exemplary V2X system 100 suitable for implementing various embodiments. With reference to FIG. 1, vehicles 101, 102, 103 may include V2X-equipped equipment 111, 112, 113, respectively, that may be configured to establish wireless communication links 121, 122, 123 between each other. Similarly, a pedestrian 50 may carry a wireless device 105, also equipped with V2X-equipped equipment, that may also be configured to establish wireless communication links 131, 132, 133 with the V2X-enabled vehicles 101, 102, 103. Additionally, one or more roadside units 109 may also be equipped with V2X-equipped equipment that may be configured to establish wireless communication links 141, 142, 143, 144 with the vehicles 101, 102, 103 and the wireless device 105. Those wireless communication links 121, 122, 123, 131, 132, 133, 141, 142, 143, 144 may be configured to periodically carry Basic Safety Messages (BSMs) or other V2X communications broadcast between the vehicles 101, 102, 103, the wireless devices 104, and / or the roadside units 109. In contrast, the vehicle 104 may represent other vehicles that are not equipped with the capability to transmit or receive V2X messages.

[0035] 1 reflects a situation in which a vehicle operator of a first vehicle 101 having a particular point of view (POV) may be able to easily observe pedestrians 50 and other objects located in Zone A, but may have an obstructed or partially obstructed view of objects located at the far end of Zone B. In this situation, the operator of the first vehicle 101 may not be aware that a third vehicle 103 located in the obstructed portion of Zone B is heading north on the road 10. If not fully informed of the situation, the operator of the first vehicle 101 may erroneously believe that he or she is free to exit into northbound traffic, potentially resulting in a collision.

[0036] By sharing vehicle / device location, speed, direction, behavior such as velocity, acceleration / deceleration, turning, etc., the vehicles 101, 102, 103 and pedestrians can maintain a safe separation and identify and avoid potential collisions (i.e., threats). For example, a first vehicle 101 beginning to exit a parking space can receive a V2X communication (e.g., BSM, CAM) that includes information regarding a V2X-identified threat to the vehicle or vehicle occupants. Any V2X-equipped equipment, such as from one or more of the other vehicles 102, 103, wireless device 105, and / or roadside unit 109, can transmit and / or relay the V2X communication to the first vehicle 101. Once received, a processing system of the first vehicle 101 can use the information contained in or accompanying the V2X communication (i.e., the V2X-identified threat) to determine the location and trajectory of the third vehicle 103. The first vehicle 101 may then generate an alert to inform the operator of the imminent risk of a collision with the third vehicle 103. Such an alert may allow the operator of the first vehicle 101 to remain in the parking space a little longer (e.g., at least until the third vehicle 103 has passed) to avoid a collision.

[0037] V2X-Equipped Equipment Additionally, the V2X-Equipped Equipment 111 in the first vehicle 101 may receive, either within the received V2X message or as part of a separate V2X message, information regarding a threat associated with a pedestrian 50 walking towards the first vehicle 101. However, unlike the third vehicle 103 in the obstructed portion of Zone B, the pedestrian 50 is walking within line of sight from the POV of the operator of the first vehicle 101. Thus, according to various embodiments, the processing system of the first vehicle 101 may determine the location and trajectory of the pedestrian 50 and decide not to generate an alert regarding the threat because there is an indication that the vehicle operator has recognized the threat, i.e., the fact that the threat from the pedestrian 50 (i.e., the risk of causing a collision with the pedestrian) is or should be clearly visible to the vehicle operator. For example, the OMS and / or internal sensors in the first vehicle 101 can detect the direction of the vehicle operator's gaze (i.e., the current direction in which the vehicle operator is looking, either stationary, backwards, or otherwise), which the vehicle processor can use to determine that the pedestrian 50 is within the field of view of the vehicle operator in that direction of the vehicle operator's gaze. A threat located within the vehicle operator's field of view can be interpreted as an implicit acknowledgment of the threat (i.e., threat acknowledgment). Alternatively, the vehicle operator may provide an explicit threat acknowledgment, such as a gesture, verbal cue, or other system input that indicates to the vehicle processor that the vehicle operator is aware of the threat.

[0038] Additionally, the vehicles 101, 102, 103, wireless devices 105, and / or roadside units 109 may transmit data and information regarding V2X identified threats and / or other V2X communications to the ITS over a communications network (e.g., V2X, cellular, WiFi, etc.). Elements of the ITS may be configured to communicate with each other over wired or wireless networks to exchange information, such as details about objects and conditions that may cause or be associated with V2X identified threats.

[0039] Various embodiments may be implemented in a variety of vehicles, of which an exemplary vehicle 101 is shown in Figures 2A and 2B. With reference to Figures 1-2B, vehicle 101 may represent any one or more of vehicles 101, 102, 103 described with respect to Figure 1. In various embodiments, vehicle 101 may include a V2X onboard equipment (e.g., 111) and a control unit 240 that may include a number of sensors 211-238, including a satellite geographic positioning system receiver 213, occupancy sensors 212, 216, 218, 226, 228, tire pressure sensors 214, 220, cameras 222, 236, microphones 224, 234, crash sensor 230, radar 232, and lidar 238. A number of sensors 211-238 disposed in or on the vehicle may be used for various purposes, such as autonomous and semi-autonomous navigation and control, collision avoidance, position determination, and to provide sensor data regarding objects and people in or on the vehicle 101. The sensors 211-238 may include one or more of a wide variety of sensors capable of detecting various information useful for navigation and collision avoidance. Each of the sensors 211-238 may be in wired or wireless communication with the control unit 240 and with each other. In particular, the sensors may include one or more cameras 222, 236, or other optical or photo-optic sensors. The sensors may further include other types of object detection and ranging sensors, such as radar 232, lidar 238, IR sensors, and ultrasonic sensors. The sensors may further include tire pressure sensors 214, 220, humidity sensors, temperature sensors, satellite geographic positioning system sensors 213, control input sensors 211, accelerometers, vibration sensors, gyroscopes, gravity meters, crash sensors 230, strength meters, stress meters, strain sensors, fluid sensors, chemical sensors, gas content analyzers, pH sensors, radiation sensors, Geiger counters, neutron detectors, biological material sensors, microphones 224, 234, occupancy sensors 212, 216, 218, 226, 228, proximity sensors, and other sensors.

[0040] According to various embodiments, the vehicle control unit 240 may be configured to steer signals from one or both of the narrowband Wi-Fi emitters 205F, 205R. Additionally, the control unit 240 may have a default setting for one or both of the narrowband Wi-Fi emitters 205F, 205R, such as a setting to not steer or a setting to automatically direct one or both of the narrowband Wi-Fi emitters 205F, 205R to follow the steering wheel. When the control unit 240 is not actively steering one or both of the narrowband Wi-Fi emitters 205F, 205R, the default setting may be followed.

[0041] The vehicle control unit 240 may be configured with processor-executable instructions for performing navigation and collision avoidance operations using information received from various sensors, particularly the cameras 222, 236. In some embodiments, the control unit 240 may supplement the processing of the camera images using distance and relative position (e.g., relative azimuth) that may be obtained from the radar 232 and / or lidar 238 sensors. The control unit 240 may be further configured to control the steering, braking, and speed of the vehicle 101 when operating in an autonomous or semi-autonomous mode using information about other vehicles determined using various embodiments.

[0042] FIG. 2C is a component diagram of an exemplary vehicle system 200 suitable for implementing various embodiments. With reference to FIGS. 1-2C, the system 200 may include a vehicle 101 including a V2X-on-board equipment 111 (e.g., a telematics control unit or on-board unit (TCU / OBU)). The V2X-on-board equipment 111 may communicate with various systems and devices, such as an in-vehicle network 250, an infotainment system 252, various sensors 254, various actuators 256, and a radio frequency (RF) module 172. The V2X-on-board equipment 111 may also communicate with various other vehicles 201, a roadside unit 109, a base station 260, and other external devices. The V2X-on-board equipment 111 may be configured to perform operations to present relevant alerts to a vehicle operator, as described further below.

[0043] 2C, the V2X-onboard equipment 111 includes a processor 164, a memory 166, an input module 168, an output module 170, and a radio module 172. The processor 164 may be configured with processor-executable instructions for controlling the steering, navigation, and / or other operation of the vehicle 101, including decisions regarding alerting a vehicle operator of relevant warnings, and, optionally, the operation of various embodiments. The processor 164 may be coupled to the memory 166. The processor 164 may also be coupled to an output module 170, which may control an on-board display to generate an alert regarding a V2X-identified threat at a determined display location.

[0044] The V2X-equipped equipment 111 may include a V2X antenna (e.g., RF module 172) and may be configured to communicate with one or more ITS participants (e.g., stations), such as another vehicle 201, a roadside unit 109, and a base station 260 or another suitable network access point. In various embodiments, the V2X-equipped equipment 111 may receive information from multiple sources, such as an in-vehicle network 250, an infotainment system 252, various sensors 254, various actuators 256, and the RF module 172. The V2X-equipped equipment 111 may be configured to alert the vehicle operator of relevant alerts, as described further below.

[0045] Examples of in-vehicle networks 250 include a Controller Area Network (CAN), a Local Interconnect Network (LIN), a network using the FlexRay protocol, a Media Oriented System Transport (MOST) network, and an in-vehicle Ethernet network. Examples of vehicle sensors 254 include location determination systems (such as Global Navigation Satellite System (GNSS) systems), cameras, radar, lidar, ultrasonic sensors, infrared sensors, and other suitable sensor devices and systems. Examples of vehicle actuators 256 include various physical control systems such as steering, braking, engine operation, lighting, turn signals, etc.

[0046] FIG. 2D is a conceptual diagram illustrating an example V2X communications protocol stack 270 suitable for implementing various embodiments.

[0047] FIG. 3A is a component block diagram illustrating components of an exemplary vehicle threat management system 300. The vehicle threat management system 300 may include various subsystems, communication elements, computational elements, computing devices, or computing units that may be utilized within the vehicle 101. With reference to FIGS. 1-3A, the various computational elements, computing devices, or units within the vehicle threat management system 300 may be implemented within a system of interconnected computing devices (i.e., subsystems) that communicate data and commands with each other (e.g., as indicated by the arrows in FIG. 3A). In some implementations, the various computational elements, computing devices, or units within the vehicle threat management system 300 may be implemented within a single computing device, such as separate threads, processes, algorithms, or computational elements. Thus, each subsystem / computational element illustrated in FIG. 3A is also generally referred to herein as a "layer" within the computational "stack" that makes up the vehicle threat management system 300. However, the use of the terms layer and stack in describing various embodiments does not imply or require that the corresponding functions are implemented within a single autonomous (or semi-autonomous) vehicle management system computing device, although this is a possible implementation embodiment. Rather, use of the term "layer" is intended to encompass subsystems having independent processors, computational elements (e.g., threads, algorithms, subroutines, etc.) that execute within one or more computing devices and combinations of subsystems and computational elements.

[0048] The vehicle threat management system 300 may include a radar perception layer 302, a camera perception layer 304, a positioning engine layer 306, a map fusion and arbitration layer 308, a V2X communications layer 310, a sensor fusion and road world model (RWM) management layer 312, a threat assessment layer 314, and an operator perception assessment layer 316. The layers 302-316 are only examples of some layers in one exemplary configuration of the vehicle threat management system 300. Other configurations may include other layers, such as additional layers for other perception sensors (e.g., a LIDAR perception layer, etc.), additional layers for generating alerts and / or display selections, additional layers for modeling, and / or may exclude certain of the layers 302-316 from the vehicle threat management system 300. Each of the layers 302-316 may exchange data, computations, and commands, as illustrated by the arrows in FIG. 3A. Additionally, the vehicle threat management system 300 may receive and process data from sensors (e.g., radar, lidar, cameras, inertial measurement units (IMUs), etc.), navigation systems (e.g., Global Position System (GPS) receivers, IMUs, etc.), vehicle networks (e.g., Controller Area Network (CAN) bus), and databases (e.g., digital map data) in memory. The vehicle threat management system 300 may output alerts and / or commands to generate alerts regarding V2X-identified threats on selected on-board display locations. The vehicle threat management system 300 configuration shown in FIG. 3A is merely an exemplary configuration, and other configurations of the vehicle management system and other vehicle components may be used. As an example, the vehicle threat management system 300 configuration shown in FIG. 3A may be used in a vehicle configured for autonomous or semi-autonomous operation, although different configurations may be used in non-autonomous vehicles.

[0049] The radar perception layer 302 may receive data from one or more detection and ranging sensors, such as radar (e.g., 132) and / or lidar (e.g., 138), and may process that data to recognize and determine the location of other vehicles and objects within the vicinity of the vehicle 101. The radar perception layer 302 may include the use of neural network processing and artificial intelligence methods to recognize objects and vehicles, and may convey such information to the sensor fusion and RWM management layer 312.

[0050] The camera perception layer 304 may receive data from one or more cameras, such as cameras (e.g., 122, 136), and process that data to recognize and determine observations regarding the location of other vehicles and objects within the vicinity of the vehicle 100, as well as the operator (e.g., the direction of the operator's gaze). The camera perception layer 304 may include the use of neural network processing and artificial intelligence methods to recognize objects and vehicles, and may convey such information to the sensor fusion and RWM management layer 312.

[0051] The positioning engine layer 306 may receive data from various sensors and process the data to determine the position of the vehicle 101. The various sensors may include, but are not limited to, GPS sensors, IMUs, and / or other sensors connected via a CAN bus. The positioning engine layer 306 may also utilize inputs from one or more cameras, such as cameras (e.g., 122, 136), and / or any other available sensors, such as radar, LIDAR, etc.

[0052] The vehicle threat management system 300 may include or be coupled to a vehicle wireless communications subsystem 330. The wireless communications subsystem 330 may be configured to communicate with other vehicle computing devices and remote V2X communications systems, such as via V2X communications.

[0053] The map fusion and arbitration layer 308 may access sensor data received from other V2X system participants, receive output received from the positioning engine layer 306, and process the data to further determine the location of the vehicle 101 in a map, such as a location in a travel lane, a location in a street map, etc. The sensor data may be stored in a memory (e.g., memory 166). For example, the map fusion and arbitration layer 308 may convert latitude and longitude information from a GPS to a location in a road surface map included in the sensor data. GPS position determinations include errors, so the map fusion and arbitration layer 308 may function to determine a best guess location of the vehicle in the road based on an arbitration between the GPS coordinates and the sensor data. For example, the GPS coordinates may identify the location of the vehicle near the center of a two-lane road in the sensor data, but the map fusion and arbitration layer 308 may determine from the heading direction that the vehicle is most likely aligned in the heading lane that matches the heading direction. The map fusion and arbitration layer 308 may pass the map-based location information to the sensor fusion and RWM management layer 312.

[0054] The V2X communications layer 310 can receive and utilize sensor data and other inputs from the ITS to gather information about moving objects and conditions near and around the vehicle (e.g., 101). The V2X communications received by the V2X communications layer 310 can provide many types of information. For example, other vehicles or roadside units may provide camera images or sensor readings that can be analyzed (i.e., measuring proximity, movement, trajectory, etc.). The V2X communications layer 310 can pass V2X messaging information to the sensor fusion and RWM management layer 312. However, use of the V2X messaging information by other layers, such as the sensor fusion and RWM management layer 312, is not required. For example, other stacks can control the vehicle (e.g., control one or more vehicle displays) without using the received V2X messaging information.

[0055] In some embodiments, the processor may employ a machine learning process to initially determine or improve detector settings for an ITS participant type. In some embodiments, the processor may provide multiple observations of parameters of the ITS participant type to a trained model, such as an ITS participant type model, and may receive as output from the model (e.g., the ITS participant type model) detector settings for one or more detectors configured to evaluate aspects of the ITS participant.

[0056] In some embodiments, the processor may store the detector settings for each ITS participant type in a memory accessible by the vehicle processing system (e.g., by ITS participant type). In some embodiments, the processor may receive updates from a central source that distributes detector settings and setting updates to some or all ITS participants in a region via a wireless network (e.g., V2X from roadside units (RSUs), 5G networks, etc.). Such distributed detector settings or detector setting updates may be provided based on the ITS participant type. The detector settings or detector setting updates may be distributed among the ITS participants at any level of specificity or granularity, including one or more parameters of a single ITS participant type. For example, the ITS participant processor may receive and store an update indicating a maximum reasonable speed of the vehicle. In this manner, the detector settings may be received, updated, and / or stored in a compartmentalized manner, for example, individually and / or by ITS participant type. By allowing detector settings to be received, updated, and / or stored for as few as one detector setting for an ITS participant type, the detector settings may be more robust against tampering or the introduction of erroneous or false settings. For example, an update that includes an incorrect setting for the maximum reasonable speed of a vehicle may affect detections associated with cars, but not detections associated with other ITS participant types (e.g., buses, bikes, pedestrians, etc.).

[0057] In some embodiments, the processor may determine whether the information in the V2X message is valid or invalid using a detector configuration based on the ITS participant type. In some embodiments, the processor may determine whether the message is invalid based on whether the information in the V2X message meets (e.g., is greater than or equal to) a maximum or minimum validity parameter for the detector. In some embodiments, the processor may determine whether the message is valid or invalid based on the output of some detectors. In some embodiments, in response to determining that the information in the V2X message is invalid, the processor may perform a security action. For example, the processor may send an abuse report about the ITS participant to an ITS network element, such as a security server, or to a network element performing a similar or suitable function.

[0058] For ease of reference, some of the embodiments are described in this application with respect to vehicles using V2X systems and protocols. However, it should be understood that various embodiments encompass any or all of V2X or vehicle-based communication standards, messages, protocols, and / or techniques. Thus, nothing in this application should be construed as limiting the claims to a particular system (e.g., V2X) or message or messaging protocol (e.g., Basic Safety Message (BSM)) unless expressly recited as such in the claims. In addition, the embodiments described herein may refer to a V2X processing system in a vehicle. Other embodiments are contemplated in which the V2X processing system may operate in or be included in a mobile device, mobile computer, roadside unit (RSU), and other devices equipped to monitor road and vehicle conditions and participate in V2X communications.

[0059] The sensor fusion and RWM management layer 312 may receive data and outputs produced by the radar perception layer 302, the camera perception layer 304, the map fusion and arbitration layer 308, and the V2X communications layer 310, and may use some or all of such inputs to estimate or refine the location and state of the vehicle 101 relative to the road, other vehicles on the road, and other objects or creatures within the vicinity of the vehicle 101. For example, the sensor fusion and RWM management layer 312 may combine image data from the camera perception layer 304 with arbitrated map location information from the map fusion and arbitration layer 308 to refine the determined position of the vehicle within a traffic lane. As another example, the sensor fusion and RWM management layer 312 may combine object recognition and image data from the camera perception layer 304 with object detection and ranging data from the radar perception layer 302 to determine and refine the relative positions of other vehicles and objects in the vicinity of the vehicle. As another example, the sensor fusion and RWM management layer 312 may receive information regarding other vehicle position and heading from V2X communications (such as via a CAN bus or wireless communications subsystem 330) and combine that information with information from the radar perception layer 302 and camera perception layer 304 to improve the location and movement of other objects. The sensor fusion and RWM management layer 312 may output improved location and status information of the vehicle 101, as well as improved location and status information of other vehicles and objects in the vicinity of the vehicle, to the threat assessment layer 314 and / or the operator perception assessment layer 316.

[0060] As a further example, the sensor fusion and RWM management layer 312 may monitor perception data from various sensors, such as perception data from the radar perception layer 302, the camera perception layer 304, other perception layers, etc., and / or data from one or more sensors themselves to analyze conditions in the vehicle sensor data. The sensor fusion and RWM management layer 312 may be configured to detect conditions in the sensor data, such as whether a sensor measurement is at, above, or below a threshold, some type of sensor measurement is made, and may output the sensor data to the operator perception evaluation layer 316 and / or as part of improved location and status information of the vehicle 101 provided via wireless communication to devices remote from the vehicle 100, such as data servers, other vehicles, etc., through a V2X connection, other wireless connection, etc.

[0061] The improved location and status information may include vehicle descriptors associated with the vehicle and vehicle owner and / or operator, such as vehicle specifications (e.g., size, weight, color, onboard sensor types, etc.), vehicle location, speed, acceleration, heading, attitude, orientation, destination, fuel / power levels, and other status information, vehicle emergency status (e.g., whether the vehicle is an emergency vehicle or an individual in an emergency), vehicle restrictions (e.g., heavy / wide load, turning restrictions, high occupancy vehicle (HOV) permits, etc.), capabilities (e.g., all wheel drive, four wheel drive, snow tires, chains, supported connection types, onboard sensor operating status, onboard sensor resolution level, etc.), equipment issues (e.g., low tire pressure, weak brakes, sensor outages, etc.), owner / operator travel preferences (e.g., preferred lanes, roads, routes, and / or destinations, preference to avoid tolls or highways, fastest route preference, etc.), permission to provide sensor data to a data authority server (e.g., network server 184), and / or owner / operator identification information.

[0062] The operator perception assessment layer 316 of the vehicle threat management system 300 may use the improved location and status information of the vehicle 101 and the location and status information of other vehicles and objects output from the sensor fusion and RWM management layer 312 to predict future behavior of other vehicles and / or objects. For example, the operator perception assessment layer 316 may use such information to predict future relative positions of other vehicles in the vicinity of the vehicle based on its own vehicle position and velocity and the positions and velocities of the other vehicles. Such predictions may take into account information from local dynamic map data and route planning to anticipate changes in relative vehicle positions as the host vehicle and other vehicles follow the road. The operator perception assessment layer 316 may output behavior and location predictions of other vehicles and objects to the threat assessment layer 314. Additionally, the operator perception assessment layer 316 may plan using object behavior in combination with location predictions and generate control signals to control the operation of the vehicle 101. For example, based on the route planning information, the improved location in the road information, and the relative locations and operations of other vehicles, the operator perception assessment layer 316 may determine that the vehicle 101 needs to change lanes and accelerate, such as to maintain or achieve a minimum separation from other vehicles and / or to prepare for a turn or exit. As a result, the operator perception assessment layer 316 may calculate or otherwise determine the steering angle to the wheels and the change in throttle setting to be commanded to the threat assessment layer 314 and the vehicle control unit 240, along with such various parameters necessary to accomplish such lane change and acceleration. One such parameter may be the calculated steering wheel command angle.

[0063] In some embodiments, the operator perception evaluation layer 316 may calculate the operator's obstructed (or degraded) line of sight to identified threats, including V2X identified threats. Such calculations may be based on inputs received from one or more on-board sensors and / or the OMS, which may include inputs detecting the operator's behavior, posture, movement, etc. In some embodiments, the operator perception evaluation layer 316 may calculate the perception area available to the operator (i.e., the available view of the outside world that the operator can see if the operator looks in that direction). In such operations, the operator perception evaluation layer 316 may evaluate blind spots, external obstacles, and other structures or operator restrictions that may block or limit the operator's perception in the direction of the threat, as well as identify areas or zones that the operator can see (e.g., potential areas of perception). Additionally, the operator perception evaluation layer 316 may evaluate areas of possible perception based on where the operator has looked within the past few seconds or based on the current direction / angle of the operator's line of sight. By comparing whether an identified threat, including a V2X threat, enters an obstructed line of sight or within an area of ​​perception, the operator perception evaluation layer 316 can determine whether the operator likely saw the identified threat. Additionally, the operator perception evaluation layer 316 can evaluate internal sensor data or OMS output related to operator actions or reactions to determine whether the operator acknowledged or reacted to the identified threat. For example, the operator perception evaluation layer 316 may evaluate whether the operator took an operator action (e.g., touched the brakes, changed the throttle level, turned the wheel, etc.) or exhibited facial changes or body movements that indicate the operator perceived the threat (e.g., focused his / her gaze on the threat source, turned his / her head towards the threat source, etc.).

[0064] The threat assessment layer 314 may receive data and information output from the sensor fusion and RWM management layer 312 and the behavior of other vehicles and objects, as well as location predictions from the operator perception assessment layer 316, and may use this information to plan and generate control signals for controlling the operation of the vehicle 101 and to verify that such control signals meet safety requirements for the vehicle 101. For example, based on the route planning information, the improved location in the road information, and the relative locations and movements of other vehicles, the threat assessment layer 314 may verify and generate one or more alerts regarding V2X identified threats on the determined display location.

[0065] In various embodiments, the wireless communications subsystem 330 may communicate with other V2X system participants via wireless communications links to transmit sensor data, location data, vehicle data, and data collected by on-board sensors about the environment surrounding the vehicle. Such information may be used by the other V2X system participants to update stored sensor data for relay to the other V2X system participants.

[0066] In various embodiments, the vehicle threat management system 300 may include functionality to perform safety inspection or oversight of various commands, planning or other decisions of various layers that may affect the safety of the vehicle and its occupants. Such safety inspection or oversight functionality may be implemented in a dedicated layer or may be distributed among the various layers and included as part of the functionality. In some embodiments, various safety parameters may be stored in memory, and the safety inspection or oversight functionality may compare determined values ​​(e.g., relative spacing to nearby vehicles, distance from road centerline, etc.) with corresponding safety parameters, and may issue warnings or commands if the safety parameters are or will be violated. For example, a safety or oversight functionality in the operator perception assessment layer 316 (or in a separate layer) may determine (e.g., based on a world model improved by the sensor fusion and RWM management layer 312) a current or future separation distance between the vehicle and another vehicle (as defined by the sensor fusion and RWM management layer 312), compare the separation distance with the safety separation distance parameters stored in memory, and issue commands to the threat assessment layer 314.

[0067] 3B illustrates an example of a subsystem, computational element, computing device, or computing unit in a vehicle management system 350 that may be utilized in a vehicle 101. With reference to FIGS. 1-3B, in some embodiments, layers 302, 304, 306, 308, 310, 312, and 316 of the vehicle threat management system 350 may be similar to those described with reference to FIG. 3A, and the vehicle threat management system 350 may operate similarly to the vehicle threat management system 300, except that the vehicle threat management system 350 may pass various data or instructions to the vehicle safety and collision avoidance system 352. For example, the configuration of the vehicle threat management system 350 and the vehicle safety and collision avoidance system 352 illustrated in FIG. 3B may be used in a non-autonomous vehicle.

[0068] In various embodiments, the operator perception assessment layer 316 and / or the sensor fusion and RWM management layer 312 may output data to the vehicle safety and collision avoidance system 352. For example, the sensor fusion and RWM management layer 312 may output sensor data as part of improved location and state information of the vehicle 101 provided to the vehicle safety and collision avoidance system 352. The vehicle safety and collision avoidance system 352 may use the improved location and state information of the vehicle 101 to make safety decisions regarding the vehicle 101 and / or its occupants. As another example, the operator perception assessment layer 316 may output behavior models and / or predictions related to operator awareness, gaze direction, and / or threat confirmation to the vehicle safety and collision avoidance system 352. The vehicle safety and collision avoidance system 352 may use the behavior models and / or predictions to make safety decisions regarding the vehicle 101 and / or its occupants.

[0069] In various embodiments, the vehicle safety and collision avoidance system 352 may include functionality to perform safety inspections or oversight of various commands, various layers of planning or other decisions, and human operator actions that may affect the safety of the vehicle and its occupants. In some embodiments, various safety parameters may be stored in memory, and the vehicle safety and collision avoidance system 352 may compare determined values ​​(e.g., relative spacing to nearby vehicles, distance from a road centerline, etc.) to corresponding safety parameters, and may issue warnings or commands if a safety parameter is or will be violated. For example, the vehicle safety and collision avoidance system 352 may determine (e.g., based on a world model improved by the sensor fusion and RWM management layer 312) a current or future separation distance between the vehicle and another vehicle (as defined by the sensor fusion and RWM management layer 312), compare the separation distance to safety separation distance parameters stored in memory, and may issue commands to the operator to accelerate, decelerate, or turn if the current or predicted separation distance violates the safety separation distance parameters. As another example, the vehicle safety and collision avoidance system 352 may compare a human operator's changes in steering wheel angle to safe wheel angle limits or parameters and may issue an override command and / or a warning in response to a steering wheel angle exceeding the safe wheel angle limits.

[0070] 4A is a process flow diagram of an exemplary method 400 for presenting relevant alerts to a vehicle operator, according to various embodiments. With reference to FIGS. 1-4A, the operations of method 400 may be performed by a processing system (e.g., 164, 240, 270, 300, 350) of a V2X-equipped equipment (e.g., 111, 112, 113) of a vehicle (e.g., 101, 102, 103).

[0071] In block 421, the processing system may receive a vehicle-to-everything (V2X) communication that includes information regarding a V2X-identified threat to the vehicle or vehicle occupant. For example, the received V2X communication may include information regarding a vehicle, object, creature, or condition that may pose a threat (i.e., a V2X-identified threat) to the vehicle or vehicle occupant. The received V2X communication may include information regarding multiple threats to the vehicle or vehicle occupant. Such V2X-identified threats may relate to an approaching vehicle, object, or creature, a vehicle, object, creature, or condition that the vehicle is approaching, and / or a vehicle, object, creature, or condition that is within a predetermined proximity of the vehicle. The received V2X communication that includes information regarding one or more V2X-identified threats may be from an on-board system, a V2X infrastructure, another vehicle, another external system, and / or any source or vector of V2X communication.

[0072] In response to receiving a V2X communication including information regarding a V2X-identified threat, the vehicle may perform a self-assessment of the V2X-identified threat in block 421. For example, the processing system may have narrower considerations than the V2X system that identified and reported the V2X-identified threat.

[0073] As part of the operations in block 421, the processing system may evaluate a wide range of factors related to the risk posed to the vehicle and / or its occupants when performing a self-assessment of the V2X identified threats. In some embodiments, in block 421, the processing system may evaluate area classifications of areas proximate to the vehicle or in the general vicinity of the vehicle, such as conditions that may affect the vehicle's maneuverability, the behavior of other vehicles / objects / creatures, and the location information of the vehicle (e.g., urban streets, parking lots, presence of suburban or narrower buildings, suburban, rural, etc.). In some embodiments, in block 421, the processing system may evaluate the road configuration of the road area proximate to the vehicle, such as conditions that may affect visibility, vehicle maneuvering, and vehicle behavior. For example, the processing system may evaluate the presence of intersections, sidewalks, narrow roads, straight vs. curved roads, road topography (e.g., rugged or flat), pedestrian crossings, bike paths, etc., which may affect the risk posed to the vehicle or occupants, as well as the threat the vehicle may pose to other vehicles, pedestrians, property, animals, etc. In some embodiments, in block 421, the processing system may determine historical risk information for the area proximate to the vehicle, such as whether the area has a history of accidents, the type of accidents, and the frequency of accidents.

[0074] In some embodiments, in block 421, the processing system may evaluate the observed behavior of the other vehicle, object, or creature, such as whether another vehicle is maneuvering erratically, whether it is driving in the subject vehicle's lane or on or near a road, whether it is driving on a sidewalk or bike path, etc. In some embodiments, in block 421, the processing system may classify the other vehicle, object, and / or creature, such as whether it is a car, truck, cyclist, pedestrian, animal, etc.

[0075] In some embodiments, in block 421, the processing system may evaluate local weather conditions, such as conditions that may affect visibility, maneuverability, and vehicle operation, including the presence of fog, rain, ice, wet or slippery roads, etc., as well as daylight or lighting conditions, such as dawn or dusk, nighttime, daylight, the presence or absence of street lighting at night.

[0076] In some embodiments, at block 421, the processing system may evaluate a plurality of factors to determine a risk score for the risk posed by the vehicle to other vehicles, objects, and / or living things. In some embodiments, the evaluation of each of the factors may increase or decrease the risk score. In such embodiments, the processing system may determine the risk score as an aggregation of the evaluation of the factors relevant to determining the risk posed by the vehicle to other vehicles, objects, and / or living things. In some embodiments, the processing system may associate each of the plurality of factors with a weighting factor. In such embodiments, the processing system may determine the risk score as an aggregation of the factors relevant to determining the risk posed by the vehicle to other vehicles, objects, and / or living things and their assigned weighting factors. In some embodiments, the processing system may adjust the weights applied to some of the factors based on the value or influence of other detected factors, for example, increasing the weights associated with some of the factors when the predicted separation distance between the vehicle and other vehicles, objects, and / or living things at the predicted closest point of approach meets or is below a threshold value.

[0077] The processing system may evaluate multiple factors in various ways when performing a self-assessment of a V2X identified threat in block 421. In some embodiments, the processing system may assign a numerical value to each determined factor and may aggregate the assigned numerical values ​​to determine a risk score. In some embodiments, the processing system may apply the determined factors to a decision tree configured to generate a risk score as an output. In some embodiments, the processing system may provide the multiple factors (or the numerical values ​​assigned to each of the factors) as inputs to a weighted formula configured to generate a risk score as an output. In some embodiments, the processing system may apply the multiple factors to a trained neural network model that provides a risk score as an output. Other techniques for evaluating the multiple factors in block 406 are also possible. In some embodiments, the processing system may determine whether the risk score meets an alert threshold. For example, the processing system may compare the risk score to a threshold stored in memory. In some embodiments, the threshold may vary depending on circumstances such as the operator of the vehicle, whether the vehicle is operating autonomously, the predicted separation distance between the vehicle and other vehicles, objects, and / or living things at the predicted closest approach point, etc.

[0078] The means for performing the operations of block 420 include the processing system 164 , 240 , 270 , 300 , 350 , the memory 166 , the input module 168 , the output module 170 , the wireless module 172 , and the threat assessment module 314 .

[0079] In block 422, the processing system may determine whether the vehicle operator recognizes the V2X identified threat. As part of the operations of block 422, the processing system may evaluate the operator's current field of view, as well as internal and external factors that potentially affect the operator's view of the identified threat.

[0080] In some embodiments, determining whether the vehicle operator recognizes the V2X-identified threat may include determining that the vehicle operator does not recognize the V2X-identified threat in response to determining that the location of an area outside the vehicle that includes the V2X-identified threat is not within the operator's line of sight. Thus, a V2X-identified threat may not be visible to the operator because a threat object (e.g., another vehicle) is located in an area the operator cannot see, such as when the operator's view of the object is obstructed by an external vehicle, object, creature, condition, or the like. In some embodiments, in block 422, the processing system may take into account environmental factors that may impede or obscure visibility, such as sunrise, sunset, glare, fog, smog, rain, and the like.

[0081] In some embodiments, determining whether the vehicle operator is not aware of the V2X-identified threat may include determining that the vehicle operator is not aware of the V2X-identified threat in response to determining that the location of an area outside the vehicle that includes the V2X-identified threat is within the vehicle operator's perceived blind spot. For example, based on the vehicle operator's perspective, the vehicle data may identify known vehicle blind spots that are areas outside the vehicle that are obstructed from the vehicle operator's view due to vehicle structure or characteristics. As a further example, the processing system may obtain external environment information that may identify external elements or factors that may cause blind spots, such as sunset, sunrise, fog, etc. In this manner, one or more areas outside the vehicle where the external elements or factors are located may be designated as the vehicle operator's perceived blind spot.

[0082] In some embodiments, determining whether the vehicle operator is not aware of the V2X-identified threat may include determining that the vehicle operator is not aware of the V2X-identified threat in response to receiving V2X occlusion data indicating that the operator may not be able to see the V2X-identified threat.

[0083] In some embodiments, determining whether the vehicle operator is not aware of the V2X-identified threat may include determining that the vehicle operator is not aware of the V2X-identified threat in response to determining that the operator has not responded to indicate that the operator saw and confirmed the V2X-identified threat.

[0084] In some embodiments, at block 422, the processing system can calculate the operator's line of sight and corresponding field of view taking into account any occlusions or obstacles indicated from the V2X communications (i.e., from the perspective of the remote sensors), which can provide information regarding occluded objects. In addition, the processing system may calculate the operator's line of sight and corresponding field of view taking into account the operator's perspective of the object (e.g., using on-board sensors and / or the OMS providing inputs such as seat height). Furthermore, the processing system analyzes the operator's behavior to add contextual and / or dynamic data to the analysis using gaze angle / direction, facial pose, and / or body movements.

[0085] Using the combination of all this information, the processing system can calculate the true area of ​​relevance outside the vehicle for threat assessment in block 422. If the processing system determines that an identified threat is located outside the true area(s) of relevance, then the operator does not need to be notified of the identified threat.

[0086] The determination made by the processing system in block 422 may be in response to the processing system determining, through a self-assessment of the V2X identified threat, that the V2X identified threat poses a significant risk to the vehicle and / or vehicle occupants. The means for performing the operations of block 422 include the processing system 164, 240, 270, 300, 350, the memory 166, the input module 168, the output module 170, the wireless module 172, the operator perception assessment module 316, and the vehicle safety and collision avoidance module 352.

[0087] In block 424, the processing system may determine a display location that is most likely to receive the vehicle operator's attention in response to determining that the vehicle operator does not recognize the V2X-identified threat. In some embodiments suitable for vehicles with a large display, such as a head-up display that encompasses most or all of the windshield, determining the display location that is most likely to receive the vehicle operator's attention may include determining a portion or segment of the large display that is visible within (i.e., aligned with) the operator's field of view in a determined direction of the operator's line of sight. In some embodiments suitable for vehicles with multiple vehicle displays, determining the display location that is most likely to receive the vehicle operator's attention may include determining which of multiple vehicle displays are visible within the operator's field of view in a determined direction of the operator's line of sight. For example, if the V2X-identified threat is coming from the operator's right side (see FIG. 1), but the operator is looking to the left (i.e., in the opposite direction), a display location located on the left side of the dashboard (e.g., to the left of the full windshield display, on a separate display located on the left side of the dashboard, or on a portion of the HMD augmented reality display) may be selected by the processing system for use in presenting the warning to the operator. In some embodiments, determining the display location most likely to receive the attention of the vehicle operator may include determining which of a plurality of vehicle displays is designated as a preferred display for the V2X identified threat. In this manner, the processing system may calculate the optimal or best available location for displaying a warning to the operator. The means for performing the operations of block 424 include the processing system 164, 240, 270, 300, 350, the memory 166, the input module 168, the output module 170, the wireless module 172, the threat assessment module 314, the operator perception assessment module 316, and the vehicle safety and collision avoidance module 352.

[0088] At block 426, the processing system may generate an alert for the V2X-identified threat on the determined display location. In some embodiments, the processing system may generate an alert for the V2X-identified threat on the determined display location that excludes alerts for threat conditions that the vehicle operator recognizes or can see (i.e., are within the operator's current field of view and are unobstructed). Also, in certain embodiments, the associated alert or alerts may be presented only at the selected display location rather than at multiple display locations, which may clutter the display and potentially reduce the likelihood that the operator will recognize and evaluate the alert(s). In some embodiments, the processing system may generate an alert for the V2X-identified threat on the determined display location that includes only the V2X-identified threat on the determined display location in response to determining that the V2X-identified threat has a higher priority than other identified threats. In other words, the processing system may determine that the V2X-identified threat is more important than another threat and therefore generate an alert for the V2X-identified threat but not the other threats. For example, in the situation described with respect to the environment of Figure 1, if the processing system determines that the threat posed by a third vehicle (e.g., 103) is greater than the threat posed by a pedestrian (e.g., 50), the processing system may generate an alert for the third vehicle at the determined display location that does not mention the pedestrian. As another example, if the processing system determines that the operator has already seen or recognized the threat posed by the third vehicle (e.g., 103), the processing system may generate an alert for the pedestrian (e.g., 50) at the determined display location that does not mention the third vehicle.

[0089] The means for performing the operations of block 426 include the processing system 164, 240, 270, 300, 350, the memory 166, the input module 168, the output module 170, the wireless module 172, the threat assessment module 314, the operator perception assessment module 316, and the vehicle safety and collision avoidance module 352.

[0090] In some embodiments, the processing system may obtain or retrieve an alert threshold from memory, In such embodiments, in response to determining that the risk score satisfies the alert threshold retrieved from memory, the processing system may transmit an alert message to the determined display location.

[0091] In some embodiments, the processing system may determine the alert threshold based on information received from one or more sensors and / or the ITS. For example, in a situation where four vehicles are in close proximity to a vehicle, the vehicles may communicate information to one another (e.g., via one or more V2X messages), including information such as their location, distance from the vehicle, and direction. In such an embodiment, the processing system (e.g., of one or more of the vehicles) may select the alert threshold such that it is statistically more likely that one or two of the vehicles will send a warning message to the vehicle, rather than all four vehicles sending a warning message.

[0092] 4B-4G are process flow diagrams of example operations 402, 404, 406, 408, 410, and 412 that may be performed as part of a method 400 for presenting relevant alerts to a vehicle operator according to various embodiments. Operations 402, 404, 406, 408, 410, and 412 may be performed by a processing system (e.g., 164, 240, 270, 300, 350) of a V2X-onboard equipment (e.g., 111, 112, 113) of a vehicle (e.g., 101, 102, 103).

[0093] FIG. 4B illustrates operations 402 that may be performed by a processing system in some embodiments to determine the location of an area outside the vehicle that contains a V2X identified threat.

[0094] 1-4B, after receiving the V2X communication in block 420 of the method 400, the processing system may determine in block 428 the location of an area outside the vehicle that includes the V2X identified threat. For example, the processing system may determine where the V2X identified threat is located within a 360 degree radius of the vehicle. In addition, the processing system may determine how far away the threat is, how fast the threat is moving, if applicable, and / or the current trajectory of the threat, if applicable. As described above, the processing system may receive information from various sensors on the vehicle and / or in the V2X communication. Means for performing the operations of block 428 include the processing system 164, 240, 270, 300, 350, the memory 166, the input module 168, the output module 170, the wireless module 172, the threat assessment module 314, and the vehicle safety and collision avoidance module 352.

[0095] In block 430, the processing system may determine the direction of the operator's gaze. The processing system may use internal vehicle sensors, including internal cameras, and / or OMS output to determine the operator's gaze direction. The processing system may analyze inputs related to the operator's gaze, such as head tilt / rotation angles and / or the operator's eye focus. Means for performing the operations of block 430 include the processing system 164, 240, 270, 300, 350, the memory 166, the input module 168, the output module 170, the wireless module 172, the operator perception assessment module 316, and the vehicle safety and collision avoidance module 352. Following the operations in block 430, the processing system may perform the operations in block 422 of the method 400 as described.

[0096] FIG. 4C illustrates an operation 406 that may be performed by a processing system in some embodiments. Referring to FIGS. 1-4C, after determining the location of the area outside the vehicle containing the V2X identified threat (block 428 of operation 402), the processing system may determine in block 432 whether the location of the area outside the vehicle containing the V2X identified threat is within a recognized blind spot of the vehicle operator. For example, certain areas outside the vehicle may be known to be blocked from view by the operator due to structural columns and other vehicle elements that limit the operator's view. From the operator's point of view (POV), these blocked areas are referred to as blind spots. Means for performing the operations of block 430 include the processing system 164, 240, 270, 300, 350, the memory 166, the input module 168, the output module 170, the wireless module 172, the operator perception assessment module 316, and the vehicle safety and collision avoidance module 352. Following the operation at block 432, the processing system may perform the operation at block 422 of the method 400 as described.

[0097] 4D illustrates operation 406 that may be performed by a processing system in some embodiments. With reference to FIGS. 1-4D, after receiving a V2X communication in block 420, the processing system may determine a direction of the operator's gaze in block 430, as described with reference to operation 402. Following the operation in block 430, the processing system may perform the operation in block 422 of method 400, as described.

[0098] FIG. 4E illustrates operation 408 that may be performed by a processing system in some embodiments. Referring to FIGS. 1-4E, after receiving V2X communication in block 420, the processing system may determine in block 434 whether a threat acknowledgment has been received from an operator regarding the V2X identified threat. For example, in some embodiments, the threat acknowledgment received from the operator may be an explicit input received from the operator (e.g., verbal input / command, user interface input, user gesture such as a predefined blinking pattern to acknowledge acknowledgment). Alternatively or additionally, in some embodiments, the threat acknowledgment received from the operator may be an implicit acknowledgment (e.g., pupil dilation, smart glass observation, attention to the direction of the associated threat, machine learning detection of the user's reaction to the associated threat as learned through training cycles, the direction of the user's gaze, etc.). The means for performing the operations of block 434 include the processing system 164, 240, 270, 300, 350, the memory 166, the input module 168, the output module 170, the wireless module 172, the operator perception assessment module 316, and the vehicle safety and collision avoidance module 352. Following the operations at block 434, the processing system may perform the operations at block 422 of the method 400 as described.

[0099] FIG. 4F illustrates operations 410 that may be performed by a processing system in some embodiments. Referring to FIGS. 1-4F, after receiving V2X communications in block 420, the processing system may receive V2X occlusion data in block 436 indicating that conditions exist that reduce visibility of a V2X identified threat to an operator. For example, in block 436, the processing system may receive information regarding weather conditions (e.g., darkness, fog, rain, snow, smog, etc.) that may make it difficult for the operator to see, nearby obstacles (e.g., one or more other vehicles, objects, structures, creatures), etc. Such conditions and / or elements may be considered occlusions if they partially or completely block and / or impede the operator's view. Means for performing the operations of block 436 include a processing system 164, 240, 270, 300, 350, a memory 166, an input module 168, an output module 170, a wireless module 172, an operator perception assessment module 316, and a vehicle safety and collision avoidance module 352. Following the operation at block 436, the processing system may perform the operation at block 422 of the method 400 as described.

[0100] FIG. 4G illustrates operations 412 that may be performed by the processing system in some embodiments. With reference to FIGS. 1-4G, the processing system may perform operations to increase a notification priority of a V2X-identified threat in response to the vehicle operator determining that the V2X-identified threat is not recognized by the vehicle operator at block 422 of method 400. In response to the processing system determining that the vehicle operator is not recognized by the V2X-identified threat at block 438, the processing system may increase a notification priority of the V2X-identified threat in response to the vehicle operator determining that the V2X-identified threat is not recognized by the vehicle operator at block 438. For example, the processing system may determine that the current direction of the operator's gaze is not in the direction of the V2X-identified threat, implying that the operator is not aware of the threat. As another example, the processing system may observe that there is no change (e.g., dilation) in the operator's pupils or no change in facial movement or expression, which may suggest that the operator is not aware of the threat. As described above, the processing system may receive information from various sensors in the vehicle and / or in the V2X communications. In some embodiments, the threat assessment module 314 or other vehicle systems may maintain rankings of various threats to the vehicle or vehicle occupants and provide a hierarchy of notifications regarding those threats. In this manner, the rankings of various threats may be adjusted (i.e., increased or decreased) depending on whether the operator has confirmed the threat. The means for performing the operations of block 438 include the processing system 164, 240, 270, 300, 350, the memory 166, the input module 168, the output module 170, the wireless module 172, the threat assessment module 314, and the vehicle safety and collision avoidance module 352.

[0101] In block 440, the processing system may determine whether the increased notification priority of the V2X identified threat is a higher priority than other identified threats. For example, a particular V2X identified threat may normally have a relatively low risk ranking, but because the vehicle operator is not aware of the threat, its low risk ranking may increase and become a higher ranking risk than other risks that previously had a higher ranking. In such a case, the processing system may determine that the increased notification priority of the V2X identified threat is a higher priority than other identified threats. The means for performing the operations of block 430 include the processing system 164, 240, 270, 300, 350, the memory 166, the input module 168, the output module 170, the wireless module 172, the threat assessment module 314, and the vehicle safety and collision avoidance module 352.

[0102] Following the operations at block 440, the processing system may perform operations of identifying a display location at block 424 and generating an alert regarding the V2X-identified threat of method 400 at block 426, as described. In particular, the processing system may generate an alert at block 426 that includes V2X-identified threats of increased notification priority, but excludes information regarding other lower priority identified threats as described. Also, the processing system may present the alert to only a single display location (versus multiple redundant renderings of the alert).

[0103] A number of different cellular and mobile communication services and standards are available or are contemplated in the future, all of which may implement and benefit from the various embodiments, including, but not limited to, Third Generation Partnership Project (3GPP), Long Term Evolution (LTE) systems, third generation wireless mobile communication technologies (3G), fourth generation wireless mobile communication technologies (4G), fifth generation wireless mobile communication technologies (5G), Global System for Mobile Communications (GSM), Universal Mobile Telecommunications System (UMTS), 3GSM, General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA) systems (cdmaOne, CDMA1020™, etc.), Enhanced Data Rates for GSM Evolution (EDGE), Advanced Mobile Phone System (AMPS), Digital AMPS (IS-136 / TDMA), Evolution Data Optimized (EV-DO), Digital Enhanced Cordless Telecommunications (DECT), and the like. Examples of common standards include wireless telecommunications, Worldwide Interoperability for Microwave Access (WiMAX), Wireless Local Area Network (WLAN), Wi-Fi Protected Access I & II (WPA, WPA2), and Integrated Digital Enhanced Network (iDEN). Each of these technologies involves, for example, the transmission and reception of voice, data, signaling, and / or content messages. It should be understood that any reference to terminology and / or technical details relating to a particular telecommunications standard or technology is for illustrative purposes only and does not limit the scope of the claims to any particular communications system or technology unless specifically recited in the claim language.

[0104] Implementation examples are described in the following paragraphs. Some of the implementation examples below are described with respect to example methods, but further example implementations may include the example methods described in the following paragraphs implemented by a processing system of a V2X-equipped device including a processor configured with processor-executable instructions for performing operations of the methods of the following implementation examples, and the example methods described in the following paragraphs performed by a processing system of a V2X-equipped device including means for performing functions of the methods of the following examples, and the example methods described in the following paragraphs may be implemented as a non-transitory processor-readable storage medium having stored thereon processor-executable instructions configured to cause the processing system of the V2X-equipped device to perform operations of the methods of the following examples.

[0105] Example 1. A method executed by a processor of a vehicle for presenting relevant warnings to a vehicle operator, the method comprising: receiving a V2X communication including information regarding a V2X-identified threat to the vehicle or vehicle occupants; determining whether the vehicle operator recognizes the V2X-identified threat; and, in response to determining that the vehicle operator does not recognize the V2X-identified threat, determining a display location that is most likely to receive the vehicle operator's attention; and generating an alert regarding the V2X-identified threat on the determined display location.

[0106] Example 2. The method of example 1, wherein generating an alert regarding a V2X identified threat on the determined display location further includes generating a display that excludes alerts regarding any threat conditions recognized by the vehicle operator.

[0107] Example 3. A method as described in either of Examples 1 or 2, further comprising determining a location of an area outside the vehicle including the V2X-identified threat and determining a direction of the vehicle operator's line of sight, and determining whether the vehicle operator recognizes the V2X-identified threat comprises determining that the vehicle operator does not recognize the V2X-identified threat in response to determining that the location of the area outside the vehicle including the V2X-identified threat is not within the vehicle operator's field of view in the determined direction of the vehicle operator's line of sight.

[0108] Example 4. The method of any of Examples 1 to 3, further comprising: determining a location of an area outside the vehicle including the V2X-identified threat; and determining whether the location of the area outside the vehicle including the V2X-identified threat is within a recognized blind spot of the vehicle operator, wherein determining whether the vehicle operator has recognized the V2X-identified threat comprises determining that the vehicle operator does not recognize the V2X-identified threat in response to determining that the location of the area outside the vehicle including the V2X-identified threat is within a recognized blind spot of the vehicle operator.

[0109] Example 5. The method of any of Examples 1 to 4, further comprising determining a direction of the vehicle operator's gaze, wherein determining the display locations most likely to receive the vehicle operator's attention comprises identifying display locations that are visible within the vehicle operator's field of view in the determined direction of the vehicle operator's gaze.

[0110] Example 6. The method of any of Examples 1 to 5, wherein determining the display location most likely to receive the attention of a vehicle operator includes determining which of a plurality of vehicle displays is designated as a preferred display for a V2X-identified threat.

[0111] Example 7. The method of any of Examples 1 to 6, further comprising determining whether a threat confirmation has been received from the vehicle operator regarding the V2X identified threat, wherein determining whether the vehicle operator has recognized the V2X identified threat comprises determining that the vehicle operator has not recognized the V2X identified threat in response to determining that a threat confirmation has not been received from the vehicle operator regarding the V2X identified threat.

[0112] Example 8. The method of any of Examples 1 to 7, further comprising receiving V2X occlusion data indicating the presence of conditions that reduce visibility of the V2X-identified threat to the vehicle operator, and determining whether the vehicle operator is aware of the V2X-identified threat comprises, in response to receiving the V2X occlusion data, determining that the vehicle operator is not aware of the V2X-identified threat.

[0113] Example 9. The method of any of Examples 1 to 8, further comprising: in response to determining that the vehicle operator is not aware of the V2X-identified threat, increasing a notification priority of the V2X-identified threat; and determining whether the increased notification priority of the V2X-identified threat is a higher priority than other identified threats; and generating an alert regarding the V2X-identified threat on the determined display location comprises, in response to determining that the increased notification priority of the V2X-identified threat is a higher priority than other identified threats, generating an alert regarding the V2X-identified threat but excluding alerts regarding the other identified threats on the determined display location.

[0114] Example 10. A method according to any of Examples 1 to 9, wherein the V2X communication including information regarding a V2X identified threat is received from a source remote from the vehicle.

[0115] The various embodiments shown and described are provided merely as examples to illustrate various features of the claims. However, features shown and described with respect to any given embodiment are not necessarily limited to the associated embodiment, and may be used with or combined with other embodiments shown and described. Moreover, the claims are not limited by any one exemplary embodiment.

[0116] The above method descriptions and process flow diagrams are provided as illustrative examples only and do not require or imply that the operations of the various embodiments must be performed in the order presented. As will be appreciated by one of ordinary skill in the art, the order of operations in the above-described embodiments may be performed in any order. Furthermore, words such as "thereafter," "then," and "next" do not limit the order of operations. These words are used to guide the reader through the method descriptions. Furthermore, any reference to a claim element in the singular, for example using the articles "a," "an," or "the," should not be construed as limiting the element to the singular.

[0117] The various exemplary logical blocks, modules, components, circuits, and algorithmic operations described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or a combination of both. To clearly illustrate this interchangeability of hardware and software, the various exemplary components, blocks, modules, circuits, and operations have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends on the particular application and design constraints imposed on the overall system. Those skilled in the art may implement the described functionality in various ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the claims.

[0118] The hardware used to implement the various example logic, logic blocks, modules, and circuits described in connection with the embodiments disclosed herein may be implemented or performed using general purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. A general purpose processor may be a microprocessor, but alternatively, the processor may be any conventional processor, controller, microcontroller, or state machine. A processor may also be implemented as a combination of receiver smart objects, e.g., a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration. Alternatively, some operations or methods may be performed by circuitry specific to a given function.

[0119] In one or more embodiments, the functions described may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored as one or more instructions or codes on a non-transitory computer-readable or processor-readable storage medium. The operations of a method or algorithm disclosed herein may be embodied in a processor-executable software module or processor-executable instructions that may reside on a non-transitory computer-readable or processor-readable storage medium. A non-transitory computer-readable or processor-readable storage medium may be any storage medium that may be accessed by a computer or processor. By way of example and not limitation, such a non-transitory computer-readable or processor-readable storage medium may include RAM, ROM, EEPROM, FLASH memory, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage smart objects, or any other medium that may be used to store desired program code in the form of instructions or data structures and that may be accessed by a computer. Disk and disc as used herein include compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk, and Blu-ray disc, where disks typically reproduce data magnetically and discs reproduce data optically using lasers. Combinations of the above are also included within the scope of non-transitory computer-readable and processor-readable media. Additionally, operations of a method or algorithm may be present as one or any combination or set of code and / or instructions on a non-transitory processor-readable storage medium and / or a non-transitory computer-readable storage medium, which may be incorporated into a computer program product.

[0120] The foregoing description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the claims. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other embodiments without departing from the scope of the claims. Thus, the present disclosure is not intended to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the following claims and the principles and novel features disclosed herein.

Claims

1. A method executed by a vehicle's processor to present relevant warnings to a vehicle operator, comprising: Receiving V2X communication containing information about threats identified by vehicle-to-everything (V2X); Determining whether the vehicle operator has recognized the threats identified by the V2X; In response to determining that the vehicle operator has not recognized the threats identified by the V2X, determining the display location most likely to draw the vehicle operator's attention; Generating an alert regarding the threat identified by the V2X on the determined display location, wherein the determined display location changes such that an alert message moves from one display of the vehicle to another in response to the operator's eye or head movement The method comprising.

2. The step of generating the alert regarding the threat identified by the V2X on the determined display location includes generating a display excluding alerts regarding any threat conditions recognized by the vehicle operator. The method according to claim 1.

3. Determining the location of an area outside the vehicle that includes the threat identified by the V2X; Determining the direction of the vehicle operator's line of sight; and Further comprising, The step of determining whether the vehicle operator has recognized the threat identified by the V2X includes determining that the vehicle operator has not recognized the threat identified by the V2X in response to determining that the location of the area outside the vehicle that includes the threat identified by the V2X is not within the vehicle operator's field of view in the determined direction of the vehicle operator's line of sight. The method according to claim 1.

4. Determining the location of an area outside the vehicle that includes the threat identified by the V2X; Determining whether the location of the area outside the vehicle that includes the threat identified by the V2X is within the recognized blind spot of the vehicle operator; and Further comprising, The step of determining whether the vehicle operator has recognized the threat identified by the V2X includes determining that the vehicle operator has not recognized the threat identified by the V2X in response to determining that the location of the area outside the vehicle including the threat identified by the V2X is within the recognized blind spot of the vehicle operator. The method according to claim 1.

5. The method according to claim 1, further comprising the step of determining the direction of the vehicle operator's line of sight, and the step of determining the display location most likely to receive the vehicle operator's attention includes identifying a display location visible within the vehicle operator's field of view in the determined direction of the vehicle operator's line of sight.

6. The step of determining the display location most likely to receive the vehicle operator's attention includes determining which of a plurality of vehicle displays is designated as a preferred display for a threat identified by V2X. The method according to claim 1.

7. Further comprising the step of determining whether a threat confirmation has been received from the vehicle operator regarding the threat identified by the V2X. The step of determining whether the vehicle operator has recognized the threat identified by the V2X includes determining that the vehicle operator has not recognized the threat identified by the V2X in response to determining that no threat confirmation has been received from the vehicle operator regarding the threat identified by the V2X. The method according to claim 1.

8. Further comprising the step of receiving V2X occlusion data indicating that there are conditions for reducing the visibility of the threat identified by the V2X to the vehicle operator. The step of determining whether the vehicle operator has recognized the threat identified by the V2X includes determining that the vehicle operator has not recognized the threat identified by the V2X in response to receiving the V2X occlusion data. The method according to claim 1.

9. In response to determining that the vehicle operator has not recognized the threat identified by the V2X, increasing the notification priority of the threat identified by the V2X. Determining whether the increased notification priority of the threat identified by the V2X is higher than the priority of other identified threats further comprising The step of generating the alert regarding the threat identified by the V2X on the determined display location includes generating the alert regarding the threat identified by the V2X in response to determining that the increased notification priority of the threat identified by the V2X is higher than the priority of other identified threats, and excluding alerts regarding other identified threats on the determined display location The method according to claim 1

10. The V2X communication including the information regarding the threat identified by the V2X is received from a remote source from the vehicle, the method according to claim 1

11. A vehicle, comprising A wireless module A processor coupled to the wireless module, the processor Receiving V2X communication including information regarding a threat identified by vehicle-to-everything (V2X) Determining whether a vehicle operator has recognized the threat identified by the V2X In response to determining that the vehicle operator has not recognized the threat identified by the V2X, determining a display location where the vehicle operator is most likely to receive attention Generating an alert regarding the threat identified by the V2X on the determined display location, wherein the determined display location changes such that an alert message moves from one display of the vehicle to another in response to the operator's eye or head movement A processor configured with processor-executable instructions for performing the above A vehicle comprising the above

12. The vehicle according to claim 11, wherein the processor is further configured with processor-executable instructions for generating the alert regarding the threat identified by the V2X on the determined display location, and excluding alerts regarding any threat conditions determined by the processor that the vehicle operator has recognized

13. The processor Determining the location of an area outside the vehicle including the threat identified by the V2X Determining the direction of the vehicle operator's line of sight and is further configured with processor-executable instructions for the above The processor is further configured with processor-executable instructions for determining that the vehicle operator does not recognize the threat identified by the V2X in response to determining that a location of the area outside the vehicle that includes the threat identified by the V2X is not within the field of view of the vehicle operator in the determined direction of the vehicle operator's line of sight. The vehicle according to claim 11.

14. The processor is determining a location of an area outside the vehicle that includes the threat identified by the V2X, and determining whether the location of the area outside the vehicle that includes the threat identified by the V2X is within a recognized blind spot of the vehicle operator. The processor is further configured with processor-executable instructions for determining that the vehicle operator does not recognize the threat identified by the V2X in response to determining that the location of the area outside the vehicle that includes the threat identified by the V2X is within the recognized blind spot of the vehicle operator. The processor is further configured with processor-executable instructions for The vehicle according to claim 11.

15. A non-transitory processor-readable storage medium storing thereon processor-executable instructions configured to cause a processor of a vehicle to perform operations, the operations including: receiving V2X communication including information regarding a threat identified by vehicle-to-everything (V2X); determining whether a vehicle operator recognizes the threat identified by the V2X; determining a display location that is most likely to receive the attention of the vehicle operator in response to determining that the vehicle operator does not recognize the threat identified by the V2X; and generating an alert regarding the threat identified by the V2X on the determined display location, wherein the determined display location changes such that an alert message moves from one display of the vehicle to another display in response to the movement of the operator's eyes or head. A non-transitory processor-readable storage medium including the above.

Images