Autonomous driving at intersections based on perception data

Abstract

System that includes the following: an intersection component that is configured to determine that a parent vehicle is near an intersection; a camera system configured to capture an image of a proximal vehicle; a boundary component configured to identify a subsection of the image containing a direction indicator on the proximal vehicle; and a prediction component configured to predict the future movement of the proximal vehicle through the intersection based on the state of the direction indicator.

Description

TECHNICAL FIELD

[0001] The disclosure relates generally to methods, systems, and apparatus for automated driving or for assisting a driver, and more specifically relates to methods, systems, and apparatus for autonomously making decisions at intersections using predictive modeling of the behavior of surrounding vehicles enabled by perception of vehicle motion, turn signals, driver body language, and / or vehicle-to-vehicle (V2V) or vehicle-to-infrastructure (V2X) information regarding previous states of the vehicles. BACKGROUND

[0002] Motor vehicles provide a significant portion of transportation for commercial, public, and private entities. Autonomous vehicles and driver assistance systems are currently being developed and deployed to provide safety, reduce much of the required user input, or even eliminate user involvement entirely. Some driver assistance systems, such as crash avoidance systems, can monitor the driving, positions, and speed of the vehicle and other objects while a human is driving. If the system detects that a crash or impact is imminent, the crash avoidance system can intervene and apply braking, steer the vehicle, or perform other avoidance or safety maneuvers. As another example, autonomous vehicles can drive and navigate a vehicle with little or no user input.However, due to the hazards involved in driving and the cost of vehicles, it is critical that even in situations where both autonomous and human-driven vehicles are present, autonomous vehicles and driver assistance systems operate safely and are able to accurately navigate roads and avoid other vehicles.

[0003] US 2011 / 0 050 460 A1 discloses a method and apparatus for improving safety when a driver of a vehicle, e.g., an automobile, is having a conversation with another party (or parties) using a mobile telecommunications device, e.g., a mobile phone. In particular, a situation requiring an increased level of attention from the driver is automatically detected, and in response thereto, an acoustic warning signal is automatically issued to at least one of the remote conversation participants. The detection of a situation requiring increased attention from the driver and / or the generation of the acoustic warning signal for the (one or more) remote party(ies) can be performed by the driver's mobile telecommunications device (e.g., the mobile phone) or by a network element of the telecommunications network used to conduct the call.

[0004] US 2014 / 0 100 770 A1 discloses a vehicle comprising a control unit and a display. At least four sensors are coupled to the control unit to detect the distance and direction of other vehicles around the user's vehicle. An environmental object pattern generator is coupled to the control unit to generate an environmental object pattern based on the detected distance and direction, and the environmental pattern is displayed on the display.

[0005] US 2015 / 0 166 062 A1 discloses a control system for a vehicle comprising a camera mounted on a vehicle and having a field of view outside the vehicle. An image processor is operable to process the image data acquired by the camera to at least detect objects or other vehicles. The control system is operable to control a steering system of the vehicle in response to determining a traffic condition. The control system responds, at least in part, to image processing of the acquired image data to detect a lane-splitting vehicle approaching or adjacent to the vehicle. While the vehicle is traveling in an occupied lane and in response to detecting the lane-splitting vehicle approaching the vehicle, the control system controls the steering system of the vehicle to move the vehicle in a direction away from a side region of the occupied lane in which the lane-splitting vehicle is detected. BRIEF DESCRIPTION OF THE DRAWINGS

[0006] Non-limiting and non-exhaustive implementations of the present disclosure are described with reference to the following figures, in which like reference numerals refer to like parts throughout the several views unless otherwise specified. The advantages of the present disclosure will be better understood with reference to the following description and the accompanying drawings, in which: Fig. 1 is a schematic block diagram illustrating an implementation of a vehicle control system including an automated driving / assistance system; Fig. Figure 2 illustrates a perspective view of a vehicle; Fig. 3 is a schematic diagram illustrating a plan view of an exemplary road environment; Fig. 4 is a schematic block diagram illustrating the prediction of future vehicle movements according to one implementation; Fig. 5 is a schematic block diagram illustrating example components of a driver intent component according to one implementation; Fig. 6 is a schematic block diagram illustrating a method for predicting vehicle movements according to one implementation; and Fig. 7 is a schematic block diagram illustrating another method for predicting vehicle movements according to one implementation. DETAILED DESCRIPTION

[0007] The applicants have recognized that the first generations of autonomous vehicles must operate safely alongside human drivers. Specifically, intersections, including intersections of four equal-level roads, are one of the more challenging scenarios an autonomous vehicle must handle, because decisions about when to proceed into intersections must be made on a case-by-case basis in the presence of human drivers whose behavior is not standardized.

[0008] Consequently, it is desirable for these autonomous vehicles to be able to predict the movement of surrounding vehicles, even when driven by a human. Applicants have recognized that autonomous vehicles can more accurately predict what a human driver will do or intends to do based on the movement of other vehicles, the body language of the drivers, the vehicles' turn signals (or lack thereof), and wireless communications, such as vehicle-to-vehicle (V2V) or vehicle-to-infrastructure (V2X). For example, an autonomous vehicle can interpret the body language of a human driver or the other above information about a vehicle other than the autonomous vehicle. Consequently, the autonomous vehicle does not depend on the other vehicle or a driver of the other vehicle to provide specific information about what the human driver intends to do.Furthermore, the autonomous vehicle may be able to adapt to the actions of a human driver, which may be unpredictable compared to more standardized behavior in an autonomous driving system.

[0009] This disclosure presents systems, methods, and apparatus for autonomous driving or driver assistance systems for utilizing information from V2V or V2X, including information about previous states of other vehicles that cannot be detected through perception alone, with information from the perception systems about the body language of drivers in other vehicles and the motion signals and turn signals of other vehicles. These various types of information can be fused to form predictions about the movement of other vehicles at intersections. These predictions can then support the autonomous decision-making process of whether to wait or proceed into the intersection.In one embodiment, neural networks or other models can be trained to correlate the past movement of vehicles, driver body language, current vehicle movement, and / or turn signal information with one or more expected subsequent movements of these surrounding vehicles. Perception can be performed using cameras, light ranging and detection (LIDAR), radar, ultrasonic, or other range sensors to assist in determining areas of interest within a given camera image.

[0010] In one embodiment, the perception algorithms, along with previous vehicle states, provide the locations of other vehicles via V2V or V2X communication, using this information to track the vehicles. In one embodiment, the systems can identify regions of interest to extract information related to driver intent, such as head orientation, gaze direction, turn signals, and vehicle movement. According to one embodiment, an autonomous vehicle can locate vehicles within a current 360-degree frame of all data sensors.The autonomous vehicle can define bounding boxes around the areas of the image where the drivers of these vehicles would be found and provide the image data and range sensor data within the bounding boxes to algorithms that estimate the driver's head orientation and gaze direction, as well as detect the driver's gestures. This information can be interpreted to determine the driver's intent regarding the predicted following vehicle movement of the other vehicles. These predictions can be considered to decide when to enter the intersection.

[0011] In one embodiment, the prediction algorithms for predicting what a human driver will do based on body language or other information may be hard-coded or learned and modified using artificial intelligence. For example, a human driver's gestures or body language may be monitored or learned during a driving event, and the gestures or body language may then be associated with a resulting vehicle movement or a timeline of the vehicle movement.

[0012] In the following disclosure, reference is made to the accompanying drawings, which form a part hereof, and in which is shown by way of illustration specific implementations in which the disclosure may be practiced. It is to be understood that other implementations may be utilized and structural changes may be made without departing from the scope of the present disclosure. References throughout this specification to "one embodiment," "an embodiment," "an exemplary embodiment," etc., indicate that the described embodiment may include a particular feature, structure, or characteristic, but every embodiment may not necessarily include the particular feature, structure, or characteristic. Further, such terms do not necessarily refer to the same embodiment.Furthermore, when a particular feature, structure, or characteristic is described in connection with one embodiment, it is asserted that it is within the knowledge of one skilled in the art to affect such feature, structure, or characteristic in connection with other embodiments, whether or not explicitly described.

[0013] An “autonomous vehicle,” as used herein, may be a vehicle that acts or operates entirely independently of a human driver; or may be a vehicle that, in some cases, acts or operates independently of a human driver while, in other cases, a human driver may be capable of operating the vehicle; or may be a vehicle that is predominantly operated by a human driver but with the assistance of an automated driving / assistance system.

[0014] Implementations of the systems, devices, and methods disclosed herein may include or utilize a special-purpose or general-purpose computer, including computer hardware such as one or more processors and system memory, as discussed in more detail below. Implementations within the scope of the present disclosure may also include physical and other computer-readable media for carrying or storing computer-executable instructions and / or data structures. Such computer-readable media may be any available media accessible by a general-purpose or special-purpose computer system. Computer-readable media that stores computer-readable instructions are computer storage media (computer storage devices). Computer-readable media that carry computer-executable instructions are transmission media.Thus, by way of example and not limitation, implementations of the disclosure may include at least two distinct types of computer-readable media: computer storage media (computer storage devices) and transmission media.

[0015] Computer storage media (computer storage devices) include RAM, ROM, EEPROM, CD-ROM, solid state drives ("SSDs") (e.g., based on RAM), flash memory, phase change memory ("PCM"), other types of memory, other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired program code means in the form of computer-executable instructions or data structures and that can be accessed by a general-purpose or special-purpose computer.

[0016] An implementation of the devices, systems, and methods disclosed herein may communicate over a computer network. A "network" is defined as one or more data connections that enable the transport of electronic data between computer systems and / or modules and / or other electronic devices. When information is transmitted to or provided to a computer over a network or other communications connection (either hardwired, wireless, or a combination of hardwired and wireless), the computer properly considers the connection to be a transmission medium. The transmission media may include a network and / or data connections that can be used to transmit the desired program code means in the form of computer-executable instructions or data structures and that can be accessed by a general-purpose or special-purpose computer.Combinations of the above should also be included within the scope of the computer-readable media.

[0017] Computer-executable instructions include, for example, instructions and data that, when executed in a processor, cause a general-purpose computer, a special-purpose computer, or a special-purpose processing device to perform a particular function or set of functions. The computer-executable instructions may be, for example, binary programs, intermediate format instructions such as assembly language, or even source code. Although the subject matter has been described in language specific to structural features and / or other methodological acts, it is recognized that the subject matter defined in the appended claims is not necessarily limited to the features or acts described above. Instead, the described features and acts are disclosed as exemplary forms for implementing the claims.

[0018] Those skilled in the art will recognize that the disclosure may be practiced in network computing environments having many types of computer system configurations, including a dashboard computer, personal computers, desktop computers, laptop computers, message processors, handheld devices, multiprocessor systems, microprocessor-based or programmable consumer electronics, network PCs, minicomputers, mainframe computers, cellular phones, PDAs, tablets, pagers, routers, switches, various storage devices, and the like. The disclosure may also be practiced in distributed system environments in which both local and remote computer systems linked by a network (either through hardwired data links, wireless data links, or through a combination of hardwired and wireless data links) perform tasks.In a distributed system environment, the program modules can be located in both local and remote storage devices.

[0019] Furthermore, the functions described herein may be implemented in one or more of the following, where appropriate: hardware, software, firmware, digital components, or analog components. For example, one or more application-specific integrated circuits (ASICs) may be programmed to perform one or more of the systems and procedures described herein. Certain terms are used throughout the following description and claims to refer to specific system components. As one skilled in the art will recognize, the components may be referred to by various names. This document does not intend to distinguish between components that differ in name but not in function.

[0020] The figures illustrate Fig. 1 shows a vehicle control system 100 including an automated driving / assistance system 102. The automated driving / assistance system 102 may be used to automate or control the operation of a vehicle, such as a passenger car, a truck, a minibus, a bus, a large truck, an emergency vehicle, or any other motor vehicle for transporting people or goods, or to provide assistance to a human driver. The automated driving / assistance system 102 may, for example, control one or more of the vehicle's braking, steering, acceleration, lighting, alarms, driver notifications, radio, or any other auxiliary systems. In another example, the automated driving / assistance system 102 may not be capable of providing any control of driving (e.g.,The automated driving / assistance system 102 may provide guidance for specific vehicle actions (e.g., steering, acceleration, or braking), but may provide notifications and alerts to assist a human driver in driving safely. The automated driving / assistance system 102 includes a driver intent component 104 that may predict future movement of another vehicle based on the V2V or V2X communications and / or the turn signals and / or the driver's body language and / or the vehicle's motion. For example, the driver intent component 104 may estimate an intent of the driver of another vehicle (e.g., a vehicle that does not include the vehicle control system 100) based on a turn signal state, a driver gesture, a driver gaze direction, and / or a driver head orientation.

[0021] The vehicle control system 100 also includes one or more sensor systems / devices for detecting the presence of nearby objects or for determining the location of a host vehicle (e.g., a vehicle including the vehicle control system 100) or nearby objects. The vehicle control system 100 may include, for example, one or more radar systems 106, one or more LIDAR systems 108, one or more camera systems 110, a global positioning system (GPS) 112, and / or one or more ultrasonic systems 114. The vehicle control system 100 may include a data store 116 for storing relevant or useful data for navigation and safety, such as map data, driving history, or other data.The vehicle control system 100 may also include a transceiver 118 for wireless communication with a cellular or wireless network, other vehicles, the infrastructure, or any other communication system. The vehicle control system 100 may include vehicle control actuators 120 for controlling various aspects of the vehicle's operation, such as electric motors, switches, or other actuators for controlling braking, acceleration, steering, or the like. The vehicle control system 100 may also include one or more displays 122, speakers 124, or other devices so that messages can be provided to a human driver or passenger.The display 122 may include a head-up display, an instrument panel display or indicator, a display screen, or any other visual indicator visible to a driver or passenger of a vehicle. The speakers 124 may include one or more speakers of a vehicle's sound system or may include a speaker dedicated to driver notification.

[0022] It is recognized that the embodiment according to Fig. 1 is provided merely by way of example. Other embodiments may include fewer or additional components without departing from the scope of the disclosure. In addition, the illustrated components may be combined or included within other components without limitation. For example, the driver intent component 104 may be separate from the automated driving / assistance system 102, and the data store 116 may be included as part of the automated driving / assistance system 102 and / or part of the driver intent component 104.

[0023] Radar system 106 may include any radar system known in the art. Generally, radar system 106 operates by transmitting radio signals and detecting reflections from objects. In ground applications, radar may be used to detect physical objects, such as other vehicles, parking barriers or wheel stops, landscapes (such as trees, cliffs, rocks, hills, or the like), roadsides, signs, buildings, or other objects. Radar system 106 may use the reflected radio waves to determine a size, shape, distance, surface texture, or other information about a physical object or material. For example, radar system 106 may sweep an area to obtain data about the objects within a specific range and viewing angle of radar system 106.In one embodiment, radar system 106 is configured to generate perception information from an area proximate the vehicle, such as one or more areas near the vehicle or one or more areas surrounding the vehicle. For example, radar system 106 may obtain data about the areas of the ground or vertical area immediately adjacent to or near the vehicle. Radar system 106 may include one of many widely available commercially available radar systems. In one embodiment, radar system 106 may provide the perception data, including a two-dimensional or three-dimensional map or model, to automated driving / assistance system 102 for reference or processing.

[0024] The LIDAR system 108 may include any LIDAR system in the art. Generally, the LIDAR system 108 operates by emitting lasers with visible wavelengths or infrared wavelengths and detecting the reflections of the laser light from the objects. In ground applications, the lasers may be used to detect physical objects, such as other vehicles, parking barriers or wheel stops, landscapes (such as trees, cliffs, rocks, hills, or the like), roadsides, signs, buildings, or other objects. The LIDAR system 108 may use the reflected laser light to determine a size, shape, distance, surface texture, or other information about a physical object or material. For example, the LIDAR system 108 may sweep an area to obtain data or objects within a specific range and viewing angle of the LIDAR system 108. For example, the LIDAR system 108 may B.Obtain data about the areas of the ground or vertical space immediately adjacent to or near the vehicle. The LIDAR system 108 may include one of many widely available commercially available LIDAR systems. In one embodiment, the LIDAR system 108 may provide the perception data including a two-dimensional or three-dimensional model or map of the detected objects or surfaces.

[0025] The camera system 110 may include one or more cameras, such as visible wavelength cameras or infrared cameras. The camera system 110 may provide a video feed or periodic images that may be processed for object detection, road identification and positioning, or other detection or positioning. In one embodiment, the camera system 110 may include two or more cameras that may be used to obtain a ranging measurement for objects within the field of view (e.g., detect a distance for objects within the field of view). In one embodiment, image processing on the captured camera images or video may be used to detect vehicles, turn signals, drivers, gestures, and / or a driver's body language. In one embodiment, the camera system 110 may include cameras that obtain images for two or more directions around the vehicle.

[0026] GPS system 112 is one embodiment of a positioning system that can provide a geographic location of the vehicle based on satellite and cell tower signals. GPS systems 112 are well known in the art and widely available. Although GPS systems 112 can provide very accurate positioning information, GPS systems 112 generally provide little or no information about the distances between the vehicle and other objects. Instead, they simply provide a location that can then be compared with other data, such as maps, to determine distances to other objects, roads, or points of interest.

[0027] The ultrasonic system 114 can be used to detect objects or distances between a vehicle and objects using ultrasonic waves. For example, the ultrasonic system 114 can emit ultrasonic waves from a location on or near the bumper or a location on the sidewall of a vehicle. The ultrasonic waves, which can travel short distances through the air, can be reflected by other objects and detected by the ultrasonic system 114. Based on a time period between the emission and reception of the reflected ultrasonic waves, the ultrasonic system 114 may be able to detect the precise distances between a bumper or sidewall and any other objects. Due to their shorter range, the ultrasonic systems 114 may be more useful for detecting objects while parked or detecting impending collisions while driving.

[0028] In one embodiment, the radar system(s) 106, the LIDAR system(s) 108, the camera system(s) 110, and the ultrasonic system(s) 114 may detect environmental features or obstacles near a vehicle. For example, the systems 106-110 and 114 may detect other vehicles, pedestrians, people, animals, a number of lanes, a lane width, a shoulder width, a curvature of the road surface, a bend in the direction of the road, speed bumps, lane markings, the presence of intersections, traffic signs, bridges, overpasses, barriers, medians, curbs, and any other details about a road. As another example, the systems 106-110 and 114 may detect environmental features that provide information about structures, objects, or surfaces near the road, such as:the presence of driveways, parking areas, exits / entrances to parking areas, sidewalks, footpaths, trees, fences, buildings, parked vehicles (on or near the road), gates, signs, parking strips or any other structures or objects.

[0029] The data storage 116 stores map data, a driving history, and other data, which may include other navigation data, settings, or operating instructions for the automated driving / assistance system 102. The map data may include location data, such as GPS location data, for streets, parking areas, marked parking spaces, or other locations where a vehicle may be driven or parked. The location data for the streets may include, for example, the location data for specific lanes, such as the direction of the lane, merging lanes, highway or freeway lanes, exit lanes, or any other lane or division of a road. The location data may also include the locations for one or more marked parking spaces within a parking area or for the marked parking spaces along a road.In one embodiment, the map data includes location data about one or more structures or objects on or near the roads or parking lots. For example, the map data may include data regarding the GPS location of signs, bridges, buildings, or other structures, or the like. In one embodiment, the map data may include precise location data with an accuracy within a few meters or within submeter accuracy. The map data may also include location data for trails, dirt roads, or other roads or paths traversable by a land vehicle.

[0030] The driving history (or driving history) may include location data for previous trips or parking locations of the vehicle. For example, the driving history may include GPS location data for previously taken trips or routes. As one example, the driving history may include distance or relative location data regarding guidelines, signs, lane markings, or other objects or features on or near the roadways. The distance or relative location data may be determined based on GPS data, radar data, LIDAR data, camera data, or other sensor data collected during previous or earlier trips undertaken by the vehicle. In one embodiment, the automated driving / assistance system 102 is configured to log the driving data for and during any trips or excursions undertaken by the vehicle in the data store 116.

[0031] The transceiver 118 is configured to receive signals from one or more other data or signal sources. The transceiver 118 may include one or more radios configured to communicate according to various communication standards and / or using various different frequencies. For example, the transceiver 118 may receive signals from other vehicles. Receiving the signals from another vehicle is referred to herein as vehicle-to-vehicle (V2V) communication. In one embodiment, the transceiver 118 may also be used to send information to other vehicles to potentially assist them in locating vehicles or objects.During V2V communication, the transceiver 118 may receive information from other vehicles about their locations, previous locations or conditions, other traffic, accidents, road conditions, the locations of parking barriers or wheel stops, or any other details that may assist the vehicle and / or the automated driving / assistance system 102 in driving accurately or safely. For example, the transceiver 118 may receive updated models or algorithms for use by the driver intent component 104 in detecting vehicle movement, turn signals, or the body language of a driver of another vehicle.

[0032] The transceiver 118 may receive signals from other signal sources located at fixed locations. The infrastructure transceivers may be located at a specific geographical location and may transmit their specific geographical location with a timestamp. Consequently, the automated driving / assistance system 102 may be able to determine a distance from the infrastructure transceivers based on the timestamp and then determine its location based on the location of the infrastructure transceivers. In one embodiment, receiving or transmitting location data from devices or masts at fixed locations is referred to herein as vehicle-to-infrastructure (V2X) communication. V2X communication may also be used to provide information about the locations of other vehicles, their previous states, or the like. The V2X communications may, for example,Contains information about how long a vehicle has been stopped or waiting at an intersection. In one embodiment, the term V2X communication may also encompass V2V communication.

[0033] In one embodiment, the transceiver 118 may send and receive the location data over a cellular network or cell phone connection. For example, the transceiver 118 may receive updated location data for a specific area as a vehicle moves along a road. Similarly, the transceiver 118 may receive historical driving data for a host vehicle or other vehicles that have traveled along a road or parked in a parking area at that location or in that marked parking space. For example, the transceiver 118 may receive data indicating the locations of signs, parking barriers or wheel stops, or other objects that may be detectable using a radar system 106, a LIDAR system 108, a camera system 110, a GPS system 112, or an ultrasound system 114.If the transceiver 118 can receive signals from three or more infrastructure transceivers, the automated driving / assistance system 102 may be able to triangulate its geographic location.

[0034] In one embodiment, the transceiver 118 may send and receive data regarding driver body language. For example, one or more gestures or gaze directions detected during an event and a resulting action by a driver may be uploaded to a server for processing as part of machine learning for estimating driver intent.

[0035] In one embodiment, the automated driving / assistance system 102 is configured to control the driving or navigation of a host vehicle. For example, the automated driving / assistance system 102 may control the vehicle control actuators 120 to travel a path on a road, a parking area, through an intersection, a driveway, or another location. For example, the automated driving / assistance system 102 may determine a path to travel and a speed to travel based on the information or perception data provided by any of the components 106-118. As another example, the automated driving / assistance system 102 may determine when to wait or proceed at an intersection, when to change lanes, when to leave room for another vehicle to change lanes, or the like.

[0036] In one embodiment, the driver intent component 104 is configured to determine an intent of a driver of a nearby vehicle and / or to predict future movement and timing for the movement of a vehicle under the control of a human driver.

[0037] Fig. 2 illustrates an image 200 captured by a camera or other sensor of a vehicle or an autonomous vehicle. The image 200 includes a perspective view of a vehicle 202 with a human driver 204. The view of the vehicle 202 may, for example, be captured by a camera at or near an intersection. The driver intent component 104 may identify the vehicle 202 as a vehicle within the captured image 200. The driver intent component 104 may determine a subregion 206 of the image 200 corresponding to a location or likely location of a driver 204. For example, the driver intent component 104 may identify a portion of the vehicle 202 where a driver 204 would normally sit, such as where a driver seat would be located, etc.

[0038] In one embodiment, the driver intent component 104 may identify one or more windows of the vehicle 202, such as the driver-side window 208 and / or a windshield 210. In one embodiment, the driver intent component 104 may define a subregion 206 of the vehicle 202 based on the location of the likely location of a driver 204, one or more windows 208, 210 of the vehicle 202, or by detecting a face of the driver 204. A subregion 206, which includes where the driver may be located or can reasonably be expected to be located, may be large enough to accommodate the movements of the driver 204 (e.g., the movements of the arms and / or head of the driver 204).By selecting or defining the sub-region 206, the driver intent component 104 may reduce computational power because a smaller region or fewer pixels than the entire image 200 may be analyzed to detect the driver 204, a gaze direction, a head orientation, and / or a gesture.

[0039] The driver intent component 104 may determine one or more subregions 212 of the image 200 corresponding to a location or likely location of a turn signal 214. The turn signal 214 may include a left or right turn signal or other indicator indicating a future turn, a future merge, or a future lane change. For example, a driver 204 may have controls to activate or deactivate one or more turn signals to indicate a direction in which the driver 204 intends to turn or drive the vehicle 202. In one embodiment, the driver intent component 104 may determine the one or more subregions 212 by identifying a headlight and / or tail lamp and / or a bumper or the like. For example, the driver intent component 104 mayIdentify an area of ​​the vehicle 202 that may likely include a lamp or turn signal. As another example, the driver intent component 104 may execute an edge detection and / or other algorithm to identify a turn signal, headlight, tail lamp, or the like on the vehicle 202. The driver intent component 104 may define a boundary surrounding the turn signal, which may be processed to detect a state of the turn signal (e.g., whether the signal is flashing, off, or the like).

[0040] In one embodiment, the driver intent component 104 may determine a sub-region containing a driver 204 and one or more turn signals 214. For example, the driver intent component 104 may determine a single sub-region of the vehicle 202 that includes both the driver 204 and the turn signal 214. In one embodiment, the driver intent component 104 may select a sub-region of the image 200 that includes all or most of the vehicle 202. The sub-region containing the vehicle 202 may, for example, be processed to detect both the body language of the driver 204 and a state of the indicator 214.

[0041] Fig. 3 is a schematic plan view of a road 300 with a vehicle 302 traveling on the road. The vehicle 302 can control the system 100 according to Fig. 1. In one embodiment, one or more sensors, such as camera system 110, may collect perception data of the road 300 and the areas around vehicle 302. A field of view of camera system 110, LIDAR system 108, radar system 106, or other system may extend in any direction or in all directions around vehicle 302. Vehicle 302 or a driver intent component 104 of vehicle 302 may receive the perception data from the sensors and detect the presence of other vehicles, objects, surfaces, or the like within a field of view of vehicle 302. For example, vehicle 302 may detect and identify vehicle 304 as another vehicle.

[0042] In one embodiment, the vehicle 302 may identify a subsection of the perception data corresponding to a location where the driver is likely located, such as a window of the vehicle or an area of ​​the vehicle where a driver is likely sitting. The vehicle 302 may analyze the subsection of the perception data to infer a driver intent. For example, the vehicle 302 or a driver intent component 104 of the vehicle 302 may infer a driver intent to drive, stop, wait, or perform some other driving maneuver. In one embodiment, the vehicle 302 may identify a subsection of the perception data corresponding to a location where a turn signal is or is likely to be located, such as on or near a front or rear of the vehicle 304, near a bumper, near the headlights or taillights, or the like.The vehicle 302 may analyze the subsection of the perception data to determine a state of the turn signal. For example, the vehicle 302 or a driver intent component 104 of the vehicle 302 may determine that a turn signal for the vehicle 304 is off or on and may determine a direction (left or right) corresponding to the turn signal. The vehicle 302 may infer an intent of the driver of the vehicle 304 based on the state of the turn signal. Based on the inferred intent, the vehicle 302 may decelerate, accelerate, and / or turn to avoid a potential collision.

[0043] In addition to the perception data, the vehicle 302 may receive information from a stored map, a stored driving history, or from wireless signals. For example, an infrastructure transmitter 306 is shown near the road 300, which may provide specific positioning, environmental feature details, or other information to the vehicle 302. As further examples, the vehicle 302 may receive information from other vehicles, such as a vehicle 304, or from a wireless communications network, such as a cellular communications network. In one embodiment, the vehicle 302 may receive information about previous states or locations of the vehicle 304. For example, the vehicle 302 may receive an indication of the amount of time the vehicle 304 waited at the intersection from the vehicle 304 or the infrastructure transmitters 306.

[0044] Fig. 4 is a schematic block diagram illustrating a method 400 for determining the intent of a driver in another vehicle. Perception data, such as camera data, LIDAR data, radar data, and ultrasound data, is obtained at 402, and a driver intent component 104 identifies and locates the vehicle based on the perception data at 404. The driver intent component 104 may, for example, identify an area or a field of view or a region of an image corresponding to a vehicle. The driver intent component 104 finds a region of interest including the driver of the vehicle at 406. The region of interest may include a window, a region of the vehicle corresponding to a driver seat, or the like. The driver intent component 104 also finds a region of interest including a turn signal, such as a turn signal, at 408.an area near a front or rear bumper, a headlight, and / or a tail lamp. The driver intent component 104 estimates a head posture and gaze direction at 410 and performs gesture recognition at 412. The driver intent component 104 also determines a turn signal state at 414. For example, the driver intent component 104 may determine at 414 whether a turn signal is off or flashing.

[0045] The driver intent component 104 detects one or more vehicle movements at 416. For example, an acceleration event, a deceleration event, a stop event, a turn event, or any other movement of the vehicle may be detected at 416. The driver intent component 104 may also determine one or more previous vehicle states for a detected vehicle at 418 based on the V2V or V2X data at 420. The previous vehicle states may include, for example, a length of time the vehicle was in a location, one or more previous locations, accelerations, decelerations, or the like.Based on a head posture, a gaze direction, a detected gesture, a turn signal state, a detected vehicle movement, and / or previous vehicle states, the driver intent component 104 interprets 422 the intent of a driver and / or the driver intent component 104 predicts the future movement of the driver's vehicle.

[0046] Fig. 5 is a schematic block diagram illustrating the components of a driver intent component 104 according to one embodiment. The driver intent component 104 includes a perception data component 502, a boundary component 504, a body language component 506, a turn signal component 508, a previous state component 510, a vehicle motion component 512, an intersection component 514, a prediction component 516, and a maneuver component 518. Components 502-518 are provided for illustrative purposes only and may not all be included in all embodiments. Indeed, some embodiments may include only one or any combination of two or more of the components 502-518. Some of the components 502-518 may be located external to the driver intent component 104, such as within the automated driving / assistance system 102 or elsewhere.

[0047] The perception data component 502 is configured to receive sensor data from one or more sensor systems of the vehicle. For example, the perception data component 502 may receive data from the radar system 106, the LIDAR system 108, the camera system 110, the GPS 112, the ultrasound system 114, or the like. In one embodiment, the perception data may include the perception data for one or more areas proximate the vehicle. For example, the vehicle's sensors may provide a 360-degree view around the vehicle. In one embodiment, the camera system 110 captures an image of a vehicle. The vehicle may, for example, be proximal to a host vehicle of the driver intent component 104. In one embodiment, the camera system 110 captures an image of a proximal vehicle near an intersection.

[0048] The bounding component 504 is configured to identify a sub-region of the perception data that corresponds to a driver of a vehicle or an area near a driver of a vehicle. For example, the bounding component 504 may identify a sub-region of an image or other sensor data frame that likely contains a driver of the vehicle, if present. The sub-region may be large enough to account for the driver's movement, such as gestures. In one embodiment, the sub-region may correspond to an area where the body language algorithms can be applied without wasting processing power detecting gestures or drivers in areas where they are unlikely to be located.

[0049] In one embodiment, boundary component 504 is configured to locate one or more vehicles within the images or other perception data. For example, object detection algorithms may be used to identify detected objects or obstacles as vehicles. In one embodiment, boundary component 504 may identify a boundary of the vehicle and identify pixels or objects within that area as corresponding to the vehicle. Image processing algorithms for finding edges or boundaries may be used to locate the edges of the vehicle.

[0050] In one embodiment, the boundary component 504 may identify or define a subportion of the image corresponding to an area where a driver of a vehicle is located or likely to be located. In one embodiment, the boundary component 504 may look within a boundary of the vehicle to locate or define a boundary corresponding to a location of the driver. In one embodiment, the boundary component 504 is configured to identify the subportion of the image based on a location of one or more windows, a location of a driver's seat or steering wheel of a vehicle, and / or a location of a detected human in the vehicle. In one embodiment, the boundary component 504 may identify a boundary of one or more windows or estimate a location of a front seat on the driver's side of a vehicle.In one embodiment, the sub-area may correspond to or include the boundary of a window. For example, the sub-area may have a boundary that follows the boundary of one or more windows. In one embodiment, the sub-area may have a boundary that is slightly larger than a likely area where the driver is located to account for movement or error in defining the boundary.

[0051] In one embodiment, the boundary component 504 is configured to identify a sub-region of an image containing a turn signal of a vehicle. The sub-region of the image may include, for example, a turn signal lamp positioned at a front, a rear, or any other location of a vehicle. In one embodiment, the sub-region may include an area at or near a bumper and / or an area near a headlight or tail lamp of the vehicle. In one embodiment, the boundary component 504 may identify the sub-region of the image containing the turn signal using object recognition or edge detection image processing algorithms. The boundary component 504 may identify, for example, an edge or boundary of an indicator lamp, headlight, tail lamp, or the like.In one embodiment, the boundary component 504 may determine an area surrounding the turn signal or an area larger than the turn signal, so that the states of the turn signal can be accurately determined even if the boundary is not perfectly aligned with or centered around the turn signal.

[0052] The body language component 506 is configured to detect the body language of a driver. The body language component 506 may detect the body language of a vehicle by identifying the orientation of a driver's head and / or a gaze direction and / or a gesture. The head orientation may be an orientation of a person's head. The head orientation may, for example, describe a direction in which the driver's face is located. The head orientation may be described as one or more angles relative to a location of a host vehicle of the driver intent component 104. The angles may, for example, include an angle of the gaze direction relative to a line drawn between a center of a vehicle (or another zero-origin location used by an automated driving / assistance system 102) and the human driver. The gaze direction may include a direction in which a driver is looking.In some situations, the gaze direction may correspond to a head orientation, but the gaze direction may also be different if the driver's eyes are not looking straight ahead. The gaze direction can be described as one or more angular offsets from the head orientation or can be described as angular directions independent of the head orientation.

[0053] The gestures may include any other type of body language, including gestures, movements, or positions assumed by the driver's hands, arms, shoulders, neck, torso, face, head, or other parts of a driver's body. Example gestures may include: waving gestures to signal another driver, a pedestrian, or a bicyclist to proceed at an intersection, road junction, or lane merging; head movements indicating no or yes; halt or stop hand signals in which a hand is held up with the palm facing up toward another vehicle or person; shoulder shrugs; side-to-side head movements to check for crossing vehicles; or any other gesture.Additional gestures may be identified, including gestures detected through machine learning or analysis of sensor data. In one embodiment, the body language component 506 may detect one or more of these gestures or other gestures performed by a driver of a nearby vehicle.

[0054] In one embodiment, the body language component 506 may conserve processing power by, at least initially, only attempting to detect body language within a boundary of an image or other perceptual data frame determined by the boundary component 504. For example, only the pixels or the like within the boundary may be processed by the body language detection algorithms. Because gesture recognition or other body language recognition can be computationally intensive, this can result in significant time, processing, and energy savings for the driver intent component 104 or the automated driving / assistance system 102.

[0055] The turn signal component 508 is configured to determine a state of a turn signal of a proximal vehicle. For example, the turn signal component 508 may determine a state of one or more turn signals of a vehicle located at an intersection near a host vehicle. In one embodiment, the turn signal component 508 may process one or more subregions determined by the boundary component 504 to determine the state of the turn signal. For example, the turn signal component 508 may detect whether a turn signal is emitting light or flashing based on one or more images of the turn signal. In one embodiment, the turn signal component 508 may determine whether a left turn signal and / or a right turn signal are on or flashing.For example, the turn signal component 508 may determine whether only the left turn signal is flashing, only the right turn signal is flashing, both the left turn signal and the right turn signal are flashing, or neither the left turn signal nor the right turn signal is flashing.

[0056] The previous state component 510 is configured to determine one or more previous states of a vehicle, such as a proximal vehicle at an intersection. In one embodiment, the previous state component 510 may determine the previous states that are not or cannot be determined based on the perception data. For example, the previous state component 510 may determine the one or more previous states of a proximal vehicle based on wireless communications indicating the one or more previous states of the proximal vehicle. The wireless communications may include V2V or V2X communications.The wireless communications may indicate a previous location of the proximal vehicle, a previous speed of the proximal vehicle, a length of time the proximal vehicle was near the intersection, or any other information about the previous movements, locations, or conditions of the proximal vehicle.

[0057] The vehicle motion component 512 is configured to detect one or more movements of a proximal vehicle. For example, the vehicle motion component 512 may detect the movements of the proximal vehicle based on the perception data or other sensor data received by the perception data component 502. In one embodiment, the vehicle motion component 512 may determine one or more accelerations, decelerations, turns, or the like of a proximal vehicle. In one embodiment, the vehicle motion component 512 may detect the specific movements of a proximal vehicle, such as an acceleration quickly followed by a deceleration. For example, an acceleration quickly followed by a deceleration may indicate that a human driver is about to move or that the human driver is hesitant to proceed through an intersection.In one embodiment, the correlations between the movements and future actions by a human-driven vehicle may be determined by the prediction component 516.

[0058] The body language component 506, the turn signal component 508, and / or the vehicle motion component 512 may include models, neural networks, machine-learned parameters, or the like to detect body language, turn signal states, and vehicle motion. Guided or unguided machine learning algorithms, for example, may process perception data from real-world or virtual environments to learn the shapes, movements, or other image content corresponding to body language, turn signal states, or vehicle motion. The results of these machine learning algorithms may be incorporated into models or databases for use by the respective components to detect body language, turn signal states, or vehicle motion during vehicle travel.

[0059] The intersection component 514 is configured to determine that a host vehicle is near an intersection. For example, the intersection component 514 may process the sensor data to locate road intersections, signs, pavement markings or painted lines, stopped vehicles, or other aspects that may be correlated with an intersection. In one embodiment, the intersection component 514 may reference a map and the GPS location to determine that the host vehicle is at or approaching an intersection. Because it may be difficult for an autonomous vehicle to make driving decisions at an intersection, the intersection component 514 may specifically identify that the vehicle should operate according to an intersection scenario. For example, specific driving rules may apply to intersections or other driving scenarios.

[0060] The prediction component 516 may infer a driver's intent or predict the future movements of a nearby vehicle based on the body language component 506, the turn signal component 508, the previous states component 510, the vehicle motion component 512, and / or the intersection component 514. For example, the prediction component 516 may predict future movement and / or a timing for the future movement based on a driver's body language, a turn signal state, the previous states of a vehicle, and / or the detected vehicle movements.

[0061] In one embodiment, the prediction component 516 determines the driver's intent or predicts future movements based on the body language detected by the body language component 506. For example, the prediction component 516 may determine a timing and movement the driver intends to perform. Example movements include turning onto another road, waiting at an intersection, merging into traffic, changing lanes, exiting a road, entering a road, parking a vehicle, exiting a parking space, or the like.

[0062] In one embodiment, the prediction component 516 references or processes a database or model to determine a predicted movement or an intended movement of another vehicle. For example, the prediction component 516 may include or access a database or model that correlates body language with one or more future vehicle movements. In one embodiment, the database or model may correlate a head orientation, a gaze direction, and / or a gesture with one or more vehicle movements likely to occur following the head orientation, gaze direction, and / or gesture.

[0063] In one embodiment, the database or model may correlate future vehicle movements with a hand waving motion and / or a hand gesture including a palm facing the vehicle with the fingers pointing upward, and / or a driver gaze direction during a threshold period (such as when a driver is staring at another vehicle and waiting for the vehicle to move, or when a driver is staring straight ahead to move forward or continue moving forward, a sequence of hand gestures, and a sequence of rapid changes in gaze direction (e.g., checking for crossing traffic)). In one embodiment, the database or model may include a correlation between a gaze direction and a future direction of travel. For example, the model may indicate a correlation between a driver looking in a specific direction followed by vehicle movement in that specific direction.In one embodiment, the database or model correlates looking over a shoulder with a future vehicle movement involving a lane change or turn. In one embodiment, the database or model may include one or more machine-learned rules or correlations. For example, the database or model may include correlations and probabilities based on actual sensor data, head orientation, gaze directions, and gestures learned by a machine.

[0064] In one embodiment, the database or model further correlates a driver's intent or a future driving maneuver based on a current driving context. For example, some gestures may mean different things based on whether the nearby vehicle or host vehicle is stopped at an intersection, approaching an intersection, traveling down a road with one or more vehicles nearby, merging onto a road, exiting a road, entering a parking area or parking space, exiting a parking area or parking space, or the like. Consequently, the gestures and the current driving context can be used to accurately infer a driver's intent or predict a future driving maneuver.The prediction component 516 may provide the predicted driving maneuvers or driver intent to the driving maneuver component 518 or the automated driving / assistance system 102 for making decisions regarding the maneuvers or actions to be taken by the automated driving / assistance system 102 or a host vehicle.

[0065] In one embodiment, the prediction component 516 determines the driver's intent, or the prediction component 516 predicts future movements of the vehicle based on a state of a turn signal determined by the turn signal component 508. For example, the prediction component 516 may predict a timing and direction of travel of the vehicle to travel through the intersection. In one embodiment, the prediction component 516 references the database or model, or the prediction component 516 processes the database or model to determine the predicted movement or intended movement of another vehicle. For example, the prediction component 516 may include or access a database or model that correlates the turn signal states with one or more future vehicle movements.In one embodiment, the database or model may correlate a flashing light direction with the vehicle turning through the intersection in that direction. In one embodiment, the database or model may correlate a flashing of the hazard lights (e.g., both turn signals flashing) with a vehicle moving quickly through the intersection in an unpredictable direction without waiting according to normal protocols. Consequently, an earlier timing for the vehicle with the flashing hazard lights to move through the intersection may be predicted.

[0066] In one embodiment, the prediction component 516 determines the driver's intent, or the prediction component 516 predicts the future movements of the vehicle based on one or more previous states of the proximal vehicle determined by the previous state component 510. For example, the prediction component 516 may predict a timing and direction of travel of the vehicle to travel through the intersection. In one embodiment, the prediction component 516 references the database or model, or the prediction component 516 processes the database or model to determine the predicted movement or intended movement of another vehicle. For example, the prediction component 516 may include or access a database or model thatthat correlates the previous states of a vehicle with one or more future vehicle movements. In one embodiment, the database or model may correlate a longer wait time of a vehicle at the intersection with faster movement through the intersection. In one embodiment, a vehicle that has been near the intersection for beyond a threshold period may be determined to be parked or malfunctioning. Consequently, the prediction component 516 may determine that the vehicle that has waited longer than the threshold period may wait for an even longer period before proceeding through the intersection.

[0067] The prediction component 516 may also predict future movements of the proximal vehicle based on previous locations, accelerations, decelerations, or the like.

[0068] In one embodiment, the prediction component 516 determines the driver's intent, or the prediction component 516 predicts future movements based on the detected movements of the vehicle determined by the vehicle movement component 512. The prediction component 516 may, for example, predict a timing and direction of travel for the vehicle to travel through the intersection. In one embodiment, the prediction component 516 references the database or model, or the prediction component 516 processes the database or model to determine the predicted movement or intended movement of another vehicle. The prediction component 516 may, for example, include or access a database or model that correlates one or more detected movements with one or more future movements. The database may, for example,contain an acceleration, a speed, a deceleration or other movement information with a predicted further movement through an intersection.

[0069] In one embodiment, the prediction component 516 may determine a driver's intent or future movement of a vehicle based on a combination of the data from the body language component 506, the turn signal component 508, the previous states component 510, the vehicle motion component 512, and the intersection component 514. For example, the model or database may include machine learning values ​​or correlations based on the movement of one or more vehicles, the driver's body language, previous vehicle states, and turn signal information during previous intersection driving scenarios. Thus, the prediction component 516 may predict future movements based on a combination of body language, turn signal status, previous states, vehicle motion, and / or location at an intersection.

[0070] The driving maneuver component 518 is configured to select a driving maneuver for a host vehicle based on the predicted driver intent or the future driving maneuver of another vehicle. The driving maneuver component 518 may, for example, receive one or more predicted driving maneuvers for one or more nearby vehicles from the prediction component 516. The driving maneuver may determine a path to avoid a collision with the other vehicles if they perform the predicted driving maneuvers. The driving maneuver component 518 may, for example, determine whether to decelerate, accelerate, and / or turn a steering wheel of the host vehicle. In one embodiment, the driving maneuver component 518 may determine a timing for the driving maneuver. The driving maneuver component 518 may, for example,determine that a host vehicle should wait at an intersection for a period of time because another vehicle is likely to proceed through the intersection during that period. As another example, the driving maneuver component 518 may determine a timing for the host vehicle to proceed through the intersection based on the predicted future movement of the proximal vehicle.

[0071] In Fig. 6 is a schematic diagram of a flowchart of a method 600 for predicting a movement of a vehicle according to one embodiment. The method 600 may be implemented by an automated driving / assistance system or a driver intent component, such as the automated driving / assistance system 102 of Fig. 1 or the driver intention component 104 according to the Fig. 1 or Fig. 5, are executed.

[0072] Method 600 begins, with an intersection component 514 determining, at 602, that a host vehicle is near an intersection. A boundary component 504 identifies, at 604, a subportion of the image that includes a turn signal on the proximal vehicle. A prediction component 516 predicts, at 606, the future movement of the proximal vehicle through the intersection based on a state of the turn signal.

[0073] In Fig. Figure 7 is a schematic diagram of a flowchart of a method 700 for predicting a movement of a vehicle according to one embodiment. The method 700 may be performed by an automated driving / assistance system or a driver intent component, such as the automated driving / assistance system 102 of Fig. 1 or the driver intention component 104 according to the Fig. 1 or Fig. 5, are executed.

[0074] The method 700 begins with a perception data component 502 receiving an image of a proximal vehicle near an intersection at 702. The proximal vehicle may, for example, be near a host vehicle of a driver intent component 104. A bounding component 504 identifies, at 704, a subportion of the image that includes a turn signal on the proximal vehicle. The bounding component 504 identifies, at 706, a subportion of the image that includes a driver of the proximal vehicle. The prediction component 516 predicts, at 708, a future movement of the proximal vehicle through the intersection based on a state of the turn signal and the driver's body language. A maneuver component 518 determines, at 710, a time at which the host vehicle will proceed through the intersection based on the predicted future movement of the proximal vehicle. Examples

[0075] The following examples relate to further embodiments.

[0076] Example 1 is a system that includes an intersection component, a camera system, a boundary component, and a prediction component. The intersection component is configured to determine that a host vehicle is near an intersection. The camera system is configured to capture an image of a proximal vehicle. The boundary component is configured to identify a subportion of the image that includes a turn signal on the proximal vehicle. The prediction component is configured to predict the future movement of the proximal vehicle through the intersection based on a state of the turn signal.

[0077] In Example 2, the system of Example 1 further includes a turn signal component configured to process the image data in the sub-portion of the image to determine the state of the turn signal.

[0078] In Example 3, the system of any of Examples 1-2 further includes a previous state component configured to determine one or more previous states of the proximal vehicle based on the wireless communications indicating one or more previous states of the proximal vehicle. The prediction component is configured to predict the future movements of the vehicle based on the one or more previous states of the proximal vehicle.

[0079] In Example 4, the wireless communication in Example 3 includes one or more of a V2V communication and a V2X communication.

[0080] In Example 5, the one or more preceding states in any of Examples 3-4 indicate a duration of time the proximal vehicle was in the vicinity of the intersection.

[0081] In Example 6, the system of any of Examples 1-5 further includes a vehicle motion component configured to determine one or more vehicle motions of the proximal vehicle. The prediction component is configured to predict the future motions of the proximal vehicle based on the one or more vehicle motions of the proximal vehicle.

[0082] In Example 7, the boundary component in any of Examples 1-6 is further configured to identify a subportion of the image corresponding to a driver's location. The system further includes a body language component configured to detect a driver's body language by identifying at least one of a driver's head orientation, a gaze direction, and a gesture. The prediction component is configured to predict future movements of the vehicle based on the driver's body language.

[0083] Example 8 is a computer-implemented method. The method includes receiving an image of a proximal vehicle proximate an intersection and storing the image in computer memory. The method includes identifying, using one or more processors, a subportion of the image that includes a turn signal on the proximal vehicle and identifying, using the one or more processors, a subportion of the image that includes a driver of the proximal vehicle. The method includes predicting, using the one or more processors, future movement of the proximal vehicle through the intersection based on a state of the turn signal and body language of the driver.The method includes determining, using the one or more processors, a timing for the host vehicle to proceed through the intersection based on the predicted future movement of the proximal vehicle. The method includes causing the host vehicle to execute a driving maneuver based on the determined timing for proceeding through the intersection.

[0084] In Example 9, the method of Example 8 further includes determining that a host vehicle is near an intersection.

[0085] In Example 10, the method of any one of Examples 8-9 further includes processing image data in the subportion of the image including a turn signal to determine the state of the turn signal, and processing image data in the subportion of the image including a driver of the proximal vehicle to determine the body language of the driver.

[0086] In Example 11, determining the body language of the driver in Example 10 includes identifying the orientation of a driver's head and / or a gaze direction and / or a gesture.

[0087] In Example 12, the method of any of Examples 8-11 further includes determining one or more previous states of the proximal vehicle based on a wireless communication. Predicting the future movements of the proximal vehicle includes predicting based on the one or more previous states of the proximal vehicle.

[0088] In Example 13, the wireless communication of Example 12 includes one or more of a V2V communication and a V2X communication.

[0089] In Example 14, the method of any of Examples 8-13 further includes detecting one or more vehicle movements of the proximal vehicle. Predicting the future movements of the proximal vehicle includes predicting based on the one or more vehicle movements of the proximal vehicle.

[0090] In Example 15, the method of any of Examples 8-14 further includes accessing or processing a model or database that correlates the state of the turn signal and the body language of the driver with the predicted future movement.

[0091] In Example 16, the model or database of Example 15 includes the machine learning values ​​or correlations based on the movement of one or more vehicles, driver body language, and turn signal information during previous intersection driving scenarios.

[0092] Example 17 is a computer-readable storage medium storing instructions that, when executed by one or more processors, cause the processors to determine that a host vehicle is near an intersection. The instructions cause the processor to capture multiple images of a proximal vehicle. The instructions cause the processor to identify a subregion in each of the multiple images that includes a turn signal on the proximal vehicle. The instructions further cause the processor to predict future movement of the proximal vehicle through the intersection based on a state of the turn signal.

[0093] In Example 18, the instructions of Example 17 further cause the processor to process the image data in the subsection of the plurality of images to determine the state of the turn signal.

[0094] In Example 19, the instructions of any of Examples 17-18 further cause the processor to: determine one or more previous states of the proximal vehicle based on wireless communications indicating the one or more previous states of the proximal vehicle; and predict the future movements of the vehicle based on the one or more previous states of the proximal vehicle.

[0095] In Example 20, the instructions of any of Examples 17-19 further cause the processor to: identify a subportion of the image corresponding to a location of a driver; detect body language of a driver by identifying at least one of the orientation of a driver's head, a gaze direction, and a gesture; and predict future movements of the vehicle based on the body language of the driver.

[0096] It should be noted that the sensor embodiments discussed above may include computer hardware, software, or firmware, or any combination thereof, to perform at least a portion of their functions. For example, a sensor may include computer code configured to execute on one or more processors and may include hardware logic / electrical circuitry controlled by the computer code. These example devices are provided herein for illustrative purposes and are not intended to be limiting. The embodiments of the present disclosure may be implemented in other types of devices, as would be known to those skilled in the relevant art(s).

[0097] Embodiments of the disclosure have been directed to computer program products including such logic (e.g., in the form of software) stored in any computer-usable medium. Such software, when executed in one or more data processing devices, causes a device to operate as described herein.

[0098] While various embodiments of the present disclosure have been described above, it should be understood that they have been presented by way of example and not limitation. It will be apparent to those skilled in the relevant art that various changes in form and details may be made therein without departing from the spirit and scope of the disclosure. Accordingly, the breadth and scope of the present disclosure should not be limited by any of the exemplary embodiments described above, but should be defined only in accordance with the following claims and their equivalents. The foregoing description has been presented for purposes of illustration and description. It is not intended to be exhaustive or to limit the disclosure to the precise form disclosed.In light of the above teachings, many modifications and variations are possible. Furthermore, it should be noted that any or all of the above-mentioned alternative implementations may be used in any combination desired to form additional hybrid implementations of the disclosure.

[0099] Furthermore, although specific implementations of the disclosure have been described and illustrated, the disclosure is not limited to the specific forms or arrangements of parts so described and illustrated. The scope of the disclosure is intended to be defined by the claims appended hereto and any future claims filed here and in other applications, and their equivalents.

Claims

[1] System comprising: an intersection component configured to determine that a host vehicle is near an intersection; a camera system configured to capture an image of a proximal vehicle; a boundary component configured to identify a subportion of the image that includes a turn signal on the proximal vehicle; and a prediction component configured to predict future movement of the proximal vehicle through the intersection based on a state of the turn signal. [2] The system of claim 1, further comprising a turn signal component configured to process the image data in the subportion of the image to determine the state of the turn signal. [3] The system of claim 1, further comprising a previous state component configured to determine one or more previous states of the proximal vehicle based on the wireless communications indicating one or more previous states of the proximal vehicle, wherein the prediction component is configured to predict the future movements of the proximal vehicle based on the one or more previous states of the proximal vehicle. [4] The system of claim 3, wherein the wireless communication comprises one or more of vehicle-to-vehicle (V2V) communication and vehicle-to-infrastructure (V2X) communication. [5] The system of claim 3, wherein the one or more preceding states indicate a period of time during which the proximal vehicle has been in the vicinity of the intersection. [6] The system of claim 1, further comprising a vehicle motion component configured to determine one or more vehicle motions of the proximal vehicle, wherein the prediction component is configured to predict the future motions of the proximal vehicle based on the one or more vehicle motions of the proximal vehicle. [7] The system of claim 1, wherein the boundary component is further configured to identify a subportion of the image corresponding to a location of a driver, the system further comprising a body language component configured to detect the body language of a driver by identifying at least one of the orientation of a driver's head, a gaze direction, and a gesture, the prediction component being configured to predict the future movements of the proximal vehicle based on the body language of the driver. [8] Computer-implemented method comprising: Receiving an image of a proximal vehicle near an intersection and storing the image in a computer memory; identifying, using one or more processors, a subportion of the image that includes a turn signal on the proximal vehicle; identifying, using the one or more processors, a subportion of the image that includes a driver of the proximal vehicle; predicting, using the one or more processors, future movement of the proximal vehicle through the intersection based on a state of the turn signal and the body language of the driver; Determining, using the one or more processors, a timing for a host vehicle to proceed through the intersection based on the predicted future movement of the proximal vehicle; and Causing the host vehicle to perform a driving maneuver based on the determined time to proceed through the intersection. [9] The method of claim 8, further comprising determining that the host vehicle is near an intersection. [10] The method of claim 8, further comprising processing image data in the subportion of the image containing the turn signal to determine the state of the turn signal, and processing image data in the subportion of the image containing a driver of the proximal vehicle to determine the body language of the driver. [11] The method of claim 10, wherein determining the driver's body language comprises identifying at least one of the orientation of a driver's head and a gaze direction and a gesture. [12] The method of claim 8, further comprising determining one or more previous states of the proximal vehicle based on a wireless communication, wherein predicting the future movements of the proximal vehicle comprises predicting based on the one or more previous states of the proximal vehicle. [13] The method of claim 12, wherein the wireless communication comprises one or more of vehicle-to-vehicle (V2V) communication and vehicle-to-infrastructure (V2X) communication. [14] The method of claim 8, further comprising detecting one or more vehicle movements of the proximal vehicle, wherein predicting the future movements of the proximal vehicle comprises predicting based on the one or more vehicle movements of the proximal vehicle. [15] The method of claim 8, further comprising accessing or processing a model or database that correlates the state of the turn signal and the driver's body language with the predicted future movement. [16] The method of claim 15, wherein the model or database comprises the machine learning values ​​or correlations based on the movement of one or more vehicles, the driver's body language, and the turn signal information during previous intersection driving scenarios. [17] Computer-readable storage medium that stores instructions that, when executed by one or more processors, cause the processors to: to determine that a host vehicle is near an intersection; to capture multiple images of a proximal vehicle; identify a sub-region in each of the plurality of images that includes a turn signal on the proximal vehicle; and predict the future movement of the proximal vehicle through the intersection based on a turn signal state. [18] The computer-readable storage medium of claim 17, wherein the instructions further cause the processor to process image data in the sub-portion of the plurality of images to determine the state of the turn signal. [19] The computer-readable storage medium of claim 17, wherein the instructions further cause the processor to: determine one or more previous states of the proximal vehicle based on wireless communications indicating the one or more previous states of the proximal vehicle; and predict the future movements of the proximal vehicle based on the one or more previous states of the proximal vehicle. [20] The computer-readable storage medium of claim 17, wherein the instructions further cause the processor to: identify a subsection of the image corresponding to a driver location; detect a driver's body language by identifying the orientation of a driver's head and / or a gaze direction and / or a gesture; and predict the future movements of the proximal vehicle based on the driver's body language.

Images