Turning path visualization to improve spatial and situational awareness during turning maneuvers

Abstract

Methods encompassing the control of a vehicle: Receiving output from one or more sensors; Identifying one or more obstacles according to the outputs of one or more sensors; Determining one or more paths of one or more obstacles, at least based on an object tracking technique; (a) Determining a turning path according to the locations of the one or more obstacles and the one or more paths of the one or more obstacles; (b) Presenting a visual representation of the turning path on a display device coupled to the control system for execution by a driver of the vehicle. Performing (a) and (b) in response to receiving an initial turn indication from the driver of the vehicle, wherein the initial turn indication comprises at least one of the following: a change in the steering wheel angle; Activating a direction indicator; and Capturing a turn in navigation data for a route selected by the driver, wherein (b) further includes: capturing a current steering angle of the vehicle; Determining a desired steering angle according to the turning path; and Displaying a second indicator of the difference between the current steering angle and the desired steering angle; and Selecting a color for the second indicator according to the magnitude of the difference between the current steering angle and the desired steering angle, wherein the color for the second indicator is determined based on a series of predetermined limits for the magnitude, and wherein the series of predetermined limits for the magnitude varies at least partially depending on the vehicle speed.

Description

GENERAL STATE OF THE TECHNOLOGY - AREA OF INVENTION

[0001] This invention relates to the provision of sensor-based driving aids. GENERAL STATE OF THE ART

[0002] Vehicles have become much safer over the decades. In addition to safety equipment that protects passengers in the event of an accident, many vehicles offer sensors that can alert the driver to obstacles. Publications DE 10 2014 223 275 A1, DE 10 2016 212 751 A1, DE 10 2016 120 508 A1, US 2016 / 0 325 753 A1 and DE 103 24 812 A1 demonstrate the corresponding state of the art. However, simply providing a warning can be distracting and may not be sufficient to trigger a timely reaction from the driver.

[0003] Accordingly, an improved approach to using sensor data is needed to improve passenger safety.

[0004] This problem is solved by the subject matter of the independent claims. Preferred embodiments are the subject matter of the dependent claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0005] To facilitate understanding of the advantages of the invention, a more detailed description of the invention briefly described above is provided by reference to specific embodiments illustrated in the accompanying drawings. Under the understanding that these drawings represent only typical embodiments of the invention and are therefore not to be considered as limiting its scope, the invention is described and explained with additional specificity and detail by means of the accompanying drawings, wherein: Fig. 1 a schematic block diagram of components that implement a system according to an embodiment of the present invention; Fig. 2 a schematic block diagram of an exemplary computing device suitable for implementing methods according to embodiments of the invention; Fig. 3 a process flow diagram of a method for providing turn guidance and visualization according to an embodiment of the present invention; Fig. Figure 4 illustrates an exemplary display of a turn visualization and guidance system according to an embodiment of the present invention; and Fig. 5 is another exemplary display of a turn visualization and guidance according to an embodiment of the present invention. DETAILED DESCRIPTION

[0006] With reference to Fig. 1. The methods disclosed herein can be carried out using the illustrated system 100. As discussed in more detail herein, a controller 102 can be programmed to carry out the methods disclosed herein and can also perform other functions performed by a vehicle control unit (e.g., electronic control unit (ECU)) or in-vehicle infotainment (IVI) system. The controller 102 can be housed in a vehicle that has all the structures and features of any prior art vehicle known in the field, including wheels, a drivetrain coupled to the wheels, an engine coupled to the drivetrain, a steering system, a braking system, and other known systems that are known in the field to be incorporated in a vehicle.

[0007] The controller 102 can receive one or more outputs from one or more data sources 104. For example, one or more cameras 106a can be mounted on the vehicle and output image streams to the controller 102. The data sources 104 can include sensors such as an ultrasonic sensor 106b, a radar (radio detection and ranging) sensor 106c, a lidar (light detection and ranging) sensor 106d, a sonar (sound navigation and ranging) sensor 106e, and the like.

[0008] Other data sources 104 may include a GPS (Global Positioning System) receiver 106f and a vehicle-to-vehicle (Car2Car) transmitter / receiver 106g. The Car2Car transmitter / receiver can communicate with other vehicles according to a wireless protocol, such as a DSRC (Dedicated Short Range Communication) protocol.

[0009] The controller 102 can execute a guidance module 108, which receives the outputs from the data sources 104. The guidance module 108 can include an obstacle identification module 110a, a road evaluation module 110b, a path generator 110c, and a visualization module 110d.

[0010] The obstacle identification module 110a analyzes the outputs of data sources 104 and identifies potential obstacles, including people, animals, vehicles, buildings, curbs, and other objects and structures. In particular, the obstacle identification module 110a can identify vehicle images in the sensor outputs.

[0011] The road evaluation module 110b evaluates available road surfaces and the boundaries or subdivisions of a road surface, such as curbs and lane markings. The road evaluation module 110b can also evaluate map data 112 relating to a GPS location from the GPS receiver 106f to identify some or all of this information. For example, the number of lanes, whether a particular lane is a turning lane, the width of a road, the width of the lanes, the locations of intersections, and similar information may be recorded in map data 112, which may relate to the current position of the vehicle as determined by the GPS receiver 106f.

[0012] It should be noted that lane boundaries and the locations of curbs or other obstacles can also be determined from sensor data, such as the outputs of some or all of sensors 106a-106e. For example, lane boundaries and / or curb locations can be compared with map data to confirm the accuracy of the sensors or to determine the relative location of the vehicle more accurately.

[0013] The path generator 110c can use the obstacle locations and road data from modules 110a and 110b to determine a path that corresponds to the driver's intention. The approach disclosed herein is particularly helpful for drivers performing turning maneuvers, such as U-turns, turns in heavy traffic, turns in a busy parking lot, turns at blind or narrow intersections, or under other circumstances with poor visibility. Accordingly, the driver's intention may be to turn. The path generator 110c can use vehicle properties 114, which describe the vehicle, to determine a path that corresponds to the driver's intention and also avoids obstacles and remains within an intended area of ​​a road, as detected by the road evaluation module 110b.The vehicle characteristics 114 can include static properties such as width, length, wheelbase, turning radius, and turning dynamics (body roll, tire grip, etc.). The vehicle characteristics 114 can also include variable properties such as the current steering angle, current speed, the curvature of the curve as specified by the driver's intention, or other vehicle characteristics.

[0014] The path generator 110c can identify a path according to any prior art approach. In particular, the approach described herein provides guidance for a human driver. However, the path generated by the path generator 110c can be a path selected according to any known autonomous navigation technique. In particular, a control algorithm, such as that used in an autonomous vehicle, is programmed to select and execute a path for a given set of inputs, such as the obstacles and road data discussed above. Accordingly, a path can be generated according to such an algorithm for execution by a human driver.

[0015] The visualization module 110d can present the route on a display device 116, such as a heads-up display (HUD) in front of the driver, for example by projecting information onto the vehicle's windshield. The display device 116 can also be a screen of an in-vehicle infotainment system (IVI) or part of a digital instrument cluster. The display device 116 can also provide other visual warnings and information to assist a driver in performing a turning maneuver.

[0016] Fig. Figure 2 is a block diagram illustrating an exemplary computing device 200. The computing device 200 can be used to perform various procedures, such as those discussed in this document. The controller 102 can have some or all of the attributes of the computing device 200.

[0017] The computing device 200 includes one or more processors 202, one or more storage devices 204, one or more interfaces 206, one or more mass storage devices 208, one or more input / output (I / O) devices 210, and a display device 230, all connected to a bus 212. The processor(s) 202 includes one or more processors or controllers that execute instructions stored in the storage device(s) 204 and / or the mass storage device(s) 208. The processor(s) 202 may also include various types of computer-readable media, such as a buffer memory.

[0018] The storage device(s) 204 includes / include various computer-readable media, such as volatile memory (e.g., random access memory (RAM) 214) and / or non-volatile memory (e.g., read-only memory (ROM) 216). The storage device(s) 204 may also include a rewritable ROM, such as flash memory.

[0019] The mass storage device(s) 208 includes / include various computer-readable media, such as magnetic tapes, magnetic disks, optical disks, solid-state storage (e.g., flash memory), and so on. As in Fig. As shown in Figure 2, a special mass storage device is a hard disk drive 224. Various drives may also be included in the mass storage device(s) 208 to enable reading from and / or writing to the various computer-readable media. The mass storage device(s) 208 includes removable media 226 and / or non-removable media.

[0020] The I / O device(s) 210 includes various devices that enable data and / or other information to be entered into or retrieved from the computing device 200. An example of an I / O device 210 includes cursor control devices, keyboards, keypads, microphones, monitors or other display devices, loudspeakers, printers, network interface cards, modems, lenses, CCDs or other image capture devices, and the like.

[0021] The display device 230 includes any type of device capable of displaying information to one or more users of the computing device 200. Examples of a display device 230 include a monitor, a display terminal, a video projection device, and the like.

[0022] The interface(s) 206 includes / include various interfaces that enable the computing device 200 to interact with other systems, devices, or computing environments. An example of an interface 206 includes / includes any number of different network interfaces 220, such as interfaces to local area networks (LANs), wide area networks (WANs), wireless networks, and the Internet. Another interface includes / includes a user interface 218 and a peripheral device interface 222. The interface 206 may also include one or more peripheral interfaces, such as interfaces for printers, pointing devices (mice, trackpads, etc.), keyboards, and the like.

[0023] Bus 212 enables the processor(s) 202, the storage device(s) 204, the interface(s) 206, the mass storage device(s) 208, the I / O device(s) 210, and the display device 230 to communicate with each other and with other devices or components connected to bus 212. Bus 212 represents one or more of several types of bus structures, such as a system bus, PCI bus, IEEE 1394 bus, USB bus, and so on.

[0024] For illustrative purposes, programs and other executable program components are shown in this document as individual blocks, although it is understood that such programs and components reside at different times in different memory components of the computing device 200 and are executed by the processor(s) 202. Alternatively, the systems and procedures described in this document may be implemented as hardware or a combination of hardware, software, and / or firmware. For example, one or more application-specific integrated circuits (ASICs) may be programmed to execute one or more of the systems and procedures described in this document.

[0025] With reference to Fig. 3 The illustrated procedure 300 can be carried out by the guidance module 108 of the control unit 102 to provide support to a driver of the vehicle.

[0026] The procedure 300 may further include the detection 302 of obstacles according to some or all of the data sources 104. For example, obstacles and location can thus be detected using outputs from the camera 106a, radar 106c, lidar 106d, or in map data based on the vehicle's position according to the GPS receiver 106f. In particular, the locations of curbs, lane markings, and road signs can be obtained from the map data based on the vehicle's current position.

[0027] The procedure 300 can further include receiving 304 messages from other vehicles using a Car2Car communication protocol. Such messages can contain data indicating the current location, direction, speed, steering angle, accelerator pedal inputs, turn signal status, and current gear (e.g., reverse or forward) of another vehicle. Messages can also contain data describing intended future actions and their location (e.g., locations and directions of turns or other actions according to road guidance data currently presented to the other vehicle), as well as locations of obstacles detected by the other vehicle's sensors, or other information. It should be noted that the vehicle in which the control unit 102 is located can also transmit such messages to other vehicles using the Car2Car communication protocol.

[0028] Procedure 300 may further include determining obstacle paths 306, which are communicated in step 302 or as communicated via Car2Car messages in 304. For example, using the locations of detected obstacles over time, the speed of obstacles relative to the vehicle can be determined. Likewise, a path can be derived according to the earlier position determinations, such as using a Kalman filter or another object tracking technique. The way in which obstacles are detected, tracked, and corresponding paths are predicted can correspond to any prior art approach, such as a prior art approach for autonomous vehicles.

[0029] Procedure 100 can include receiving driver control inputs 308. Driver control inputs can include changes to the steering wheel angle, accelerator pedal position, brake pedal position, or other driver input. Driver control inputs can also include activating a turn signal. Furthermore, driver control inputs can include entering a destination with an instruction to the controller 102 or the IVI system to navigate to that location. Accordingly, a route generated based on driver input can include instructions to turn at specific locations and to continue along certain segments of a road.

[0030] It should be noted that steps 302-308 are all processes that can be performed during a journey and can be carried out simultaneously by the controller 102 while the vehicle is in operation. Accordingly, the sequence of steps 302-308 is as follows: Fig. 3 not restrictive. Likewise, steps 302-308 may be continued during the execution of the remaining steps of procedure 300.

[0031] Method 300 may include evaluating 310 whether the driver inputs indicate an intention to turn. For example, an intention to turn may be detected in response to the activation of a turn signal indicator, an impending turn (e.g., within a threshold distance or time) according to route guidance data, a change in the steering wheel angle toward a turning lane (e.g., left-turn lane or right-turn lane), or any other indication of an intention to turn. In some embodiments, only turns of a certain severity invoke turn assist according to the methods described below, such as only U-turns or only turns requiring a steering wheel rotation of at least 180 degrees.

[0032] In response to the detection of the turning intention, the procedure may involve calculating a turning path in the direction and at the location specified by the intention. For example, if the turn signal is illuminated, an upcoming turn can be identified in the map data according to the vehicle's current location and the direction (right or left) indicated by the turn signal. For example, if the right turn signal is illuminated, the location of the next opportunity to turn right can be identified according to the map data and the vehicle's current position and speed; that is, the next right turn the vehicle could potentially make, based on its current speed and the time required to decelerate safely to a safe turning speed.

[0033] If the intention to turn is based on route guidance data, the location of the turn, as specified by the route guidance data, can be determined.

[0034] In any case, step 312 may involve calculating a path that traverses the curve within the limits of the vehicle's dynamic boundaries, avoids obstacles (including encountering an obstacle's path simultaneously), and other factors. In some embodiments, a lane into which the vehicle is to turn may be selected. Likewise, a path to another lane after turning may be determined. The turning path may involve a multi-step turn (e.g., a three-point turn) to align the vehicle in a desired direction. The path calculated in step 312 may be a U-turn or, if the U-turn becomes unsafe, a maneuver to proceed to another turning point. Calculating a turning path may take into account some or all of the vehicle characteristics 114, in particular the vehicle's wheelbase size and turning radius.

[0035] As noted above, the calculation 312 of the turning path can be carried out using any approach to operating an autonomous vehicle, except that the turning is not actually performed autonomously by the controller 102.

[0036] Procedure 300 may further include assessing 314 whether a turning path exists that can be safely executed, that is, one that does not present a high probability of collision with an obstacle or exceeding the vehicle's operating limits. If no safe turning path exists, procedure 300 may include generating 316 a warning to the driver, such as an audible tone, a visual warning, or an intervention, such as autonomous application of the brakes.

[0037] If it is determined 314 that a safe turning path exists, the procedure 318 may involve calculating 318 a desired steering angle so that the vehicle follows the turning path. The desired steering angle may involve detecting a desired orientation of the wheels to follow the turning path, as according to any prior art approach for autonomously driving a vehicle along a path. The steering angle, which corresponds to the wheel angle, may then be determined from the configuration of the steering system, i.e., a known relationship between steering wheel angle and steering angle. The procedure 300 may further involve detecting 320 the current steering angle of the vehicle, as using a steering angle sensor according to any prior art approach.

[0038] Procedure 300 may further include displaying 322 a representation of the turning path and displaying a steering instruction 324 instructing the user to move the steering wheel from the current angle to the desired angle. Examples of how steps 322 and 324 may be performed are given below with respect to the Fig. 4 and Fig. 5 shown.

[0039] It should be noted that the steps of procedure 300 can be repeated during a turn. For example, steps 312-324 can be repeated as long as a turning intention is detected, i.e., until the vehicle has completed the turn, missed the opportunity to turn, the turn indicator has been deactivated, or certain other criteria are met. Accordingly, the path 312 can be recalculated and the displays of steps 322 and 324 can be updated while the vehicle is in motion.

[0040] It should be noted that steps 312-324 depend on a determination (310) of turn intent. Accordingly, if no turn intent is detected, turn assist and other information may be suppressed to avoid distracting the driver and overloading a HUD, IVI display, or other third-party display device.

[0041] Fig. Figure 4 illustrates an exemplary display 400 that can be generated according to method 300. The display 400 can be presented on the display device 116, which, as noted, can be a HUD, IVI display, or other display device.

[0042] In the illustrated example, the turning path calculated in step 312 is displayed as lines 402, showing one of (a) the path of the vehicle's front tires and (b) the path followed by the sides of the vehicle at a given point. The way in which the lines 402 are mapped based on the turning path from step 312 can be done by relating points in an image to real-world coordinates, such as GPS coordinates, and overlaying the turning path onto the image based on this relationship. Specifically, the location of a point on the path can be added to the point in the image that corresponds to that location. The image can be an actual output from a forward-facing camera 106a of the vehicle. Alternatively, the image can be an output from a 360-degree camera system 106a, such as a portion of such an image located in front of the vehicle.In yet another embodiment, the turning path 402 and other elements of the display 400 are shown on a HUD, so that they can be viewed together with a real view through the windshield of the vehicle.

[0043] In some embodiments, sections of the road surface in an image from camera 106a can be distinguished from obstacles, curbs, sidewalks, other vehicles, etc. The sections of path 402 that traverse currently unoccupied areas of the road surface can be highlighted or otherwise distinguished from other sections of path 402. As noted above, path 402 can be updated as the vehicle moves and the relative location of obstacles changes. In response to these changes, the highlighted section of path 402 can also be modified to include only the sections in currently unoccupied areas of the road surface.

[0044] Display 400 can also include display 404 of a desired steering angle to traverse the calculated turning path. Display 400 can further include display 406 of the vehicle's current steering angle. A range between indicators 404 and 406 can be highlighted, for example, by a segment with a color indicating the magnitude of the difference. For instance, a red segment can indicate a large difference, yellow an intermediate difference, and green an acceptable difference. Values ​​defining the limits for large, intermediate, and acceptable differences can be predefined by a manufacturer and can be speed-dependent, increasing as the vehicle's speed decreases.

[0045] Other ways to display a turn instruction can include text, such as "Turn steering wheel 15 degrees". In another example, a line can be presented with a first marker indicating the current steering angle and a second marker indicating the desired steering angle.

[0046] Various other pieces of information can be superimposed on the image from camera 106a. For example, images of vehicle 408 may display a notification 410 containing information received from that vehicle 408 via a Car2Car message, such as the vehicle's current speed and intended direction or turn. For instance, a message might include GPS coordinates for vehicle 408, which are then added to the portion of the image corresponding to that location. It should be noted that many vehicles are capable of sending Car2Car messages. Accordingly, the controller can select which messages to use for displaying notifications or warnings. For example, those within a certain proximity threshold of the vehicle's path and at a maximum distance in front of it.

[0047] In some embodiments, warning messages 412 may be superimposed on images of potential obstacles, such as pedestrians 414, animals, or other hazards. For example, the pedestrian 414 standing at the corner may possibly step into the turning lane 402. Accordingly, the superimposed warning message 412 may instruct the driver to pay special attention to the pedestrian 414 as he crosses the turning lane. The warning messages 412 may also be added to potential obstacles that are not visible to the operator, such as obstacles that can be detected by radar but are otherwise not visible.

[0048] Fig. Figure 5 shows another example display 500. In this example, the turning path 402 is shown from a top-down perspective. The turning path 402 can be superimposed on an image of the vehicle's surroundings from a top-down perspective. The image can be obtained from various sources, such as a satellite image of an area around the vehicle based on the vehicle's GPS coordinates, a portion of a map depicting the region around the vehicle, an output from a 360-degree camera system, or a composite image from cameras positioned around the vehicle and combined to form a top-down view of the area surrounding the vehicle.

[0049] Representations of vehicles 502a-502c can be added to display 500 at their locations relative to the turning path 402. Specifically, the locations of vehicles 502a-502c on display 500 can correspond to the actual locations of the vehicles, as detected by the vehicle or as reported by a Car2Car message. Specifically, vehicles 502b-502c may not be visible to the driver or their sensors due to a barrier 504. Accordingly, the display can add representations of vehicles 502b-502c at the locations specified by the messages. Regarding the example from Fig. 4. Messages may contain information regarding the speed and turning procedure of the message source. Accordingly, notifications 506a-506c may be displayed on the representations of the corresponding vehicles 502a-502c. In some cases, a vehicle 502c is in reverse but is not currently moving. Accordingly, a notification from vehicle 502c may communicate this fact, and display 500 may show a representation 506c of this fact (“R”) on the vehicle 502c display. If a path or potential path of a vehicle 502b, 502c has the potential to interfere with the turning path 402, display 500 may also include warnings 508b, 508c highlighting the potential interference.

[0050] Regarding display 400, warnings and notifications for Car2Car messages can be selected based on their relevance to the vehicle. For example, in the example from Fig. 5 considers the change in the state of vehicle 502c to reverse gear to be relevant insofar as it now has the potential to cut the turning path 402.

[0051] In some embodiments, a vehicle, e.g., vehicle 502a, can detect an obstacle 510 that is not detectable by the vehicle and report the location of the obstacle 510 to the vehicle using a Car2Car message. Accordingly, the display 500 can include a representation of the obstacle 510. The notification can further include an annotation 512, such as speed or other information regarding the trajectory of the obstacle 510, as detected by vehicle 502a or by the obstacle itself if the obstacle 510 is a Car2Car-enabled vehicle.

[0052] Display 500 may also include a representation of the vehicle 514, with the control unit 102 visible in the image, to enable the driver to understand the relative location of other features in display 500. Although not shown, display 500 may also include one or more indicators 404-406 that provide a steering angle instruction in the same manner as for display 400.

[0053] The following are examples of situations in which the display implemented according to procedure 300 from 500 can be helpful: • The vehicle is following another vehicle in traffic, and suddenly the second vehicle needs to turn without prior signaling (no center lane or protected left turn). A Car2Car message informs the vehicle of the turn in advance, as the turn instructions are presented in the second vehicle. Accordingly, a notification (see, for example, notifications 410, 506a-506c) can be displayed to the driver of the vehicle before the unexpected turn. • The vehicle is approaching a second vehicle in oncoming traffic that has stopped in the middle of the road. Will it turn? Should the vehicle slow down or speed up? A Car2Car message from the second vehicle could provide the vehicle with a notification about obstacles detected by the second vehicle's sensors. The driver of the vehicle can then conveniently pass the second vehicle or, based on this knowledge of the reported obstacles, make a turn. • The vehicle is in a busy parking lot and is looking for a parking space. The driver switches to a top-down view from Fig.5, i.e., to a long-range view. A nearby second vehicle starts and shifts into reverse. Since cars rarely shift into reverse while driving normally, the driver of the vehicle can assume that the second vehicle is pulling out of a parking space. A Car2Car message from the second vehicle provides notification of the gear change and the second vehicle's location. This information can be displayed on display 500 (see, for example, vehicle 502c). This notification and the displayed turn path 402 instruct the driver of the vehicle to move out of the way of the second vehicle and make a desired turn.

[0054] The preceding disclosure refers to the accompanying drawings, which form part of this document and illustrate specific implementations of the disclosure. It is understood that other implementations may be used and structural modifications made without deviating from the scope of this disclosure. References in the description to "an embodiment," "an exemplary embodiment," "an exemplary embodiment," etc., indicate that the described embodiment may include a particular feature, structure, or property, but not every embodiment necessarily includes that particular feature, structure, or property. Furthermore, such formulations do not necessarily refer to the same embodiment.Furthermore, it should be noted that if a particular feature, structure or property is described in connection with an embodiment, it is within the scope of the skilled person's knowledge to implement such a feature, structure or property in connection with other embodiments, whether this is expressly described or not.

[0055] Implementations of the systems, devices, and methods disclosed herein may include or utilize a specialized or general-purpose computer that incorporates computer hardware, such as one or more processors and system memory, as discussed herein. Implementations within the scope of this disclosure may also include physical and other computer-readable media for transporting or storing computer-executable instructions and / or data structures. Such computer-readable media may be any available media accessible by a general-purpose or specialized computer system. Computer-readable media on which computer-executable instructions are stored are computer storage media (devices). Computer-readable media that transport computer-executable instructions are transmission media.Thus, implementations of the disclosure may, for example, and without limitation, include at least two distinctly different types of computer-readable media: computer storage media (devices) and transmission media.

[0056] Computer storage media (devices) include RAM, ROM, EEPROM, CD-ROM, solid state drives (“SSDs”) (e.g., based on RAM), flash memory, phase-change memory (“PCM”), other types of storage, other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store desired program code resources in the form of computer-executable instructions or data structures and that can be accessed by a general-purpose or specialized computer.

[0057] An implementation of the devices, systems, and methods disclosed in this document can communicate via a computer network. A "network" is defined as one or more data connections that enable the transport of electronic data between computer systems and / or modules and / or other electronic devices. When information is transmitted to or provided to a computer via a network or other communication link (either wired, wireless, or any combination thereof), the computer correctly views the link as a transmission medium. Transmission media can include a network and / or data links that can be used to transport desired program code resources in the form of computer-executable instructions or data structures and that can be accessed by a general-purpose or specialized computer.Combinations of the above should also be included within the scope of computer-readable media.

[0058] Computer-executable instructions include, for example, instructions and data that, when executed on a processor, cause a general-purpose computer, a specialized computer, or a specialized processing device to perform a specific function or group of functions. Computer-executable instructions can be, for example, binary files, instructions in an intermediate format such as assembly language, or source code. Although the subject matter has been described in a specific language for structural features and / or methodological actions, it is understood that the subject matter defined in the appended claims is not necessarily limited to the features or actions described above. Rather, the described features and actions are disclosed as exemplary implementations of the claims.

[0059] The person skilled in the art will recognize that the disclosure can be implemented in network computing environments with many types of computer system configurations, including dashboard vehicle computers, personal computers, desktop computers, laptop computers, message processors, handheld devices, multiprocessor systems, microprocessor-based or programmable consumer electronics, network PCs, minicomputers, mainframe computers, mobile phones, PDAs, tablets, pagers, routers, switches, various storage devices, and the like. The disclosure can also be practically implemented in distributed systems environments where both local and remote computer systems connected by a network (either by wired data links, wireless data links, or a combination of both) perform tasks.In a distributed systems environment, program modules can reside in both local and remote storage devices.

[0060] Furthermore, the functions described in this document may be performed in one or more of the following: hardware, software, firmware, digital components, or analog components. For example, one or more application-specific integrated circuits (ASICs) may be programmed to perform one or more of the systems and operations described in this document. Certain terms are used throughout the description and in the claims to refer to specific system components. Those skilled in the art will understand that components may be referred to by different designations. This document does not distinguish between components that differ in name but not in function.

[0061] It should be noted that the sensor embodiments discussed above may include computer hardware, software, firmware, or any combination thereof to perform at least some of their functions. For example, a sensor may include computer code configured to run on one or more processors and may include a hardware logic / electrical circuit controlled by the computer code. These exemplary devices are provided in this document for illustrative purposes and are not intended to be limiting. Embodiments of the present disclosure may be implemented in other types of devices, as is known to the person skilled in the art.

[0062] At least some embodiments of the disclosure relate to computer program products comprising such logic (e.g., in the form of software) stored on any computer-usable medium. Such software, when executed in one or more data processing devices, causes a device to operate as described in the present document.

[0063] Although various embodiments of the present disclosure have been described above, it is understood that these serve only as examples and not as limitations. It is evident to the person skilled in the art that various modifications in form and detail can be made without departing from the spirit and scope of the disclosure. Thus, the breadth and scope of the present disclosure should not be limited by any of the exemplary embodiments described above, but rather defined solely by the following claims and their equivalents. The preceding description has been provided for illustrative purposes only. It makes no claim to completeness and is not intended to limit the disclosure to the specific form disclosed. Many modifications and variations are possible in light of the foregoing teachings.Furthermore, it should be noted that any or all of the aforementioned alternative implementations can be used in any desired combination to form additional hybrid implementations of the disclosure.

Claims

[1] Method comprising a control of a vehicle: Receiving output from one or more sensors; Identifying one or more obstacles according to the outputs of one or more sensors; Determining one or more paths of one or more obstacles, at least based on an object tracking technique; (a) Determining a turning path according to the locations of the one or more obstacles and the one or more paths of the one or more obstacles; (b) Presenting a visual representation of the turning path on a display device coupled to the control system for execution by a driver of the vehicle. Performing (a) and (b) in response to receiving an initial turn indication from the driver of the vehicle, wherein the initial turn indication comprises at least one of the following: a change in the steering wheel angle; Activating a direction indicator; and Capturing a turn in navigation data for a route selected by the driver, wherein (b) further includes: capturing a current steering angle of the vehicle; Determining a desired steering angle according to the turning path; and Displaying a second indicator of the difference between the current steering angle and the desired steering angle; and Selecting a color for the second indicator according to the magnitude of the difference between the current steering angle and the desired steering angle, wherein the color for the second indicator is determined based on a series of predetermined limits for the magnitude, and wherein the series of predetermined limits for the magnitude varies at least partially depending on the vehicle speed. [2] Method according to claim 1, wherein the one or more sensors include at least one of a camera, a LIDAR sensor (light detection and ranging sensor) and a RADAR sensor (radio detection and ranging sensor). [3] The method of claim 1, wherein the vehicle is a first vehicle, further comprising: Receiving a message from a second vehicle via a vehicle-to-vehicle (Car2Car) communication protocol; Receiving an additional obstacle location from the message; Retaining a set of vehicle characteristics of the first vehicle, wherein the set of vehicle characteristics includes at least a width, a length, a wheelbase size, a turning radius and a turning dynamics of the vehicle; and Performing (a) in accordance with the one or more obstacles, the additional obstacle and the series of vehicle characteristics. [4] Method according to claim 3, wherein the additional obstacle is an obstacle that is detected by a sensor of the second vehicle and wherein (a) according to the one or more obstacles and the additional obstacle is carried out at least partially on the basis that the additional obstacle is located within a threshold distance of a path of the first vehicle at a maximum distance in front of the vehicle. [5] The method of claim 3, further comprising: Determining an intended path for the second vehicle; and Performing (a) to avoid the intended path of the second vehicle. [6] Method according to claim 3, wherein presenting the visual representation of the turning path on the display device comprises presenting the turning path from a top-down perspective, wherein the visual representation further includes a visual representation of the additional obstacle, wherein the additional obstacle is outside the driver's field of vision. [7] System comprising one or more processing devices and one or more storage devices operatively coupled to the one or more storage devices, wherein the one or more storage devices store executable code which is effective inducing the one or more processing devices to: Receiving output from one or more sensors; Identifying one or more obstacles according to the outputs of one or more sensors; Determining one or more paths of one or more obstacles, at least based on an object tracking technique; (a) Determining a turning path according to the locations of the one or more obstacles and the one or more paths of the one or more obstacles; and (b) Presenting a visual representation of the turning path on a display device coupled to the control system for execution by a driver of the vehicle; Performing (a) and (b) in response to receiving an initial turn indication from the driver of the vehicle, wherein the initial turn indication comprises at least one of the following: a change in the steering wheel angle; Activating a direction indicator; and Capturing a turn in navigation data for a route selected by the driver, wherein the executable code further enables the one or more processing devices to (b) perform the following: Detecting the current steering angle of the vehicle; Determining a desired steering angle according to the turning path; and Displaying a second indicator of the difference between the current steering angle and the desired steering angle; and Displaying a color on the second indicator according to the magnitude of the difference between the current steering angle and the desired steering angle, wherein the color for the second indicator is determined based on a series of predetermined limits for the magnitude, and wherein the series of predetermined limits for the magnitude varies at least partially depending on the vehicle speed. [8] System according to claim 7, wherein the one or more sensors include at least one of a camera, a LIDAR sensor and a RADAR sensor. [9] System according to claim 7, wherein the vehicle is a first vehicle, and wherein the executable code further causes the one or more processing devices to: Receiving a message from a second vehicle via a vehicle-to-vehicle (Car2Car) communication protocol; Receiving an additional obstacle location from the message; Retaining a set of vehicle characteristics of the first vehicle, wherein the set of vehicle characteristics includes at least a width, a length, a wheelbase size, a turning radius and a turning dynamics of the vehicle; and Performing (a) in accordance with the one or more obstacles, the additional obstacle, and the series of vehicle characteristics. [10] System according to claim 9, wherein the additional obstacle is an obstacle detected by a sensor of the second vehicle, and wherein (a) according to the one or more obstacles and the additional obstacle is carried out at least partially on the basis that the additional obstacle is located within a threshold distance of a path of the first vehicle at a maximum distance in front of the vehicle. [11] System according to claim 9, wherein the executable code further enables the one or more processing devices to: Determining an intended path for the second vehicle; and Performing (a) to avoid the intended path of the second vehicle. [12] System according to claim 9, wherein the executable code further enables the one or more processing devices to present the visual representation of the turning path on the display device by presenting the turning path from a top-down perspective, wherein the visual representation further includes a visual representation of the additional obstacle, wherein the additional obstacle is outside the driver's field of vision.

Images