Systems and methods for controlling vehicles

Abstract

Systems, methods, and other embodiments described herein relate to controlling a vehicle. In one embodiment, a method includes displaying a guideline on a user interface that specifies a virtual trajectory for the vehicle. The guideline is based on a steering angle input and a direction input. The method includes updating a shape of the guideline based on user input along a length of the guideline on the user interface to generate an updated guideline, and sending a control signal to a vehicle control system that causes the vehicle to follow the updated guideline.

Description

[Technical Field]

[0001] The subject matter described herein relates generally to controlling vehicles, and more particularly to controlling autonomous vehicles. [Background technology]

[0002] Modern vehicles include one or more cameras and displays that can provide reverse driving assistance. Some vehicles also include cameras that provide forward driving assistance. Vehicles may use visual overlay graphics that are superimposed on the camera image to provide driving assistance. However, visual overlay graphics that represent potential paths of travel are limited to displaying a potential path of travel based on steering angle. Summary of the Invention

[0003] This section provides an overview of what is disclosed and is not an exhaustive description of the entire scope or every feature of what is disclosed.

[0004] In one embodiment, a method for controlling a vehicle is disclosed. The method includes displaying guidelines on a user interface that identify a potential trajectory for the vehicle. The guidelines are based on a steering angle input and a direction input. The method includes updating a shape of the guidelines based on user input along a length of the guidelines on the user interface to generate updated guidelines. The method further includes sending a control signal to a vehicle control system that causes the vehicle to follow the updated guidelines.

[0005] In another embodiment, a system for controlling a trailer is disclosed. The system includes a processor and a memory in communication with the processor. The memory stores computer-readable instructions that, when executed by the processor, cause the processor to display guidelines on a user interface that specify a virtual trajectory for the vehicle. The guidelines are based on a steering angle input and a direction input. The memory stores machine-readable instructions that, when executed by the processor, cause the processor to update the shape of the guidelines based on user input along a length of the guidelines on the user interface to generate updated guidelines. The memory stores machine-readable instructions that, when executed by the processor, cause the processor to send control signals to a vehicle control system that causes the vehicle to follow the updated guidelines.

[0006] In another aspect, a non-transitory computer-readable medium for controlling a vehicle is disclosed, the medium including instructions that, when executed by a processor, cause the processor to perform one or more functions. The instructions include instructions for displaying guidelines on a user interface that specify a virtual trajectory for the vehicle. The guidelines are based on a steering angle input and a direction input. The instructions include instructions for updating a shape of the guidelines based on user input along a length of the guidelines on the user interface to generate updated guidelines. The instructions store instructions for transmitting control signals to a vehicle control system that causes the vehicle to follow the updated guidelines. [Brief explanation of the drawings]

[0007] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate various systems, methods, and other embodiments of the disclosure. It will be understood that the boundaries of elements shown in the drawings (e.g., boxes, groups of boxes, or other shapes) represent one embodiment of the boundaries. In some embodiments, one element may be designed as multiple elements, or multiple elements may be designed as one element. In some embodiments, an element shown as an internal component of another element may be implemented as an external component, and vice versa. Additionally, components may not be drawn to scale. [Figure 1] FIG. 1 depicts a block diagram of a vehicle incorporating an automated track guidance and execution system. [Figure 2] FIG. 2 is a more detailed block diagram of the automated trajectory guidance and execution system of FIG. [Figure 3] FIG. 3 is an example of a method for controlling a vehicle. [Figures 4A-4E] 4A-4E are an example of an automated trajectory guidance and execution system. DETAILED DESCRIPTION OF THE INVENTION

[0008] Systems, methods, and other embodiments are disclosed that relate to controlling one or more vehicles, and more particularly, to controlling a vehicle to travel along a selected trajectory.

[0009] The vehicle may include a display located inside the vehicle cabin and on the dashboard. The display may be adapted to display guidelines that indicate a driving path for the vehicle based on the angle of the steering wheel. However, the display is limited to outputting information and does not allow a user to control the vehicle by inputting information via the display.

[0010] Thus, in one embodiment, the disclosed approach is a system that assists a user in controlling the path of a vehicle via a user interface, such as a display screen and input components. The user interface may be located inside the vehicle. Alternatively, the user interface may be located on a mobile device or any other suitable device that is external to the vehicle. Thus, the system may be used to assist a user in remotely controlling a vehicle.

[0011] The system may be activated by a control signal. The control signal may be based on the user placing the vehicle in reverse or drive mode, turning the steering wheel, or pressing a control button. In response to the control signal, the system generates first (also known as original) guidelines using the steering wheel and based on the steering angle. The curvature of the guidelines is based on the steering angle. Thus, the greater the steering angle, the steeper the guideline curve. The system displays the original guidelines on a display screen. The display may include a representation of the vehicle and guidelines extending from the front or rear of the vehicle. The system may display a representation of the vehicle and guidelines in any suitable view. As one example, the system may display a representation of the vehicle and guidelines from a bird's-eye view. As another example, the system may display a representation of the vehicle and guidelines from a panoramic view based on the perspective of a vehicle sensor, such as a vehicle camera.

[0012] As an example, the system then receives user input in the form of the user touching a user interface, such as a touchscreen, to press on a portion of the length of the original guideline to change its shape. The user may input additional guidelines (also known as branching guidelines) that begin at a point on the original guideline before or after the original guideline is changed and extend away from the original guideline. The system may update the curvature and destination point of the original guideline based on the changes from the user. Alternatively, the system may update the display to include one or more additional guidelines. The system includes launching the vehicle to autonomously travel along a trajectory based on the updated guidelines.

[0013] The current art does not disclose that a user can control a vehicle using a user interface that allows the user to use a finger (or any suitable finger), a stylus, or another input component such as a rotatable knob or slide to press the length of a displayed guide line to change the curvature of the vehicle's travel path.

[0014] The embodiments disclosed herein offer various advantages over current technology. First, the embodiments may be implemented without the use of complex maps or global positioning systems (GPS). The length of the guideline and subsequent trajectory along which the vehicle travels is limited to the range (or coverage area) of the vehicle's sensor system, e.g., a 100-yard radius. Thus, the system generates paths that are in segments, e.g., 100-yard segments. Additionally, these embodiments may be implemented with a relatively small number of vehicle sensors. Also, these embodiments may be implemented with limited computing and data storage resources.

[0015] Second, embodiments help the user maintain precise control over the vehicle as it makes each turn. Thus, if the vehicle deviates from the desired trajectory, the system provides the user with the ability to return the vehicle to the desired trajectory. These embodiments allow the user to control the vehicle remotely using a touchscreen or any other suitable input component, such as a knob, slide, switch, keypad, button, joystick, mouse, trackball, or microphone.

[0016] As an example, these systems may be useful in assisting a user who is attempting to move an inaccessible vehicle (e.g., the vehicle is parked in a tight parking spot). A user outside the vehicle may utilize the system to move the vehicle from an inaccessible location to an open space where the vehicle doors can be opened, allowing the user access to the vehicle.

[0017] Detailed embodiments are disclosed herein. However, it should be understood that the disclosed embodiments are intended to be examples only. Therefore, the specific structural and functional details disclosed herein should not be construed as limiting, but merely as a basis for the claims and as a representative basis for teaching those skilled in the art various uses of the aspects of this specification in substantially any suitable detailed configuration. Furthermore, the terms and expressions used herein are not intended to be limiting, but rather to provide an understandable description of possible implementations. Although various embodiments are shown in the drawings, the embodiments are not limited to the shown structures or applications.

[0018] It will be understood that for simplicity and clarity of the drawings, where appropriate, reference numerals have been repeated among different figures to indicate corresponding or analogous elements. Additionally, numerous specific details are disclosed to provide a thorough understanding of the embodiments described herein. However, it will be understood by those skilled in the art that the embodiments described herein may be practiced without these specific details.

[0019] 1, there is shown a block diagram of a vehicle 102 incorporating an automated track guidance and execution system 100. The vehicle 102 includes a sensor system, a user interface, the automated track guidance and execution system 100, and various other elements.

[0020] The vehicle 102 may include a user interface 130. The user interface 130 may receive input from a user (e.g., a person) or another entity and / or present output to a user or other entity. The user interface 130 includes any device, component, system, element, or arrangement, or group thereof, that enables information / data to be input into a machine. The user interface 130 also includes any device, component, system, element, or arrangement, or group thereof, that enables information / data to be presented to a user or other entity. The user interface 130 may be located on a dashboard on the vehicle 102 or in any suitable location on the vehicle 102. Additionally and / or alternatively, the user interface 130 may be located on a mobile device. By way of example, the user interface 130 may be a device that a user can see, hear, touch, push, turn, and / or speak to. In one such example, the user interface 130 may include a touchscreen 135, a multi-touch screen, a display, a knob, a slide, a switch, a keypad, a button, a joystick, a mouse, a trackball, a microphone, gesture recognition (radar, lidar, camera, or ultrasound-based), and / or combinations thereof. The user interface 130 may function as both an input device and an output device, such as the touchscreen 135. The touchscreen 135 may receive information via a stylus, a user's finger or thumb, or any other suitable device. As described above, the user interface 130 may be located on at least one of the vehicle 102 or a mobile device. Thus, the user interface 130 may be located on a device external to the vehicle 102.

[0021] It should be understood that in various embodiments, the vehicle 102 need not include all of the elements depicted in FIG. 1 . The vehicle 102 may include any combination of the various elements depicted in FIG. 1 . Furthermore, the vehicle 102 may include additional elements to those depicted in FIG. 1 . In some configurations, the vehicle 102 may be implemented without one or more of the elements depicted in FIG. 1 . While various elements are shown as being located within the vehicle 102 in FIG. 1 , it should be understood that one or more of these elements may be located external to the vehicle 102. Furthermore, the depicted elements may be physically separated by large distances. For example, as described above, one or more components of the automated path guidance and execution system 100 may be implemented within the vehicle 102, while one or more additional components of the automated path guidance and execution system 100 may be implemented within a cloud computing environment.

[0022] Some possible elements of the vehicle 102 are shown in FIG. 1 and described along with subsequent figures. However, a description of many of the elements in FIG. 1 will be provided following the description of FIGS. 2-4 for purposes of brevity of this description. Additionally, it should be understood that, for simplicity and clarity of the drawings, where appropriate, reference numerals may be repeated among different figures to indicate corresponding or similar elements. Additionally, this description outlines numerous specific details to provide a thorough understanding of the embodiments described herein. However, those skilled in the art will understand that the embodiments described herein may be implemented using various combinations of these elements. In any event, as shown in the embodiment of FIG. 1, the vehicle 102 includes an automated track guidance and execution system 100 implemented to perform the methods and other functions described herein related to controlling the vehicle. By way of example, in various embodiments, the automated track guidance and execution system 100 may be implemented partially within the vehicle 102 and may further communicate with additional elements of the automated track guidance and execution system 100 that are remote from the vehicle 102 and that support the disclosed functionality. Thus, although FIG. 2 generally depicts the automated track guidance and execution system 100 as being self-contained, in various embodiments, the automated track guidance and execution system 100 may be implemented in multiple separate devices that may be remote from the vehicle 102.

[0023] 2, a more detailed block diagram of the automated track guidance and execution system 100 is shown. The automated track guidance and execution system 100 may include a processor 110. As such, the processor 110 may be part of the automated track guidance and execution system 100, which may access the processor 110 via a data bus or another communication path. In one or more embodiments, the processor 110 is an application specific integrated circuit that may be configured to implement the functionality associated with the control module 220. More generally, in one or more aspects, the processor 110 is an electronic processor, such as a microprocessor, that, when loaded with the control module 220 and executing the encoded functions associated therewith, is capable of performing the various functions described herein.

[0024] The automated trajectory guidance and execution system 100 may include memory 210 that stores a control module 220. The memory 210 may be random access memory (RAM), read-only memory (ROM), a hard disk drive, flash memory, or other suitable memory for storing the control module 220. The control module 220 is, for example, a set of computer-readable instructions that, when executed by the processor 110, causes the processor 110 to perform various functions described herein. While in one or more embodiments, the control module 220 is a set of instructions embodied in the memory 210, in further aspects, the control module 220 includes hardware, such as processing components (e.g., controllers) or circuitry, for independently performing one or more of the aforementioned functions.

[0025] The automated trajectory guidance and execution system 100 may include a data store 115 for storing one or more types of data. Accordingly, the data store 115 may be part of the automated trajectory guidance and execution system 100, which may access the data store 115 via a data bus or another communication path. In one embodiment, the data store 115 is an electronic data structure for storing information. In at least one approach, the data store 115 is a database stored in the memory 210 or another suitable medium and configured with routines that can be executed by the processor 110 to analyze the stored data, present the stored data, organize the stored data, etc. In either case, the data store 115 stores data used by the control module 220 in performing various functions. In one embodiment, the data store 115 may be capable of storing sensor data 119 and / or other information used by the control module 220.

[0026] The data store 115 may include volatile and / or non-volatile memory. Examples of suitable data stores 250 include RAM (random access memory), flash memory, ROM (read only memory), PROM (programmable read only memory), EPROM (erasable programmable read only memory), EEPROM (electrically erasable programmable read only memory), registers, magnetic disks, optical disks, hard drives, or any other suitable storage medium, or any combination thereof. The data store 115 may be a component of the processor 110, and the data store 115 may be operatively connected to the processor 110 for use by the processor. The terms "operatively connected" or "in communication," as used throughout this specification, can include direct or indirect connections, including connections where there is no direct physical connection.

[0027] In one or more configurations, the data store 115 may include sensor data 119. The sensor data 119 may originate from a sensor system 120 of the vehicle 102. The sensor data 119 may include data from visual sensors, auditory sensors, and / or any other suitable sensors present on the vehicle 102. The sensor data 119 may include images from the front of the vehicle, the rear of the vehicle, and / or the side of the vehicle. As an example, the sensor data 119 may include geographic coordinates of the vehicle 102 and / or any vehicles or objects surrounding the vehicle 102. In another example, the sensor data 119 may include relative positions of the vehicle 102 and / or any vehicles or objects surrounding the vehicle 102.

[0028] In one embodiment, the control module 220 may include instructions that, when executed by the processor 110, cause the processor 110 to display guidelines on the user interface 130 that identify a virtual trajectory for the vehicle 102. As an example, the control module 220 may display one or more guidelines on the touch screen 135 that identify a virtual trajectory for the vehicle 102. The guidelines are based on a steering angle input and a direction input. The steering angle input may be a clockwise or counterclockwise rotation of the steering wheel. The direction input may be a forward direction, indicated by the vehicle 102 being in drive mode, or a backward direction, indicated by the vehicle 102 being in reverse mode.

[0029] In one embodiment, the control module 220 may receive data input in the form of a steering angle of the vehicle 102. Additionally, the control module 220 may receive data input in the form of a direction of a virtual driving path. The control module 220 may use a variety of active or passive techniques to obtain the steering angle and / or the direction of the virtual driving path. For example, the control module 220 may passively sniff data input from streams of electronic information provided by various sensors to additional components within the vehicle 102. In another example, the control module 220 may actively request and / or poll the steering system 143 and / or the transmission system 145 for data input. In one such example, the control module 220 may receive the steering angle from the steering system 143 and / or the direction (e.g., forward or reverse) from the transmission system 145.

[0030] The control module 220 generates guidelines including length, curvature, and / or direction based on the steering angle input and the direction input. As one example, if the steering wheel is rotated counterclockwise, the guidelines extending from the position of the vehicle 102 may extend to the left. As another example, if the steering wheel is rotated clockwise, the guidelines extending from the position of the vehicle 102 may extend to the right. The guidelines may extend from the front of the vehicle 102 if the direction is forward, or from the rear of the vehicle 102 if the direction is rearward. The control module 220 may utilize any suitable method or algorithm for determining and generating the guidelines. The control module 220 then displays the guidelines on the user interface 130, and more specifically, on the touchscreen 135. As described above, the control module 220 displays the guidelines on the touchscreen 135, specifying a virtual trajectory for the vehicle 102. The control module 220 may display a representation of the vehicle 102 and a guideline that originates from the front or rear of the representation of the vehicle 102 and leads to a destination point. The destination point of the guideline is based on the range of sensors located on the vehicle 102. In other words, the guideline may have two end points: an origin point and a destination point. The origin point of the guideline is located on the vehicle 102. The origin point of the guideline may originate from the front or rear of the vehicle 102 depending on the directional input. The destination point is the other end of the guideline. The destination point is within the range of sensors in the sensor system 120. In other words, the destination point may extend as far as it can without exceeding the area visible by any sensors on the sensor system 120 of the vehicle 102.

[0031] In one embodiment, the control module 220 may include instructions, when executed by the processor 110, that cause the processor 110 to update the shape of a guideline (also known as an original guideline) based on user input along the length of the original guideline on the user interface 130 and / or touch screen 135 to generate an updated guideline. The user input may change the curvature along the length of the original guideline and / or the destination point of the original guideline to generate the updated guideline. The user input may also generate another guideline (also known as a branching guideline). A branching guideline may include an origin point and a destination point. The origin point of a branching guideline may be anywhere along the length of either the original guideline or the updated guideline. In other words, the branching guideline may originate at a point along the length of the original guideline or the updated guideline. Additionally, the destination point of the branching guideline is within range of the sensor system 120 of the vehicle 102.

[0032] The control module 220 may receive user input from a user via a stylus or the user's finger. As one example, the control module 220 may change the curvature of the original guideline in response to the user pressing along the length of the original guideline with a stylus. As another example, the control module 220 may extend the length of the original guideline and / or the location of the destination point in response to the user extending the length of the original guideline using a stylus or finger.

[0033] As an example, the control module 220 may receive user input from a user via a user interface 130 that may include knobs, slides, switches, keypads, buttons, joysticks, mice, trackballs, microphones, gesture recognition (radar, lidar, camera, or ultrasonic based), and / or combinations thereof.

[0034] As an example, in response to a user drawing a branching guideline by touching a point along the length of the original guideline or the updated guideline, the control module 220 may generate a branching guideline that begins at a point along the length of the original guideline, extends in the direction of the user's drawing, and terminates at the destination point. In one example, the original guideline and a portion of the branching guideline become the updated guideline. In one such example, the updated guideline includes the original guideline from the point of origin to the point on the original guideline where the branching guideline begins, and the length of the branching guideline. Alternatively, the control module 220 may change the original guideline to a first updated guideline and generate a branching guideline that begins at a point on the first updated guideline. In such a case, the first updated guideline from the point of origin (on the vehicle 102) to the point on the first updated guideline where the branching guideline begins, and the length of the branching guideline, form the second updated guideline.

[0035] In one embodiment, the control module 220 may include instructions that, when executed by the processor 110, cause the processor 110 to send a control signal to a vehicle control system that causes the vehicle 102 to travel along the updated guideline. Thus, in response to the vehicle control system receiving the control signal, the vehicle control system may cause the vehicle 102 to travel along the updated guideline or the second updated guideline. The vehicle 102 may employ any suitable vehicle control system for identifying and traveling along the updated guideline. As an example, the vehicle 102 may be an autonomous vehicle and may include an autonomous driving system 160 that may control the vehicle 102 to travel along the updated guideline or the second updated guideline.

[0036] The control module 220 may detect an obstacle along the updated guideline or the second updated guideline, and in response to the vehicle sensors detecting an obstacle along the updated guideline or the second updated guideline, may control the vehicle 102 to slow down or stop traveling along the updated guideline or the second updated guideline. The control module 220 may control the vehicle 102 to switch from traveling on the original guideline or the first updated guideline to traveling on a branch guideline when the control module 220 detects a branch (or junction) in the guideline.

[0037] Figure 3 illustrates a method 300 for controlling a vehicle. Method 300 is described in terms of vehicle 102 of Figure 1 and automated track guidance and execution system 100 of Figure 2. However, method 300 may be adapted to be performed in any one of several different situations, and not necessarily by the vehicle of Figure 1 and / or automated track guidance and execution system 100 of Figure 2.

[0038] In step 310, control module 220 may cause processor 110 to display guidelines (also known as original guidelines) on the user interface that specify a virtual trajectory for the vehicle. The guidelines are based on the steering angle input and the direction input. As described above, as an example, control module 220 may generate guidelines based on the steering angle input and the direction input. Control module 220 may then display the guidelines on the user interface.

[0039] At step 320, control module 220 may cause processor 110 to update the shape of the guideline based on user input along the length of the guideline on the user interface to generate an updated guideline. As described above, control module 220 may change the shape of the guideline based on the user touching the guideline and swiping the guideline left or right. Control module 220 may generate the updated guideline based on the changed shape of the original guideline.

[0040] In step 330, the control module 220 may cause the processor 110 to send a control signal to a vehicle control system that causes the vehicle 102 to follow the updated guidelines. As described above, the control module 220 may activate a vehicle control system, such as the autonomous driving system 160, that causes the vehicle 102 to follow the updated guidelines. As an example, the control module 220 may determine the travel speed of the vehicle 102 and may also select a guideline for the vehicle 102 to follow when there are multiple guidelines, such as multiple branching guidelines.

[0041] Method 300 may then end. Alternatively, method 300 may return to step 310 or some other step.

[0042] Non-limiting examples of the operation of and / or one or more methods of the automated trajectory guidance and execution system 100 are immediately described in connection with Figures 4A-4E. Figure 4A illustrates an example driving scenario in which a user 420 is rotating a steering wheel 404 counterclockwise and the vehicle 402 is in a reverse mode 406. The vehicle 402 includes a touchscreen 408. In response to the user placing the vehicle 402 in the reverse mode 406, the control module 220 displays a representation of the vehicle 402 and guidelines 412 on the touchscreen 408.

[0043] 4B, a user 420 modifies the guideline 412 by touching two points 414A, 414B along the length of the guideline 412 on the touch screen 408. The control module 220 receives the modifications input by the user 420.

[0044] 4C, the control module 220 updates the guideline 412 to an updated guideline 412B based on the changes received from the user 420. As an example, the user 420 may activate the vehicle 402 to travel along the updated guideline 412B, and in response, the control module 220 may activate the autonomous vehicle system to cause the vehicle 402 to travel along the updated guideline 412B.

[0045] As another example, as shown in FIG. 4D, a user 420 may touch a point 414C along the length of the updated guideline 412B on the touchscreen 408 to draw another guideline (also known as a branching guideline) 412C.

[0046] As shown in FIG. 4E, the control module 220 sends a control signal to the vehicle control system that causes the vehicle 402 to travel along the updated guideline 412B until the vehicle 402 reaches point 414C where the updated guideline 412B and the branch guideline 412C merge, and the control module 220 may select the vehicle 402 to travel along the branch guideline 412C based on input from the user 420.

[0047] 1 is described in greater detail below as an example environment in which the systems and methods disclosed herein may operate. In some cases, the vehicle 102 is configured to selectively switch between an autonomous mode, one or more semi-autonomous operating modes, and / or a manual mode. Such switching may be performed in any suitable manner now known or later developed. "Manual mode" means that all or most of the navigation and / or vehicle operation is performed according to input received from a user (e.g., a human driver). In one or more configurations, the vehicle 102 may be a conventional vehicle configured to operate only in manual mode.

[0048] In one or more embodiments, the vehicle 102 may be an autonomous vehicle. As used herein, "autonomous vehicle" refers to a vehicle operating in an autonomous mode. "Autonomous mode" refers to navigating and / or maneuvering the vehicle 102 along a path with little or no input from a human driver, using one or more computing systems to control the vehicle 102. In one or more embodiments, the vehicle 102 is highly autonomous or fully autonomous. In some embodiments, the vehicle 102 is configured with one or more semi-autonomous operating modes in which one or more computing systems perform a portion of the navigation and / or maneuvering of the vehicle along a path, and an operator (i.e., driver) of the vehicle provides input to the vehicle to perform a portion of the navigation and / or maneuvering of the vehicle 102 along a path.

[0049] The vehicle 102 may include one or more processors 110. In one or more configurations, the processor 110 may be the main processor of the vehicle 102. For example, the processor may be an electronic control unit (ECU). The vehicle 102 may include one or more data stores 115 for storing one or more types of data. The data stores 115 may include volatile and / or non-volatile memory. Examples of suitable data stores 115 include RAM (random access memory), flash memory, ROM (read-only memory), PROM (programmable read-only memory), EPROM (erasable programmable read-only memory), EEPROM (electrically erasable programmable read-only memory), registers, magnetic disks, optical disks, hard drives, or any other suitable storage medium, or any combination thereof. The data stores 115 may be components of the processor 110, or the data stores 115 may be operatively connected to the processor 110 for use by the processor. The term "operably connected," as used throughout this specification, can include direct or indirect connections, including connections that do not involve direct physical contact.

[0050] One or more data stores 115 may include sensor data 119. In this context, "sensor data" means any information about sensors equipped on the vehicle 102, including performance and other information about such sensors. As described below, the vehicle 102 may include a sensor system 120. The sensor data 119 may relate to one or more sensors of the sensor system 120. As an example, in one or more configurations, the sensor data 119 may include information from one or more vehicle sensors 121 and / or environmental sensors 122 of the sensor system 120.

[0051] In some cases, at least a portion of the sensor data 119 may be stored in one or more data stores 115 onboard the vehicle 102. Alternatively, or additionally, at least a portion of the sensor data 119 may be stored in one or more data stores 115 located remotely from the vehicle 102.

[0052] As described above, the vehicle 102 may include a sensor system 120. The sensor system 120 may include one or more sensors. A "sensor" refers to any device, component, and / or system that can detect and / or sense something. The one or more sensors may be configured to detect and / or sense in real time. As used herein, the term "real time" refers to a level of processing responsiveness that a user or system perceives as being immediate enough to make a particular process or decision, or that allows a processor to keep up with some external process.

[0053] In configurations in which sensor system 120 includes multiple sensors, the sensors may operate independently of one another. Alternatively, two or more sensors may operate in combination with one another. In such cases, the two or more sensors may form a sensor network. Sensor system 120 and / or one or more sensors may be operatively connected to processor 110, data store 115, and / or other elements of vehicle 102 (including any elements shown in FIG. 1). Sensor system 120 may acquire data about at least a portion of the external environment (e.g., the vicinity of the vehicle) and the internal environment of vehicle 102.

[0054] The sensor system 120 may include any suitable type of sensor. Various examples of different types of sensors are described herein. However, it will be understood that embodiments are not limited to the particular sensors described. The sensor system 120 may include one or more vehicle sensors 121. The vehicle sensors 121 may detect, measure, and / or sense information about the vehicle 102 itself. In one or more configurations, the vehicle sensors 121 may be configured to detect and / or sense changes in the position and orientation of the vehicle 102, for example, based on inertial acceleration. In one or more configurations, the vehicle sensors 121 may include one or more accelerometers, one or more gyroscopes, an inertial measurement unit (IMU), an autonomous navigation system, a global navigation satellite system (GNSS), a global positioning system (GPS), a navigation system 147, and / or other suitable sensors. The vehicle sensors 121 may be configured to detect and / or sense one or more characteristics of the vehicle 102. In one or more configurations, the vehicle sensors 121 may include a speedometer for measuring the current speed of the vehicle 102 .

[0055] Alternatively or additionally, the sensor system 120 may include one or more environmental sensors 122 configured to acquire and / or sense data about the vehicle's surroundings. The sensor data about the vehicle's surroundings may include information about the external environment in which the vehicle is located, or one or more portions thereof.

[0056] As an example, the one or more environmental sensors 122 may be configured to detect, quantify, and / or sense vehicles or other objects and / or information / data about such vehicles or objects in at least a portion of an environment external to the vehicle 102. In the external environment, the one or more environmental sensors 122 may be configured to detect, measure, quantify, and / or sense vehicles and objects in the environment external to the vehicle 102, such as, for example, lane markers, signs, traffic signals, traffic signs, lane markings, crosswalks, curbs near the vehicle 102, off-road objects, electronic roadside devices, etc.

[0057] Described herein are various example sensors for sensor system 120. The example sensors may be part of one or more environmental sensors 122 and / or one or more vehicle sensors 121. However, it will be understood that embodiments are not limited to the particular sensors described.

[0058] By way of example, in one or more configurations, sensor system 120 may include one or more radar sensors 123, one or more LIDAR sensors 124, one or more sonar sensors 125, and / or one or more cameras 126. In one or more configurations, one or more cameras 126 may be high dynamic range (HDR) cameras or infrared (IR) cameras.

[0059] The vehicle 102 may include one or more communication modules 132. A "communication module" means a component designed to transmit and / or receive information from one source to another. The one or more communication modules 132 transmit and / or receive information over one or more communication networks. The communication networks may include an external communication network and an internal vehicle communication network.

[0060] The internal vehicle communication network may include a bus within the vehicle 102, such as a controller area network (CAN), and / or other wired and / or wireless mechanisms. Vehicle elements, such as the data store 115, the sensor system 120, and the processor 110, may be communicatively connected to one another via the internal vehicle communication network. As used herein, the term "communicatively connected" includes direct or indirect connections via communication channels or pathways or other components or systems. Each element of the vehicle 102 may include and / or execute appropriate communication software that enables the various elements to communicate with one another over the communication network and perform the functions disclosed herein.

[0061] An external communication network refers to one or more mechanisms that enable the vehicle 102 to communicate with other vehicles and / or objects, such as trailers, other vehicles, external servers, edge devices, and / or roadside units. The external communication network may be implemented as or include, without limitation, a wide area network (WAN), a local area network (LAN), a public switched telephone network (PSTN), a wireless network, a mobile network, a virtual private network (VPN), the Internet, one or more intranets, vehicle-to-vehicle (V2V) communication, vehicle-to-cloud (V2C) communication, vehicle-to-infrastructure (V2I) communication, and / or some other form of vehicle-to-everything (V2X) wireless communication. The external communication network may be implemented as or include one or more wireless networks, whether short-range (e.g., a local wireless network established using one of the IEEE 802 wireless communication protocols such as Bluetooth, 802.11a / b / g / i, 802.15, 802.16, 802.20, Wi-Fi Protected Access (WPA) or WPA2) or long-range (e.g., mobile, cellular, and / or satellite-based wireless networks such as GSM networks, TDMA networks, CDMA networks, WCDMA networks, etc.). The communication module 132 may include wired and / or wireless communication connections. The communication module 132 may include any combination of the above networks and / or other types of networks.

[0062] The vehicle 102 may include one or more vehicle systems 140. Various examples of the one or more vehicle systems 140 are shown in FIG. 1 . However, the vehicle 102 may include more, fewer, or different vehicle systems 140. While certain vehicle systems are defined separately, it should be understood that each or any of the systems or portions thereof may be otherwise combined or separated via hardware and / or software within the vehicle 102. The vehicle 102 may include a propulsion system 141, a braking system 142, a steering system 143, a throttling system 144, a transmission system 145, a signal transmission system 146, and / or a navigation system 147. Each of these systems may include one or more now known or later developed devices, components, and / or combinations thereof.

[0063] Navigation system 147 may include one or more now known or later developed devices, applications, and / or combinations thereof configured to determine the geographic location of vehicle 102 and / or determine a route for vehicle 102. Navigation system 147 may include one or more mapping applications to determine a route for vehicle 102. Navigation system 147 may include a global positioning system, a local positioning system, or a geolocation system.

[0064] The vehicle 102 may include one or more autonomous driving systems 160. The autonomous driving system 160 may include one or more now known or later developed devices, applications, and / or combinations thereof configured to control the movement, speed, steering, heading, direction, etc. of the vehicle 102. The autonomous driving system 160 may include one or more driver assistance systems, such as a lane keeping system, a lane centering system, a collision avoidance system, and / or a driver monitoring system.

[0065] Autonomous driving system 160 can be configured to receive data from sensor system 120 and / or from any other type of system capable of obtaining information about vehicle 102 and / or the environment external to vehicle 102. In one or more configurations, autonomous driving system 160 can use such data to generate one or more driving scene models. Autonomous driving system 160 can determine the position and speed of vehicle 102. Autonomous driving system 160 can determine the location of obstacles, obstacles, or other environmental features, including traffic signs, trees, shrubs, adjacent vehicles, pedestrians, etc.

[0066] The vehicle 102 may include one or more actuators 150. The actuators 150 may be any element or combination of elements operable to modify, adjust, and / or alter one or more vehicle systems 140 or components thereof in response to receiving signals or other inputs from the processor 110 and / or the autonomous driving system 160. Any suitable actuator may be used. For example, the one or more actuators 150 may include motors, pneumatic actuators, hydraulic pistons, relays, solenoids, and / or piezoelectric actuators, to list just a few possibilities.

[0067] Vehicle 102 may include one or more modules, at least some of which are described herein. The modules may be implemented as computer-readable program code that, when executed by processor 110, performs one or more of the various operations described herein. One or more modules may be components of processor 110, or one or more modules may be executed on and / or distributed across other processing systems operatively connected to processor 110. A module may include instructions (e.g., program logic) executable by one or more processors 110. Alternatively, or additionally, one or more data stores 115 may include such instructions.

[0068] In one or more configurations, one or more of the modules described herein may include elements of artificial intelligence or computational intelligence, such as neural networks, fuzzy logic, or other machine learning algorithms. Further, in one or more configurations, one or more of the modules described herein may be distributed across multiple modules described herein. In one or more configurations, two or more of the modules described herein may be combined into a single module.

[0069] Detailed embodiments have been described herein. However, it should be understood that the disclosed embodiments are intended to be examples only. Therefore, the specific structural and functional details disclosed herein should not be construed as limiting, but merely as a basis for the claims and as a representative basis for teaching those skilled in the art various uses of the aspects of the present specification in substantially any suitable detailed configuration. Furthermore, the terms and phrases used herein are not intended to be limiting, but rather to provide an understandable description of possible implementations. Although various embodiments are shown in Figures 1-4, the embodiments are not limited to the structures or applications shown.

[0070] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments. In this regard, each block in the flowcharts or block diagrams represents a module, segment, or portion of code, each of which may comprise one or more executable instructions for implementing a particular logical function. It should also be noted that in some alternative implementations, the functions noted in the blocks may occur out of the order noted in the figures. For example, two blocks shown in succession may in fact be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending on the functionality involved.

[0071] The systems, components, and / or processes described above can be implemented in hardware or a combination of hardware and software, and can be implemented in a centralized manner in one processing system or in a distributed manner where different elements are spread across several interconnected processing systems. Any type of processing system or other device configured to perform the methods described herein is suitable. A basic combination of hardware and software can be a processing system comprising computer-usable program code that, when loaded and executed, controls the processing system to perform the methods described herein. The systems, components, and / or processes can also be embodied in computer-readable storage, such as a computer program product or other data / program storage device, which is machine-readable and tangibly embodies program instructions executable by the machine to perform the methods and processes described herein. These elements can also be embodied in an application product that has all features enabling implementation of the methods described herein and that can execute these methods when loaded into a processing system.

[0072] Furthermore, the arrangements described herein may take the form of a computer program product embodied in, e.g., stored on, one or more computer-readable medium(s) having computer-readable program code embodied therein. Any combination of one or more computer-readable medium(s) may be utilized. The computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. The expression "computer-readable storage medium" refers to a non-transitory storage medium. A computer-readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of computer-readable storage media include a portable computer floppy disk, a hard disk drive (HDD), a solid-state drive (SSD), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disc read-only memory (CD-ROM), a digital versatile disc (DVD), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the document context, a computer-readable storage medium may be any tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device.

[0073] Generally, a module, as used herein, includes routines, programs, objects, components, data structures, etc. that perform a particular task or implement a particular data type. In a further aspect, a memory generally stores the aforementioned modules. The memory associated with a module may be a cache or buffer integrated into a processor, RAM, ROM, flash memory, or another suitable electronic storage medium. In a further aspect, a module contemplated by the present disclosure is implemented as an application-specific integrated circuit (ASIC), a system-on-chip (SoC) hardware component, a programmable logic array (PLA), or another suitable hardware component incorporating a defined configuration set (e.g., instructions) to perform the disclosed functions.

[0074] The program code embodied on a computer-readable medium may be transmitted using any suitable medium, including, but not limited to, wireless, wired, fiber optic, cable, RF, or the like, or any suitable combination of the foregoing. Computer program code intended to perform operations for aspects of the present configurations may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and the like, and traditional procedural programming languages ​​such as the C programming language or similar programming languages. The program code may execute entirely on the user's computer, partially on the user's computer as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be made by an external computer (e.g., through the Internet using an Internet Service Provider).

[0075] The terms "a" and "an," as used herein, are defined as one or more than one. The term "plurality," as used herein, is defined as two or more than two. The term "another," as used herein, is defined as at least a second or more. The terms "including" and / or "having," as used herein, are defined as comprising (i.e., open language). The phrase "at least one of ... and ...," as used herein, refers to and encompasses any and all possible combinations of one or more of the associated listed items. As an example, the phrase "at least one of A, B, and C" includes A only, B only, C only, and any combination thereof (e.g., AB, AC, BC, ABC).

[0076] Aspects of the present specification may be embodied in other forms without departing from the essential spirit or content thereof, and therefore, when indicating the scope of the claims, reference should be made to the following claims rather than the preceding specification.

Claims

1. 1. A method comprising: displaying, on a user interface, guidelines that specify a virtual trajectory for the vehicle, the guidelines being based on the steering angle input and the direction input; updating the shape of the guideline based on user input along the length of the guideline on the user interface to generate an updated guideline; sending a control signal to a vehicle control system that causes the vehicle to follow the updated guidelines; A method comprising:

2. 10. The method of claim 1, wherein the guideline destination is based on the range of a sensor located on the vehicle.

3. 2. The method of claim 1, wherein the directional input comprises: The forward direction and The rearward direction and A method that is one of the above.

4. 2. The method of claim 1, wherein the user interface comprises: the vehicle; A mobile device; The method is located at least one of.

5. 2. The method of claim 1, wherein the user input comprises: a curvature of the length of the guide line; and a destination point of the guideline; The method of claim 1, wherein at least one of the following is changed:

6. 10. The method of claim 1, further comprising transmitting a second control signal to the vehicle control system to slow the vehicle in response to detecting an obstacle along the updated guidelines. The method further comprises:

7. The method of claim 1 , wherein the user input generates another guideline, the other guideline including an origination point, the origination point along the length of the guideline.

8. 1. A system comprising: a processor; a memory storing machine-readable instructions; The machine-readable instructions, when executed by the processor, cause the processor to: displaying, on a user interface, guidelines that specify a virtual trajectory for the vehicle, the guidelines being based on the steering angle input and the direction input; updating the shape of the guideline based on user input along the length of the guideline on the user interface to generate an updated guideline; sending a control signal to a vehicle control system that causes the vehicle to follow the updated guidelines; A system that allows you to do this.

9. 10. The system of claim 8, wherein the guideline destination is based on the range of a sensor located on the vehicle.

10. 9. The system of claim 8, wherein the directional input comprises: The forward direction and The rearward direction and One of them is the system.

11. 9. The system of claim 8, wherein the user interface comprises: the vehicle; A mobile device; The system is located at least one of:

12. 9. The system of claim 8, wherein the user input comprises: a curvature of the length of the guide line; and a destination point of the guideline; The system modifies at least one of the following:

13. 9. The system of claim 8, wherein the memory further stores machine-readable instructions: The machine-readable instructions, when executed by the processor, cause the processor to: transmitting a second control signal to the vehicle control system to slow the vehicle in response to detecting an obstacle along the updated guidelines.

14. 9. The system of claim 8, wherein the user input generates another guideline, the other guideline including an origination point, the origination point along the length of the guideline.

15. A non-transitory computer-readable medium containing machine-readable instructions, comprising: The machine-readable instructions, when executed by a processor, cause the processor to: displaying, on a user interface, guidelines that specify a virtual trajectory for the vehicle, the guidelines being based on the steering angle input and the direction input; updating the shape of the guideline based on user input along the length of the guideline on the user interface to generate an updated guideline; sending a control signal to a vehicle control system that causes the vehicle to follow the updated guidelines; A non-transitory computer-readable medium for causing

16. 16. The non-transitory computer-readable medium of claim 15, wherein the guideline destination is based on the range of a sensor located on the vehicle.

17. 16. The non-transitory computer-readable medium of claim 15, wherein the directional input comprises: The forward direction and The rearward direction and a non-transitory computer-readable medium,

18. 16. The non-transitory computer-readable medium of claim 15, wherein the user interface comprises: the vehicle; A mobile device; a non-transitory computer-readable medium located on at least one of:

19. 16. The non-transitory computer-readable medium of claim 15, wherein the user input comprises: a curvature of the length of the guide line; and a destination point of the guideline; 10. A non-transitory computer-readable medium for modifying at least one of:

20. 16. The non-transitory computer-readable medium of claim 15, further comprising machine-readable instructions: A non-transitory computer-readable medium, the machine-readable instructions, when executed by the processor, causing the processor to send a second control signal to the vehicle control system that slows the vehicle in response to detecting an obstacle along the updated guidelines.

Images