Parking assistance with smooth handover, parking completion or parking correction

The AD system addresses the limitations of existing parking assistance by enabling smooth handover and completing or correcting parking maneuvers, providing a seamless and user-friendly parking experience.

JP7592961B2Active Publication Date: 2024-12-03ATIEVA INC(US)
View PDF 7 Cites 0 Cited by

Patent Information

Application Number
JP2023579634
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Priority Date
2021-07-20
Filing Date
2022-07-15
Publication Date
2024-12-03
Estimated Expiration
2042-07-15

AI Technical Summary

Technical Problem

Existing vehicle parking assistance systems fail to provide smooth handover from driver to system, assist in completing ongoing parking maneuvers, and modify completed parking maneuvers.

Method used

An assisted-driving (AD) system continuously monitors surroundings, detects available parking spots, plans a trajectory, and prompts the driver to hand over control at any point during the maneuver, allowing for seamless integration of system-assisted parking.

Benefits of technology

Enables convenient and fluid parking assistance by allowing drivers to hand over control to the system at any time, facilitating both parking completion and correction, enhancing user experience.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007592961000001
    Figure 0007592961000001
  • Figure 0007592961000002
    Figure 0007592961000002
  • Figure 0007592961000003
    Figure 0007592961000003
Patent Text Reader

Abstract

The computer-implemented method includes: a driver assistance (AD) system using sensors to continuously monitor the surroundings of a vehicle controlled by a driver; the AD system using the sensors to detect an available parking spot; in response to the AD system detecting the parking spot, planning a trajectory for the vehicle to park at the parking spot; and in response to the AD system detecting the parking spot, generating a prompt to the driver to have the AD system handle parking of the vehicle at the parking spot, the prompt being executed before the vehicle reaches the parking spot.
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] [CROSS REFERENCE TO RELATED APPLICATIONS] This application claims priority to U.S. patent application Ser. No. 17 / 443,067, filed July 20, 2021, entitled “PARKING ASSISTANCE WITH SMOOTH HANDOVER, PARKING COMPLETION, OR PARKING CORRECTION,” the disclosure of which is incorporated by reference in its entirety herein.

[0002] This document relates to parking assistance with smooth handover and / or parking completion and / or parking correction. [Background technology]

[0003] Some vehicles currently manufactured are equipped with one or more types of systems that may at least partially handle the operations associated with driving the vehicle. Attempts have been made to provide parking assistance. However, such systems do not provide a smooth handover from the driver to the system. As another example, such systems are not able to assist the driver in completing a parking maneuver that is in progress. As another example, such systems are not able to assist the driver in correcting a parking maneuver that has already been completed. Summary of the Invention

[0004] In a first aspect, a computer-implemented method includes: an Assisted-Driving (AD) system using sensors to continuously monitor the surroundings of a vehicle controlled by a driver; the AD system using sensors to detect an available parking spot; in response to the AD system detecting the parking spot, planning a trajectory for the vehicle to park at the parking spot; and in response to the AD system detecting the parking spot, generating a prompt to the driver to have the AD system handle parking of the vehicle at the parking spot, the prompt being executed before the vehicle reaches the parking spot.

[0005] Implementations may include any or all of the following features: When the AD system detects the parking spot, the parking spot is substantially parallel to the vehicle's travel path; When the AD system detects the parking spot, the parking spot is substantially perpendicular to the vehicle's travel path; When the AD system detects the parking spot, the parking spot is substantially oblique to the vehicle's travel path. The computer-implemented method further comprises: the AD system receiving a driver-generated input in response to a prompt, the input being generated before the vehicle enters the parking spot; and the AD system controlling the vehicle in response to the input to park the vehicle in the parking spot. The driver exits the vehicle before the AD system parks the vehicle in the parking spot. The driver begins to enter the vehicle into the parking spot without accepting a prompt by the AD system, and the method further comprises: continuing to generate a prompt to the driver during at least a portion of the vehicle maneuver in which the driver is parking the vehicle in the parking spot. Continuing to generate a prompt indicates that the AD system is available to complete parking the vehicle in the parking spot at any time. The computer-implemented method further comprises: the AD system receiving a driver-generated input in response to the AD system continuing to generate prompts, the input being generated before the vehicle comes to rest in the parking spot; and the AD system in response to the input controlling the vehicle to park the vehicle in the parking spot. The driver finishes parking the vehicle in the parking spot without accepting a prompt by the AD system, the vehicle having a first orientation within the parking spot, and the method further comprises: the AD system executing a motion plan to move the vehicle to a second orientation within the parking spot; the AD system receiving a user-generated input to modify the first orientation; and in response to the input, the AD system controlling the vehicle to assume the second orientation within the parking spot according to the motion plan.

[0006] In a second aspect, a computer-implemented method includes: a driver assistance (AD) system determining, using a sensor, that a vehicle controlled by a driver has begun to enter a parking spot; in response to the AD system determining that the vehicle has begun to enter the parking spot, generating a prompt to the driver to have the AD system handle parking of the vehicle in the parking spot; in response to the prompt, the AD system receiving input generated by the driver; and in response to the input, the AD system controlling the vehicle to finish parking the vehicle in the parking spot.

[0007] Implementations may include any or all of the following features: The prompt includes at least one of a visual message or an audio message; The input is generated using a touch screen of the vehicle; Generating the prompt indicates that the AD system is available to complete parking of the vehicle at the parking spot at any time; The driver exits the vehicle before the AD system controls the vehicle to finish parking the vehicle at the parking spot; Controlling the vehicle to finish parking the vehicle at the parking spot includes parking the vehicle at an orientation in the parking spot, the orientation being obtained using a machine learning component that analyzed previous instances in which the vehicle was parked.

[0008] In a third aspect, a computer-implemented method includes: a driving assistance (AD) system determining, using a sensor, that a vehicle is stationary in a first orientation relative to a parking spot; the AD system executing a motion plan to move the vehicle to a second orientation relative to the parking spot; in response to the AD system determining that the vehicle is stationary, generating a prompt to a user to cause the AD system to modify the first orientation; the AD system receiving input generated by the user in response to the prompt; and, in response to the input, the AD system controlling the vehicle to assume the second orientation relative to the parking spot in accordance with the motion plan.

[0009] Implementations may include any or all of the following features: The user is located outside the vehicle while generating the input. The user is located outside the vehicle while the AD system controls the vehicle to assume a second orientation within the parking spot. The vehicle is initially brought to the first orientation within the parking spot under the control of the driver, and the method further comprises: the AD system using the sensor to determine that the vehicle has begun to enter the parking spot and before determining that the vehicle is stationary; and in response to the AD system determining that the vehicle has begun to enter the parking spot, generating a prompt to the driver to have the AD system handle parking of the vehicle in the parking spot; the driver does not accept the prompt before parking the vehicle in the first orientation within the parking spot. The AD system obtains the second orientation using a machine learning component that has analyzed previous instances in which the vehicle has been parked. [Brief description of the drawings]

[0010] [Figure 1A] 1 shows an example of prompting a driver that a driver assistance (AD) system is available to handle parking of the vehicle. [Figure 1B] 1 shows an example of prompting a driver that a driver assistance (AD) system is available to handle parking of the vehicle. [Figure 1C] 1 shows an example of prompting a driver that a driver assistance (AD) system is available to handle parking of the vehicle.

[0011] [Diagram 2] 1 shows a flowchart of a method according to the example in FIGS. 1A to 1C.

[0012] [Figure 3A] An example is shown of prompting the driver that the AD system is available to complete the vehicle's ongoing parking maneuver. [Figure 3B]An example is shown of prompting the driver that the AD system is available to complete the vehicle's ongoing parking maneuver. [Figure 3C] An example is shown of prompting the driver that the AD system is available to complete the vehicle's ongoing parking maneuver. [Figure 3D] An example is shown of prompting the driver that the AD system is available to complete the vehicle's ongoing parking maneuver.

[0013] [Figure 4] 3A-3B and / or 3C-3D show flow charts of methods according to the examples in FIG.

[0014] [Figure 5A] An example of using the AD system to correct a completed parking of a vehicle is given. [Figure 5B] An example of using the AD system to correct a completed parking of a vehicle is given. [Figure 5C] An example of using the AD system to correct a completed parking of a vehicle is given. [Figure 5D] An example of using the AD system to correct a completed parking of a vehicle is given.

[0015] [Figure 6] 5A-5B and / or 5C-5D show flow charts of the methods according to the examples.

[0016] [Figure 7] An example of a system is shown.

[0017] [Figure 8] FIG. 1 shows an example sequence diagram for the availability of the AD system to complete an ongoing parking maneuver of a vehicle.

[0018] [Figure 9] FIG. 1 shows an example sequence diagram for a driver being outside a vehicle that is parked by an AD system.

[0019] [Figure 10] FIG. 1 shows an example sequence diagram for applying machine learning to an AD system that can be used to control vehicle parking.

[0020] [Figure 11] An example of a vehicle is shown.

[0021] [Figure 12A] An additional example regarding parking is given below. [Figure 12B] An additional example regarding parking is given below. [Figure 12C] An additional example regarding parking is given below.

[0022] [Figure 13] 1 illustrates an exemplary architecture of a computing device that may be used to implement aspects of the present disclosure.

[0023] Like reference numbers in the various drawings indicate like elements. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0024] This document describes examples of systems and techniques for providing parking assistance with smooth handover and / or parking completion and / or parking correction. The smooth handover from the driver to the system may be facilitated at least in part by the system's continuous monitoring of available parking spots and continuous execution of motion plans for nearby detected available parking spots. The system may suggest parking completion at any and all moments during the time the driver is manually performing the parking maneuver. The system may provide parking correction after the driver has manually completed the parking maneuver. The parking completion and / or parking correction may be performed while the driver / passenger is inside or outside the vehicle. The parking completion and / or parking correction may be performed based on previously detected manual parking maneuvers such that the vehicle placement matches the person's preferences. For example, machine learning may be used.

[0025] In some implementations, the subject matter described herein may operate in a manner that may be characterized as a "Parking Guardian." The purpose of the Parking Guardian may be to help the driver park the vehicle by taking over control from them at any given time while they are trying to manually park the vehicle. The purpose may be to make the automated parking process even more convenient and fluid by allowing the handover of control from the driver to the vehicle at any stage of the parking maneuver. The system may be used to modify the parking effort completed by the driver. The Parking Guardian may be constantly looking over the driver's shoulder to potentially assume control of the vehicle and complete the parking maneuver that the driver initiated. The subject matter described herein may continuously monitor the manual parking effort by the driver. The system may automatically detect that the driver is trying to park the car and may offer a complete handover to automatically complete the remainder of the parking. This facilitates that the driver may utilize the automated parking feature at any point in time, instead of the system manually asking the driver to select the desired parking space at the beginning of the parking effort.

[0026] In some implementations, the system is constantly scanning the environment and evaluating a path plan to assume control from the driver at any given point in the driver's manual parking effort. In some implementations, the system is flexible and may evaluate and create a trajectory at any given point in the driver's parking maneuver to complete the parking. That is, there may be no designated handover between the driver and the system. In some implementations, the system may complete a portion or the entire maneuver of the vehicle to the selected parking spot. For example, this may involve the use of machine learning techniques. In some implementations, the system may automatically evaluate the space immediately surrounding the vehicle at any given time for potential parking spots.

[0027] The subject matter described herein may provide the ability to constantly monitor the vehicle's surroundings (e.g., environment) and may have the flexibility to assume control (e.g., be handed over control by the driver) at any and all points during a manual parking maneuver. The subject matter described herein may provide the ability to detect available parking spots without having the vehicle pass the spots. The subject matter described herein may provide the ability to complete a parking maneuver entirely or partially into a particular spot. For example, machine learning may be used. The subject matter described herein may provide the ability to evaluate and create trajectories for vehicle movement at any and all points of the parking maneuver to complete parking of the vehicle. For example, there may not be a specific handover phase defined between the driver and the system. The subject matter described herein may provide the ability to automatically evaluate the space immediately surrounding the vehicle at any given time during a manual parking maneuver for potential parking spots. The subject matter described herein may provide the ability to let the driver control the automated parking maneuver by a remote device after the driver hands over control to the system. For example, this may occur during or after the manual parking maneuver. The subject matter described herein may provide the ability to automatically park a vehicle in a selected parking spot using machine learning such that the vehicle has an orientation and / or position learned from the vehicle's and / or the driver's previous parking maneuvers.

[0028] The subject matter described herein may provide one or more advantages over prior approaches. The automatic parking function may be advanced to be driver-friendly. The function design may eliminate any predefined conditions for vehicle control handover between the driver and the system. In contrast, in current automatic parking assistance functions, the driver may need to drive to the side of the spot and then hand over control to the vehicle. Similarly, in at least one existing approach to automatic parking assistance functions, the driver cannot use the automatic parking function if the driver has already manually initiated the parking effort. The proposed invention is also available to continuously monitor the driver's parking efforts and at any point take over control from the driver like a guardian to complete the remainder of the parking. The invention involves techniques including, but not limited to, machine learning, sensor fusion, path planning, trajectory generation, and remote activation of parking using a mobile app or remote key.

[0029] The examples herein refer to vehicles. A vehicle is a machine that transports passengers or cargo, or both. A vehicle can have one or more motors that use at least one type of fuel, or other energy source (e.g., electricity). Examples of vehicles include, but are not limited to, cars, trucks, and buses. The number of wheels may vary between vehicle types, and one or more (e.g., all) of the wheels may be used for propulsion of the vehicle. A vehicle may include a passenger compartment that accommodates one or more people. At least one vehicle occupant may be considered a driver; in this case, various tools, implements, or other devices may be provided to the driver. In the examples herein, any person carried by a vehicle may be referred to as a "driver" or "passenger" of the vehicle, regardless of whether that person is driving the vehicle, whether that person has access to the controls to drive the vehicle, or whether that person lacks the controls to drive the vehicle. The vehicles in this example are illustrated as being similar or identical to each other for illustrative purposes only.

[0030] Examples herein refer to driving assistance. In some implementations, driving assistance may be performed by a driving assistance (AD) system. For example, an AD system may include an Advanced Driving-Assistance System (ADAS). Driving assistance involves at least partially automating one or more dynamic driving tasks. An ADAS may perform driving assistance and is an example of a driving assistance system. Driving assistance is typically performed in part based on the output of one or more sensors located on, under, or within the vehicle. The AD system may plan one or more trajectories for the vehicle before and / or while controlling the motion of the vehicle. The planned trajectory may define a path for the travel of the vehicle. Thus, propelling the vehicle according to the planned trajectory may correspond to controlling one or more aspects of the vehicle's motion behavior, such as, but not limited to, the vehicle's steering angle, gear (e.g., forward or reverse), speed, acceleration, and / or braking.

[0031] Although an autonomous vehicle is an example of a system that performs driver assistance, not all driver assistance systems are designed to provide a fully autonomous vehicle. Several levels of driving automation have been defined by SAE International, commonly referred to as levels 0, 1, 2, 3, 4, and 5, respectively. For example, a level 0 system or driving mode may not involve persistent vehicle control by the system. For example, a level 1 system or driving mode may include adaptive cruise control, emergency brake assist, automatic emergency brake assist, lane keeping, and / or lane centering. For example, a level 2 system or driving mode may include highway assist, autonomous obstacle avoidance, and / or autonomous parking. For example, a level 3 or 4 system or driving mode may include incremental control of the vehicle by the driver assistance system. For example, a level 5 system or driving mode may not require human intervention in the driver assistance system.

[0032] Examples herein refer to sensors. A sensor is configured to detect an event and / or a change in one or more aspects of its environment and output a signal reflecting the detection. As merely illustrative examples, a sensor may indicate one or more of the distance between a vehicle and an object, the speed of the vehicle, the trajectory of the vehicle, or the acceleration of the vehicle. A sensor may generate an output without probing the surroundings with anything (e.g., such as an image sensor capturing electromagnetic radiation), or a sensor may probe the surroundings (e.g., with electromagnetic radiation and / or sound waves) and detect a response to the probe. Examples of sensors that may be used in one or more embodiments include, but are not limited to, optical sensors (e.g., cameras); scanning systems (e.g., lidar); radio frequency based sensors (e.g., radar); acoustic sensors (e.g., ultrasonic devices and / or microphones); inertial measurement units (e.g., gyroscopes and / or accelerometers); speed sensors (e.g., for the vehicle or its components); location sensors (e.g., for the vehicle or its components); orientation sensors (e.g., for the vehicle or its components); torque sensors; temperature sensors (e.g., primary or secondary thermometers); pressure sensors (e.g., for the ambient air or vehicle components); humidity sensors (e.g., rain detectors); or occupancy sensors.

[0033] Examples herein refer to parking spots. As used herein, a parking spot includes any location that may be identified as a place for parking a vehicle, regardless of the duration of the stay. A parking spot may be, but is not necessarily, defined by one or more signs or structures on the surface on which the vehicle is located. A parking spot may be located on public or private land. A parking spot may be located on developed or undeveloped land. A parking spot may be located at least partially inside a structure or may be located away from any structure. A parking spot may be located inside or outside a building. A parking spot may be located adjacent to a roadway (e.g., a street, avenue, or road) or may be located on a lot away from at least one roadway. When a parking spot is located adjacent to a roadway, the roadway may have one or more lanes. When a parking spot is located adjacent to a roadway, the roadway may be restricted to one-way travel (e.g., a one-way street) or may allow two-way traffic. A parking spot may be subject to parking regulations by government and / or private entities. A parking spot may be subject to monitoring by personnel and / or automated devices, or a parking spot may be unmonitored. A parking spot may be permanent or temporary. As used herein, a parking stall is a location that includes two or more parking spots.

[0034] Examples herein refer to continuously monitoring the surroundings of the vehicle using sensors to detect at least one parking spot that is available. As used herein, continuously monitoring the surroundings includes receiving a sensor signal that indicates that the parking spot is not currently obstructed (e.g., not occupied by another vehicle). For example, signals from the vehicle's camera and / or lidar may be used to visually detect parking spots for the vehicle. When the vehicle is moving, the continuous monitoring may continuously evaluate new surroundings as they come within the operating range of the sensor. If more than one available parking spot is detected, the system may make all of them available to the driver as potential parking spots, or the system may make a selection and show the driver fewer (e.g., one or more) of the detected parking spots. The system may consider one or more circumstances in detecting that a parking spot is available. For example, the system may assess whether the vehicle can reach the parking spot or whether there is no navigable path to the parking spot. As another example, the system may take into account one or more dimensions of the vehicle, including but not limited to length, width, height, and / or ground clearance. As another example, the system may take into account one or more parking regulations that may apply. For example, if a space that would otherwise accommodate the vehicle is labeled "No Parking" (by a sign at the location or by information otherwise accessible to the system in an electronic record), the system may not indicate that the parking spot is available.

[0035] Examples herein refer to generating a prompt. As used herein, a prompt may be generated in any manner that is noticeable and understandable to the driver. In some implementations, a prompt may be generated by presenting one or more visual messages on a display device. For example, a display device may be provided with touch screen capabilities to allow the driver to make one or more types of inputs with the device. In some implementations, a prompt may be generated by playing one or more audio messages over a speaker. In some implementations, a prompt may be generated by triggering one or more forms of haptic and / or tactile output to the driver.

[0036] Examples herein refer to a driver or passenger generating an input in response to a prompt from the system. As used herein, an input may be generated using any form of technology available in a vehicle. In some implementations, an input may be generated by activating a touch screen. In some implementations, an input may be implemented by actuating a switch (e.g., by pressing a button and / or activating a turn signal). In some implementations, an input may be detected by audio (e.g., by a microphone that detects audible speech that is interpreted as a voice command).

[0037] Examples herein refer to machine learning. As used herein, using machine learning may include employing artificial intelligence that allows a machine, such as a driver assistance system, to perceive its environment and take action to achieve one or more goals. A machine learning algorithm may apply one or more principles of data mining to derive a driving envelope from data collected about a vehicle and its associated circumstances. A machine learning algorithm may train in one or more perspectives. For example, supervised, semi-supervised, and / or unsupervised training may be performed. In some implementations, a machine learning algorithm may use one or more classifiers. For example, a classifier may assign one or more labels to instances recognized in the processed data. In some implementations, a machine learning algorithm may use one or more forms of regression analysis. For example, a machine learning algorithm may apply regression to determine one or more numerical values. In some implementations, a machine learning algorithm may be configured to collect and store data, use the data to detect events, use the data to identify contexts, and generate a driving envelope based at least in part on the detected events and contexts.

[0038] 1A-1C show examples of prompting a driver that a driver assistance (AD) system is available to handle parking of vehicle 100. Some or all of the examples for vehicle 100 may be used in conjunction with one or more other examples described elsewhere herein.

[0039] An example will be described with reference to a parking bay 102, which may have one or more entrances or exits (not shown) for vehicles. Parking spots in the parking bay 102 may be arranged in one or more rows. Row 104A includes parking spots defined by a (e.g., painted or structural) divider 106 such that vehicles are positioned essentially perpendicular to the direction of travel along row 104A. One or more rows in the parking bay 102 may have parking spots that are not separated from one another by any divider, where row 104B includes parking spots such that vehicles are positioned essentially parallel to the direction of travel along row 104B. Other approaches may be used.

[0040] Vehicle 100 is traveling within parking stall 102, as indicated diagrammatically by arrow 108. Vehicle 100 is traveling in a lane formed between rows 104A-104B. Vehicle 100 is currently essentially controlled by the driver. For example, the driver is currently searching for an available parking spot within parking stall 102 and maneuvering vehicle 100 using the steering wheel, gear selector, accelerator, and brake pedals.

[0041] The AD system of the vehicle 100 continuously monitors the surroundings of the vehicle 100 using at least one sensor to detect whether any parking spots are available. The use of a sensor is illustrated here diagrammatically by a signal 110 near the vehicle 100, but the sensor does not necessarily generate any radiation or other emissions to probe the surroundings. In the example shown in FIG. 1A, the vehicle 100 has not yet detected any available parking spots in the parking bay 102.

[0042] 1B illustrates an example from the example described above in which vehicle 100 has continued to drive into parking stall 102. Continued monitoring of vehicle 100's surroundings has now resulted in the detection of parking spot 112. Parking spot 112 is located in row 104B. Parking spot 112 is now detected before vehicle 100 arrives at parking spot 112. For example, vehicle 100 does not need to pull up beside or begin to enter parking spot 112 for detection to occur. Rather, the system detects parking spot 112 when sensors generate a signal that positively indicates that parking spot 112 is suitable for vehicle 100 and is not currently occupied.

[0043] As mentioned above, at this point in the example, vehicle 100 may be controlled only by the driver. Nevertheless, for any parking spot that the AD system detects, the AD system may plan a trajectory for vehicle 100. This may ensure that the AD system may provide a fluid handover from the driver at any time if the driver so chooses. Here, for example, if the driver wishes to park vehicle 100 in parking spot 112, the AD system plans a trajectory for vehicle 100 to enter there.

[0044] The AD system may generate prompts for the driver regarding one or more detected parking spots. Here, a user interface (UI) 114 of the vehicle 100 is illustrated in a schematic manner. The UI 114 is an example of a human-machine interface (HMI) that the vehicle 100 may use to communicate with the driver or another passenger. In some implementations, the UI 114 may include a graphical UI (GUI). For example, the GUI may be generated using a display device with touch screen capabilities. Here, the UI 114 makes a control 116 available to the driver. For example, the control 116 may be a GUI button labeled "Park" that suggests to the driver that the AD system handle the parking of the vehicle 100 in the parking spot 112. In some implementations, the UI 114 may indicate the parking spot 112 in which the AD system is currently suggesting to the driver to park the vehicle 100. For example, the UI 114 may present a map of at least a portion of the parking stall 102 with the parking spot 112 indicated; or the UI 114 may show the parking spot 112 through Augmented Reality (AR) technology applied to a captured image of the surroundings of the vehicle 100. Other approaches may be used.

[0045] The driver may accept the prompt by activating control 116. Control 116 may be available to the driver before vehicle 100 arrives at parking spot 112. For example, vehicle 100 does not need to pull in beside or begin to enter parking spot 112 for control 116 to be active. If the driver accepts the prompt by activating control 116, the AD system may take over control of some or all of vehicle 100 and steer vehicle 100 into parking spot 112. This may involve the AD system receiving inputs generated by the driver using control 116 in response to the prompt, such inputs being generated before the vehicle enters parking spot 112. The AD system may then control the vehicle in response to the inputs to park the vehicle in parking spot 112. For example, the AD system may perform this maneuver using a trajectory for vehicle 100 that the AD system generates upon detecting parking spot 112. This functionality may ensure that convenient parking assistance is provided to the driver, both to identify available parking spots and to take over steering the vehicle 100 to an available parking spot if the driver so desires at any point during the operation. To name just a few examples, the driver may actuate controls 116 using the GUI of the vehicle 100, the GUI of the portable electronic device, by giving audio commands (e.g., using the audio interface 1132 in FIG. 11 or the portable electronic device) and / or by making gestures (e.g., using the visual interface 1134 in FIG. 11 or the portable electronic device).

[0046] If there is a handover to the AD system, the driver can exit the vehicle 100 before or after control is granted to the AD system, or can remain in the vehicle 100 during the maneuver. For example, after the control 116 is available, the driver can stop the vehicle 100, exit the vehicle to the parking stall 102, and activate the control 116 using a remote control device (e.g., a portable electronic device such as a smartphone having an app that presents the control 116 for activation). As another example, after stopping the vehicle 100, the driver can open the door of the vehicle 100 and then activate the control 116 using a GUI inside the vehicle 100. The AD system can receive the driver-generated input and may wait to perform the parking maneuver until it detects that the door of the vehicle 100 is closed (at which point the driver may have exited the vehicle 100). Other approaches can be used.

[0047] Here, the driver does not accept the prompt generated by the AD system. For example, the driver may prefer to park somewhere else within the parking stall 102, such as closer to where the driver is headed and / or in a parking spot that does not involve parallel parking. The driver therefore does not actuate control 116 for the parking spot 112, but rather continues to operate the vehicle 100 in the parking stall 102.

[0048] 1C illustrates an example from the example described above in which vehicle 100 has continued to drive into parking stall 102. Continued monitoring of vehicle 100's surroundings has now resulted in the detection of parking spot 118. Parking spot 118 is located in row 104A. Parking spot 118 is now detected before vehicle 100 arrives at parking spot 118. For example, vehicle 100 does not need to pull up beside or begin to enter parking spot 118 for detection to occur. Rather, the system detects parking spot 118 when sensors generate a signal that positively indicates that parking spot 118 is suitable for vehicle 100 and is not currently occupied.

[0049] As mentioned above, at this point in the example, vehicle 100 may be controlled only by the driver. Nevertheless, for any parking spot that the AD system detects, the AD system may plan a trajectory for vehicle 100. This may ensure that the AD system may provide a fluid handover from the driver at any time if the driver so chooses. Here, for example, if the driver wishes to park vehicle 100 in parking spot 118, the AD system plans a trajectory for vehicle 100 to enter there.

[0050] The AD system may generate a prompt for the driver regarding one or more detected parking spots. Here, the UI 114 of the vehicle 100 is illustrated in a schematic manner. Here, the UI 114 makes a control 116' available to the driver. For example, the control 116' may be a GUI button labeled "PARK" that suggests to the driver to let the AD system handle parking of the vehicle 100 in the parking spot 118. In some implementations, the UI 114 may show the parking spot 118 in which the AD system is currently suggesting to the driver to park the vehicle 100. For example, the UI 114 may present a map of at least a portion of the parking bay 102 in which the parking spot 118 is shown; or the UI 114 may show the parking spot 118 by AR technology applied to a captured image of the surroundings of the vehicle 100. Other approaches may be used.

[0051] The driver may accept the prompt by activating control 116'. Control 116' may be available to the driver before vehicle 100 arrives at parking spot 118. For example, vehicle 100 does not need to pull in beside or begin to enter parking spot 118 for control 116' to be active. If the driver accepts the prompt by activating control 116', the AD system may take over control of some or all of vehicle 100 and steer vehicle 100 into parking spot 118. This may involve the AD system receiving inputs generated by the driver using control 116' in response to the prompt, such inputs being generated before the vehicle enters parking spot 118. The AD system may then control the vehicle in response to the inputs to park the vehicle in parking spot 118. For example, the AD system may perform this maneuver using a trajectory for vehicle 100 that the AD system generates upon detecting parking spot 118. This functionality may ensure that convenient parking assistance is provided to the driver, both to identify available parking spots and to take over steering the vehicle 100 to an available parking spot if the driver so desires at any point during the operation. To name just a few examples, the driver may actuate controls 116' using the GUI of the vehicle 100, the GUI of the portable electronic device, by giving audio commands (e.g., using the audio interface 1132 in FIG. 11 or the portable electronic device) and / or by making gestures (e.g., using the visual interface 1134 in FIG. 11 or the portable electronic device).

[0052] If there is a handover to the AD system, the driver can exit the vehicle 100 before or after control is granted to the AD system, or can remain in the vehicle 100 during the maneuver. For example, after the control 116' is available, the driver can stop the vehicle 100, exit the vehicle to the parking stall 102, and activate the control 116' using a remote control device (e.g., a portable electronic device such as a smartphone having an app that presents the control 116' for activation). As another example, after stopping the vehicle 100, the driver can open the door of the vehicle 100 and then activate the control 116' using a GUI inside the vehicle 100. The AD system can receive the driver-generated input and can wait to perform the parking maneuver until it detects that the door of the vehicle 100 is closed (at which point the driver may have exited the vehicle 100). Other approaches can be used.

[0053] 2 shows a flow chart of a method 200 for the example in FIGS. 1A-1C. Method 200 may be used with one or more other examples described elsewhere herein. More or fewer operations may be performed than those shown. Two or more operations may be performed in a different order unless otherwise indicated.

[0054] In operation 202, continuous monitoring of the vehicle's surroundings using sensors may be performed by the AD system. The vehicle may currently be controlled by a driver. For example, and referring briefly back to FIGS. 1A-1C, some or all of the parking stall 102 may be continuously monitored by the AD system of the vehicle 100.

[0055] The AD system may detect available parking spots in operation 204. In Figures 1A-1C, for example, parking spots 112 and / or 118 may be detected.

[0056] In operation 206, the AD system may plan a trajectory for the vehicle to park at the parking spot. For example, such a trajectory may be defined as the path the vehicle will travel as a result of sequential control of the vehicle's steering system, transmission, accelerator pedal, and / or brake pedal by the AD system.

[0057] In operation 208, the AD system may generate a prompt to the driver to have the AD system handle parking the vehicle in the parking spot. The prompt may be generated in response to detecting the parking spot or may be generated before the vehicle reaches the parking spot. In FIG. 1A-1C, for example, the AD system may provide controls 116 and / or 116′ to the driver.

[0058] At operation 210, the AD system may receive driver-generated input in response to the prompt. The input may be generated before the vehicle enters the parking spot. In FIGS. 1A-1C, for example, the driver may actuate control 116 and / or 116′.

[0059] At operation 212, the AD system may control the vehicle to park the vehicle in a parking spot in response to the input received at operation 210. In Figures 1A-1C, for example, the AD system may perform a maneuver to park vehicle 100 in parking spot 112 or 118 after control is handed over by the driver.

[0060] 3A-3D show examples of prompting the driver that the AD system is available to complete an ongoing parking maneuver of the vehicle 300. Some or all of the examples for the vehicle 300 may be used in conjunction with one or more other examples described elsewhere herein.

[0061] Examples will be described with reference to parking stall 302, which may have one or more entrances or exits (not shown) for vehicles. Parking spots within parking stall 302 may be arranged in one or more rows. One or more rows in parking stall 302 may have parking spots that are separated from one another by a (e.g., painted or structural) divider. One or more rows in parking stall 302 may have parking spots that are not separated from one another by any divider, where row 304 includes parking spots where vehicles are positioned essentially parallel to the direction of travel along row 304. Other approaches may be used.

[0062] The AD system of the vehicle 300 continuously monitors the surroundings of the vehicle 300 using at least one sensor to detect whether any parking spots are available. The use of sensors is illustrated here diagrammatically by a signal 306 near the vehicle 300, but the sensor does not necessarily generate any radiation or other emissions to probe the surroundings. In the example shown in FIG. 3A, the vehicle 300 has detected a parking spot 308 in a parking bay 302. The parking spot 308 may have been detected before the vehicle 300 arrived at the parking spot 308. For example, it is not necessary for the vehicle 300 to pull up beside or begin to enter the parking spot 308 for detection to occur. Rather, the system detects the parking spot 308 when the sensor generates a signal that reliably indicates that the parking spot 308 is suitable for the vehicle 300 and is not currently occupied.

[0063] For every parking spot it detects, the AD system can plan a trajectory for the vehicle 300. This can ensure that the AD system can provide a fluid handover from the driver at any time if the driver so chooses. Here, for example, if the driver wants to park the vehicle 300 in the parking spot 308, the AD system plans a trajectory for the vehicle 300 to enter there.

[0064] Vehicle 300 is currently essentially controlled by the driver. For example, the driver is currently steering vehicle 300 using the steering wheel, gear selector, accelerator, and brake pedals. Vehicle 300 is maneuvering into parking spot 308, as indicated diagrammatically by arrow 310. The driver is currently steering vehicle 300 in a manner that indicates to the AD system that the driver intends to park in parking spot 308. For example, the driver has begun to drive vehicle 300 into parking spot 308.

[0065] The AD system may generate prompts for the driver regarding one or more detected parking spots. Here, a UI 312 of the vehicle 300 is illustrated in a schematic manner. The UI 312 is an example of an HMI that the vehicle 300 may use to communicate with the driver or another passenger. In some implementations, the UI 312 may include a GUI. For example, the GUI may be generated using a display device with touch screen capabilities. Here, the UI 312 makes a control 314 available to the driver. For example, the control 314 may be a GUI button labeled "Complete My Parking" that suggests to the driver that the AD system completes parking the vehicle 300 in the parking spot 308. In some implementations, the control 314 may be provided in conjunction with ceasing to provide the control 116 or 116' (FIGS. 1B-1C). For example, a seamless transition may be provided between the AD system offering to assist if the driver wishes to park the vehicle 300 and the AD system instead offering to complete an ongoing parking maneuver that the driver has initiated. In some implementations, the UI 312 may indicate the parking spot 308 in which the AD system is currently suggesting the driver to complete parking the vehicle 300. For example, the UI 312 may present a map of at least a portion of the parking stall 302 in which the parking spot 308 is indicated; or the UI 312 may indicate the parking spot 308 through AR technology applied to a captured image of the surroundings of the vehicle 300. Other approaches may be used.

[0066] The AD system may continue to generate prompts during some or all of the time the driver is maneuvering the vehicle 300 into the parking spot 308. For example, continuing to generate prompts indicates that the AD system is available to take over and complete parking of the vehicle at the parking spot 308 at any time. That is, a specific handover point may not be specified, and instead the driver has the opportunity to request assistance from the AD system at any time before manually completing the parking maneuver. An example described below will show that assistance from the AD system may be available even after the driver has manually parked the vehicle 300.

[0067] That is, in response to the AD system determining that the driver has begun to enter the parking spot 308, the control 314 may be made available to the driver. The driver may accept a prompt by activating the control 314. If the driver accepts the prompt, the AD system may take over control of some or all of the vehicle 300 and complete the maneuver of the vehicle 300 into the parking spot 308. This may involve the AD system receiving inputs that the driver generates using the control 314 in response to the prompt. The AD system may then control the vehicle in response to the inputs to complete the parking of the vehicle 300 in the parking spot 308. For example, the AD system may perform this maneuver using a trajectory for the vehicle 300 that the AD system generates upon detecting the parking spot 308. This feature may ensure that the driver is provided with convenient parking assistance to take over the completion of the maneuver of the vehicle 300 into the parking spot 308 if the driver so desires at any point during the parking operation. To name just a few examples, the driver can activate controls 314 using the GUI of the vehicle 300, the GUI of the portable electronic device, by giving audio commands (e.g., using the audio interface 1132 in FIG. 11 or a portable electronic device) and / or by making gestures (e.g., using the visual interface 1134 in FIG. 11 or a portable electronic device).

[0068] If there is a handover to the AD system, the driver can exit the vehicle 300 before or after control is granted to the AD system, or can remain in the vehicle during the maneuver. For example, after control 314 is available, the driver can stop the vehicle 300, exit the vehicle to the parking stall 302, and activate the control 314 using a remote control device (e.g., a portable electronic device such as a smartphone having an app that presents the control 314 for activation). As another example, after stopping the vehicle 300, the driver can open the door of the vehicle 300 and then activate the control 314 using a GUI inside the vehicle 300. The AD system can receive the driver-generated input and may wait to complete execution of the parking maneuver until it is detected that the door of the vehicle 300 is closed (at which point the driver may have exited the vehicle 300). Other approaches can be used.

[0069] Here, the driver accepts the prompt generated by the AD system by actuating control 314. FIG. 3B shows the parking bay 302 after the AD system has finished executing its parking maneuver completion. Path 316 illustrates diagrammatically that execution of the AD system may include one or more actions by the vehicle 300 to follow the planned trajectory. For example, if the vehicle 300 is currently oriented nose-first into the parking spot 308 and the AD system is asked to take over control, path 316 may involve first driving the vehicle 300 backward, then driving the vehicle 300 forward, and finally driving the vehicle 300 backward again to enter the parking spot 308. Other approaches may be used.

[0070] The planned trajectory, according to which the AD system controls the vehicle 300 to follow the path 316, may be defined based on the current position and orientation of the vehicle 300 at the moment of planning, as well as one or more characteristics of travel during the parking maneuver and / or the intended resulting position and orientation of the vehicle 300. For example, the trajectory may be planned such that, when parking is completed, the orientation of the vehicle 300 in the parking spot 308 will match the orientation of one or more other vehicles (e.g., the closest vehicles) in the row 304. As another example, the trajectory may be planned such that, when parking is completed, the position of the vehicle 300 in the parking spot 308 will have at least a certain amount of space in front of and / or behind the vehicle 300 (e.g., a substantially equal amount of space in front of and behind the vehicle 300). As another example, the trajectory may be planned such that, when parking is completed, the position of the vehicle 300 in the parking spot 308 will have at least a certain amount of space to the left and / or right of the vehicle 300 (e.g., a substantially equal amount of space to the right and left of the vehicle 300). As another example, a trajectory may be planned such that, upon completion of parking, the position of the vehicle 300 in the parking spot 308 has a maximum predefined distance from the curb adjacent the parking spot 308. In some implementations, machine learning techniques may be used. For example, the AD system may plan a trajectory such that, upon coming to a stop, the vehicle 300 has a position and / or orientation that is consistent with a previously performed parking arrangement of the vehicle 300 and / or otherwise associated with the driver.

[0071] 3C-3D, examples will be described with reference to a parking bay 318 that may have one or more entrances or exits (not shown) for vehicles. Parking spots within the parking bay 318 may be arranged in one or more rows. One or more rows in the parking bay 318 may have parking spots that are separated from one another by a (e.g., painted or structural) divider 320. One or more rows in the parking bay 318 may have parking spots that are not separated from one another by any divider, where the row 322 includes parking spots that are positioned essentially perpendicular to the direction of travel of vehicles along the row 322. Other approaches may be used.

[0072] The AD system of the vehicle 300 continuously monitors the surroundings of the vehicle 300 using at least one sensor to detect whether any parking spots are available. The use of the sensor is illustrated here diagrammatically by a signal 306 near the vehicle 300, but the sensor does not necessarily generate any radiation or other emissions to probe the surroundings. In the example shown in FIG. 3C, the vehicle 300 has detected a parking spot 324 in a parking bay 318. The parking spot 324 may have been detected before the vehicle 300 arrived at the parking spot 324. For example, it is not necessary for the vehicle 300 to pull up beside or begin to enter the parking spot 324 for detection to occur. Rather, the system detects the parking spot 324 when the sensor generates a signal that reliably indicates that the parking spot 324 is suitable for the vehicle 300 and is not currently occupied.

[0073] For every parking spot it detects, the AD system can plan a trajectory for the vehicle 300. This can ensure that the AD system can provide a fluid handover from the driver at any time if the driver so chooses. Here, for example, if the driver wants to park the vehicle 300 in the parking spot 324, the AD system plans a trajectory for the vehicle 300 to enter there.

[0074] Vehicle 300 is currently essentially controlled by the driver. For example, the driver is currently steering vehicle 300 using the steering wheel, gear selector, accelerator, and brake pedals. Vehicle 300 is maneuvering to enter parking spot 324, as indicated diagrammatically by arrow 310. The driver is currently steering vehicle 300 in a manner that indicates to the AD system that the driver intends to park in parking spot 324. For example, the driver has begun to drive vehicle 300 into parking spot 324.

[0075] The AD system may generate prompts for the driver regarding one or more detected parking spots. Here, the UI 312 of the vehicle 300 is illustrated in a schematic manner. Here, the UI 312 makes available to the driver a control 314'. For example, the control 314' may be a GUI button labeled "Complete My Parking" that suggests to the driver that the AD system completes parking the vehicle 300 in the parking spot 324. In some implementations, the control 314' may be provided in conjunction with ceasing to provide the control 116 or 116' (FIGS. 1B-1C). For example, a seamless transition may be provided between the AD system offering to assist if the driver wishes to park the vehicle 300 and the AD system instead offering to complete an ongoing parking maneuver that the driver has initiated. In some implementations, the UI 312 may indicate the parking spot 324 in which the AD system is currently suggesting to the driver to complete parking the vehicle 300. For example, the UI 312 may present a map of at least a portion of the parking stall 318 with the parking spot 324 indicated; or the UI 312 may indicate the parking spot 324 through AR technology applied to a captured image of the surroundings of the vehicle 300. Other approaches may be used.

[0076] The AD system may continue to generate prompts during some or all of the time the driver is maneuvering the vehicle 300 into the parking spot 324. For example, continuing to generate prompts indicates that the AD system is available to take over and complete parking of the vehicle at the parking spot 324 at any time. That is, a specific handover point may not be specified, and instead the driver has the opportunity to request assistance from the AD system at any time before manually completing the parking maneuver. An example described below will show that assistance from the AD system may be available even after the driver has manually parked the vehicle 300.

[0077] That is, in response to the AD system determining that the driver has begun to enter the parking spot 324, a control 314' may be made available to the driver. The driver may accept a prompt by activating the control 314'. If the driver accepts the prompt, the AD system may take over control of some or all of the vehicle 300 and complete the maneuver of the vehicle 300 into the parking spot 324. This may involve the AD system receiving inputs that the driver generates using the control 314' in response to the prompt. The AD system may then control the vehicle in response to the inputs to complete the parking of the vehicle 300 in the parking spot 324. For example, the AD system may perform this maneuver using a trajectory for the vehicle 300 that the AD system generates upon detecting the parking spot 324. This feature may ensure that the driver is provided with convenient parking assistance to take over the completion of the maneuver of the vehicle 300 into the parking spot 324 if the driver so desires at any point during the parking operation. To name just a few examples, the driver can activate controls 314' using the GUI of the vehicle 300, the GUI of the portable electronic device, by giving audio commands (e.g., using the audio interface 1132 in FIG. 11 or a portable electronic device) and / or by making gestures (e.g., using the visual interface 1134 in FIG. 11 or a portable electronic device).

[0078] If there is a handover to the AD system, the driver can exit the vehicle 300 before or after control is granted to the AD system, or can remain in the vehicle during the maneuver. For example, after the control 314' is available, the driver can stop the vehicle 300, exit the vehicle to the parking stall 318, and activate the control 314' using a remote control device (e.g., a portable electronic device such as a smartphone having an app that presents the control 314' for activation). As another example, after stopping the vehicle 300, the driver can open the door of the vehicle 300 and then activate the control 314' using a GUI inside the vehicle 300. The AD system can receive the driver-generated input and can wait to complete the execution of the parking maneuver until it is detected that the door of the vehicle 300 is closed (at which point the driver may have exited the vehicle 300). Other approaches can be used.

[0079] Here, the driver accepts the prompt generated by the AD system by actuating control 314'. FIG. 3D shows parking space 318 after the AD system has finished executing its parking maneuver completion. Path 326 illustrates diagrammatically that execution of the AD system may include one or more actions by vehicle 300 to follow the planned trajectory. For example, if vehicle 300 is currently oriented nose-first into parking spot 324 and the AD system is asked to take over control, path 316 may involve first driving vehicle 300 backward and then driving vehicle 300 forward into parking spot 324. Other approaches may be used.

[0080] The planned trajectory, according to which the AD system controls the vehicle 300 to follow the path 326, may be defined based on the current position and orientation of the vehicle 300 at the planning moment, as well as one or more characteristics of the vehicle's travel during the parking maneuver and / or the intended resulting position and orientation of the vehicle 300. For example, the trajectory may be planned such that, when parking is completed, the orientation of the vehicle 300 in the parking spot 324 will match the orientation of one or more other vehicles (e.g., the closest vehicles) in the row 322. As another example, the trajectory may be planned such that, when parking is completed, the position of the vehicle 300 in the parking spot 324 will have at least a certain amount of space in front of and / or behind the vehicle 300 (e.g., a substantially equal amount of space in front of and behind the vehicle 300). As another example, the trajectory may be planned such that, when parking is completed, the position of the vehicle 300 in the parking spot 324 will have at least a certain amount of space to the left and / or right of the vehicle 300 (e.g., a substantially equal amount of space to the right and left of the vehicle 300). As another example, a trajectory may be planned such that, upon completion of parking, the position of the vehicle 300 in the parking spot 324 has a maximum predefined distance from the curb adjacent the parking spot 324. In some implementations, machine learning techniques may be used. For example, the AD system may plan a trajectory such that, upon coming to a stop, the vehicle 300 has a position and / or orientation that is consistent with a previously performed parking arrangement of the vehicle 300 and / or otherwise associated with the driver.

[0081] 4 shows a flow chart of a method 400 for the examples in FIGS. 3A-3B and / or 3C-3D. The method 400 may be used with one or more other examples described elsewhere herein. More or fewer operations may be performed than those shown. Two or more operations may be performed in a different order unless otherwise indicated.

[0082] At operation 402, the AD system may use sensors to determine that a driver-controlled vehicle has begun to enter a parking spot. For example, and referring briefly back to Figures 3A-3D, it may be determined that vehicle 300 has begun to enter parking spots 308 and / or 324.

[0083] In operation 404, the AD system may generate a prompt to the driver to have the AD system handle parking the vehicle in the parking spot. The prompt may be generated in response to determining that the vehicle has begun to enter the parking spot. In Figures 3A-3D, for example, the AD system may provide controls 314 and / or 314' to the driver.

[0084] The AD system may receive driver-generated input in response to the prompt at operation 406. In Figures 3A-3D, for example, the driver may actuate control 314 and / or 314'.

[0085] At operation 408, the AD system may control the vehicle to park the vehicle in the parking spot in response to the input received at operation 406. In Figures 3A-3D, for example, the AD system may perform a maneuver to complete parking of the vehicle 300 in the parking spot 308 or 324 after control is handed over by the driver.

[0086] 5A-5D show examples of using the AD system to modify completed parking of a vehicle 500. Some or all of the examples for vehicle 500 may be used in conjunction with one or more other examples described elsewhere herein.

[0087] Examples will be described with reference to a parking bay 502 that may have one or more entrances or exits (not shown) for vehicles. Parking spots within the parking bay 502 may be arranged in one or more rows. One or more rows in the parking bay 502 may have parking spots that are separated from one another by a (e.g., painted or structural) divider. One or more rows in the parking bay 502 may have parking spots that are not separated from one another by any divider, where a row 504 includes parking spots in which vehicles are positioned essentially parallel to the direction of travel along the row 504. Other approaches may be used.

[0088] The vehicle 500 is parked approximately in a parking spot 506 located in row 504. In this example, the vehicle 500 was parked by the driver, not by the AD system. In some implementations, the driver steered the vehicle 500 using the steering wheel, gear selector, accelerator, and brake pedals. For example, briefly referring back to the example of FIGS. 3A-3B, the driver of the vehicle may not have accepted the prompt from the AD system (i.e., did not actuate control 314) and instead parked the vehicle manually. Here, the vehicle 500 has an orientation that may not be considered ideal or even acceptable for parking in the parking stall 502. For example, the orientation of the vehicle 500 is characterized by the fact that the orientation does not match the orientation of the other vehicles in row 504; the space in front and behind the vehicle 500 may be uneven and / or insufficient; the space on the left and right sides of the vehicle 500 may be uneven and / or insufficient; and / or the vehicle 500 may be too far from the curb.

[0089] The AD system of the vehicle 500 may determine that the vehicle 500 is stationary within the parking bay 502. Additionally, using a sensor, the AD system may determine that the vehicle 500 has a first orientation with respect to the parking spot 506. Although the use of a sensor is illustrated here diagrammatically by a signal 508 near the vehicle 500, the sensor does not necessarily generate any radiation or other emissions to probe the surroundings.

[0090] The AD system may execute the motion plan to move the vehicle 500 to a second orientation relative to the parking spot 506. For example, executing the motion plan may include planning a trajectory from a first orientation to a second orientation. Here, for example, if the driver wants the AD system to correct the parking of the vehicle 500, the AD system may execute the motion plan for the vehicle 500.

[0091] The AD system may generate a prompt to the driver to have the AD system correct the first orientation of the vehicle. Here, a UI 510 of the vehicle 500 is illustrated in a schematic manner. The UI 510 is an example of an HMI that the vehicle 500 may use to communicate with the driver or another passenger. In some implementations, the UI 510 may include a GUI. For example, the GUI may be generated using a display device with touch screen capabilities. Here, the UI 510 makes a control 512 available to the driver. For example, the control 512 may be a GUI button labeled "Touch Up My Parking" that suggests to the driver to have the AD system correct the parking of the vehicle 500 in the parking spot 506. In some implementations, the control 512 may be provided in conjunction with ceasing to provide the control 314 (FIG. 3A). For example, a seamless transition may be provided between the AD system offering to assist in completing the ongoing parking of the vehicle 500 and the AD system instead suggesting to correct the driver's completed parking maneuver. In some implementations, the UI 510 may indicate the second orientation in which the AD system is currently proposing to correct parking of the vehicle 500. For example, the UI 510 may present a map of the parking spot 506 with the second orientation indicated; or the UI 510 may indicate the second orientation by AR techniques applied to a captured image of the parking spot 506. Other approaches may be used.

[0092] That is, in response to the AD system determining that the vehicle 500 is stationary in the first orientation, the control 512 may be made available to the driver. The driver may accept a prompt by activating the control 512. If the driver accepts the prompt, the AD system may take over control of some or all of the vehicle 500 and modify parking of the vehicle 500 with respect to the parking spot 506. This may involve the AD system receiving inputs that the driver generates with the control 512 in response to the prompt. The AD system may then control the vehicle in response to the inputs to complete parking of the vehicle 500 in the parking spot 506. For example, the AD system may execute this maneuver using a motion plan for the vehicle 500 for the second orientation that it generates. This feature may ensure that convenient parking assistance is provided to the driver if the driver so desires following a completed manual parking operation. To name just a few examples, the driver can activate controls 512 using the GUI of the vehicle 500, the GUI of the portable electronic device, by giving audio commands (e.g., using the audio interface 1132 in FIG. 11 or a portable electronic device) and / or by making gestures (e.g., using the visual interface 1134 in FIG. 11 or a portable electronic device).

[0093] If there is a handover to the AD system, the driver can exit the vehicle 500 before or after control is granted to the AD system, or can remain in the vehicle during the maneuver. For example, the driver can exit the vehicle 500 to the parking stall 502 and activate the control 512 using a remote control device (e.g., a portable electronic device such as a smartphone having an app that presents the control 512 for activation). As another example, after stopping the vehicle 500, the driver can open the door of the vehicle 500 and then activate the control 512 using a GUI inside the vehicle 500. The AD system can receive the driver-generated input and may wait to complete the parking correction until it detects that the door of the vehicle 500 is closed (at which point the driver may have exited the vehicle 500). Other approaches can be used.

[0094] Here, the driver accepts the prompt generated by the AD system by actuating control 512. FIG. 5B shows the parking bay 502 after the AD system has finished executing its parking maneuver completion. Path 514 illustrates diagrammatically that execution of the AD system may include one or more actions by the vehicle 500 to follow the planned trajectory. For example, if the vehicle 500 is currently stationary in a nose-first orientation in the parking spot 506 and the AD system is asked to correct the parking, path 514 may involve first driving the vehicle 500 backward, then driving the vehicle 500 forward, and finally driving the vehicle 500 backward again to enter the parking spot 506. Other approaches may be used.

[0095] A motion plan according to which the AD system controls the vehicle 500 to follow the path 514 may be defined based on a first orientation of the vehicle 500 at the planning moment, as well as an intended second orientation of the vehicle 500. The second orientation may be defined according to one or more characteristics. For example, the second orientation may match the orientation of one or more other vehicles (e.g., the closest vehicle) in the queue 504. As another example, the second orientation may position the vehicle 500 to have at least a certain amount of space in front of and / or behind the vehicle 500 (e.g., a substantially equal amount of space in front of and behind the vehicle 500). As another example, the second orientation may position the vehicle 500 to have at least a certain amount of space in front of and / or behind the vehicle 500 (e.g., a substantially equal amount of space in front of and behind the vehicle 500). As another example, the second orientation may position the vehicle 500 to have at most a predefined distance from a curb adjacent to the parking spot 506. In some implementations, machine learning techniques may be used. For example, the AD system may execute a motion plan such that the second orientation is consistent with a previously executed parking arrangement associated with the vehicle 500 and / or otherwise with the driver.

[0096] 5C-5D, examples will be described with reference to a parking stall 516 that may have one or more entrances or exits (not shown) for vehicles. Parking spots within the parking stall 516 may be arranged in one or more rows. One or more rows in the parking stall 516 may have parking spots that are separated from one another by a (e.g., painted or structural) divider. One or more rows in the parking stall 516 may have parking spots that are not separated from one another by any divider. Here, a row 518 includes parking spots that are positioned essentially perpendicular to the direction of travel of vehicles along the row 518. Other approaches may be used.

[0097] The vehicle 500 is parked approximately in a parking spot 520 located in row 518. In this example, the vehicle 500 was parked by the driver, not by the AD system. In some implementations, the driver steered the vehicle 500 using the steering wheel, gear selector, accelerator, and brake pedals. For example, briefly referring back to the example of FIGS. 3C-3D, the driver of the vehicle may not have accepted the prompt from the AD system (i.e., did not actuate control 314′) and instead parked the vehicle manually. Here, the vehicle 500 has an orientation that may not be considered ideal or even acceptable for parking in the parking stall 516. For example, the orientation of the vehicle 500 is characterized by the fact that the orientation does not match the orientation of the other vehicles in row 518; the space in front and behind the vehicle 500 may be uneven and / or insufficient; the space on the left and right sides of the vehicle 500 may be uneven and / or insufficient; and / or the vehicle 500 may be too far from the curb.

[0098] The AD system of the vehicle 500 may determine that the vehicle 500 is stationary in the parking stall 516. Additionally, using a sensor, the AD system may determine that the vehicle 500 has a first orientation with respect to the parking spot 520. Although the use of a sensor is illustrated here diagrammatically by a signal 522 near the vehicle 500, the sensor does not necessarily generate any radiation or other emissions to probe the surroundings.

[0099] The AD system may execute the motion plan to move the vehicle 500 to a second orientation relative to the parking spot 520. For example, executing the motion plan may include planning a trajectory from a first orientation to a second orientation. Here, for example, if the driver wants the AD system to correct the parking of the vehicle 500, the AD system may execute the motion plan for the vehicle 500.

[0100] The AD system may generate a prompt to the driver to have the AD system correct the first orientation of the vehicle. Here, a UI 510 of the vehicle 500 is illustrated in a schematic manner. Here, the UI 510 makes a control 512' available to the driver. For example, the control 512' may be a GUI button labeled "Touch Up My Parking" that suggests to the driver to have the AD system correct the parking of the vehicle 500 in the parking spot 520. In some implementations, the control 512' may be provided in conjunction with ceasing to provide the control 314' (FIG. 3C). For example, a seamless transition may be provided between the AD system suggesting to assist in completing the ongoing parking of the vehicle 500 and the AD system instead suggesting to correct the driver's completed parking maneuver. In some implementations, the UI 510 may indicate a second orientation in which the AD system is currently proposing to correct the parking of the vehicle 500. For example, the UI 510 may present a map of the parking spot 520 with the second orientation indicated; or the UI 510 may show the second orientation by AR technology applied to a captured image of the parking spot 520. Other approaches may be used.

[0101] That is, in response to the AD system determining that the vehicle 500 is stationary in the first orientation, a control 512' may be made available to the driver. The driver may accept a prompt by activating the control 512'. If the driver accepts the prompt, the AD system may take over control of some or all of the vehicle 500 and modify parking of the vehicle 500 with respect to the parking spot 520. This may involve the AD system receiving inputs that the driver generates with the control 512' in response to the prompt. The AD system may then control the vehicle in response to the inputs to complete parking of the vehicle 500 in the parking spot 520. For example, the AD system may execute this maneuver using a motion plan for the vehicle 500 for the second orientation generated by the AD system. This feature may ensure that convenient parking assistance is provided to the driver if the driver so desires following a completed manual parking operation. To name just a few examples, the driver can activate controls 512' using the GUI of the vehicle 500, the GUI of the portable electronic device, by giving audio commands (e.g., using the audio interface 1132 in FIG. 11 or a portable electronic device) and / or by making gestures (e.g., using the visual interface 1134 in FIG. 11 or a portable electronic device).

[0102] If there is a handover to the AD system, the driver can exit the vehicle 500 before or after control is granted to the AD system, or can remain in the vehicle during the maneuver. For example, the driver can exit the vehicle 500 to the parking stall 516 and actuate the control 512' using a remote control device (e.g., a portable electronic device such as a smartphone having an app that presents the control 512' for actuation). As another example, after stopping the vehicle 500, the driver can open the door of the vehicle 500 and then actuate the control 512' using a GUI inside the vehicle 500. The AD system can receive the driver-generated input and may wait to complete the parking correction until it detects that the door of the vehicle 500 is closed (at which point the driver may have exited the vehicle 500). Other approaches can be used.

[0103] Here, the driver accepts the prompt generated by the AD system by actuating control 512'. FIG. 5D shows parking space 516 after the AD system has finished executing its parking maneuver completion. Path 524 illustrates diagrammatically that execution of the AD system may include one or more actions by vehicle 500 to follow a planned trajectory. For example, if vehicle 500 is currently stationary in a nose-first orientation in parking spot 520 and the AD system is asked to correct the parking, path 524 may involve first driving vehicle 500 backward and then driving vehicle 500 forward into parking spot 520. Other approaches may be used.

[0104] A motion plan according to which the AD system controls the vehicle 500 to follow the path 524 may be defined based on a first orientation of the vehicle 500 at the planning moment, as well as an intended second orientation of the vehicle 500. The second orientation may be defined according to one or more characteristics. For example, the second orientation may match the orientation of one or more other vehicles (e.g., the closest vehicle) in the row 518. As another example, the second orientation may position the vehicle 500 to have at least a certain amount of space in front of and / or behind the vehicle 500 (e.g., a substantially equal amount of space in front of and behind the vehicle 500). As another example, the second orientation may position the vehicle 500 to have at least a certain amount of space in front of and / or behind the vehicle 500 (e.g., a substantially equal amount of space in front of and behind the vehicle 500). As another example, the second orientation may position the vehicle 500 to have at most a predefined distance from a curb adjacent to the parking spot 520. In some implementations, machine learning techniques may be used. For example, the AD system may execute a motion plan such that the second orientation is consistent with a previously executed parking arrangement associated with the vehicle 500 and / or otherwise with the driver.

[0105] 6 shows a flow chart of a method 600 for the example in Figures 5A-5D. Method 200 may be used with one or more other examples described elsewhere herein. More or less operations may be performed than those shown. Two or more operations may be performed in a different order unless otherwise indicated.

[0106] At operation 602, the AD system may determine that the vehicle is stationary in a first orientation with respect to the parking spot. For example, and referring briefly back to Figures 5A-5D, the AD system may determine that the vehicle 500 is stationary approximately within parking spot 506 or 520.

[0107] At operation 604, the AD system may execute a motion plan to move the vehicle to a second orientation relative to the parking spot. In some implementations, this may involve the AD system planning a trajectory for the vehicle to park relative to the parking spot. For example, such a trajectory may be defined as a path that the vehicle will travel as a result of sequential control of the vehicle's steering system, transmission, accelerator pedal, and / or brake pedal by the AD system.

[0108] In operation 606, the AD system may generate a prompt to the driver to cause the AD system to correct the first orientation of the vehicle. The prompt may be generated in response to detecting that the vehicle is stationary. In Figures 5A-5D, for example, the AD system may provide controls 512 and / or 512' to the driver.

[0109] The AD system may receive driver-generated input in response to the prompt at operation 608. In Figures 5A-5D, for example, the driver may actuate control 512 and / or 512'.

[0110] In operation 610, the AD system may control the vehicle to assume a second orientation with respect to the parking spot in response to the input received in operation 608. The control may be performed according to a motion plan. In Figures 5A-5D, for example, the AD system may perform a maneuver to correct parking by controlling the motion of vehicle 100 according to path 514 or 524 after control is handed over by the driver.

[0111] FIG. 7 illustrates an example of a system 700. The system 700 may be used with one or more other examples described elsewhere herein. For example, the system 700 may be implemented as part of any or all of the vehicles described herein. To name just two examples, the system 700 may be implemented in multiple units that are separate components from one another or as a single integrated component. The components of the system 700 may include one or more of the components mentioned below with reference to FIG. 13.

[0112] The system 700 includes a parking controller 702 connected to each of a number of respective components. Such components here include a remote key / phone app 704, a vehicle actuator 706, an HMI 708, sensors 710, and a vehicle data component 712.

[0113] The parking controller 702 may include one or more subcomponents, where the parking controller 702 includes a perception component 714, a state machine 716, a path planning component 718, and a machine learning component 720.

[0114] The perception component 714 may receive, condition, analyze, interpret, and / or act on sensory input regarding the vehicle. In some implementations, the sensory input may be obtained at least in part from sensors 710. For example, the sensors 710 may indicate the vehicle's surroundings, the way the vehicle is traveling, the location of the vehicle, and / or the orientation of the vehicle.

[0115] The state machine 716 can define different operational states for the vehicle, which can directly or indirectly regulate the operation of the system 700 and / or other components of the vehicle. In some implementations, the state machine can define one or more levels of operation for which assistance is provided by the AD system. For example, such assistance can include searching for a parking spot, completing an ongoing manual parking maneuver, or reworking a manually completed parking of the vehicle.

[0116] A path planning component 718 may be used for motion planning, such as by planning one or more trajectories that the AD system will use to control the vehicle if a handover occurs. For example, the path planning component may define a sequence of actions that affect the steering angle adjustments, gear selection, acceleration, and / or braking of the vehicle to navigate along a defined path.

[0117] The machine learning component 720 can receive inputs regarding the driving behavior of the vehicle and / or the driver, analyze the data of the inputs, and use the analysis to control future automatic control of the vehicle. In some implementations, the machine learning component 720 obtains signals and / or other information from the vehicle actuators 706, the sensors 710, or the vehicle data component 712. For example, the machine learning component 720 can determine how the vehicle has typically been parked in the past and / or how the driver of the vehicle has typically parked another vehicle in the past.

[0118] The remote key / phone app 704 may be used to remotely control the vehicle, either directly or indirectly. In some implementations, the remote key / phone app 704 may be used to generate prompts and receive inputs corresponding to actuation of controls 116 or 116' (FIGS. 1B-1C), controls 314 or 314' (FIGS. 3A, 3C), and / or controls 512 or 512' (FIGS. 5A, 5C). For example, the remote key / phone app 704 may enable the driver to trigger the completion of a parking maneuver and / or modification of a completed parking maneuver while outside of the vehicle.

[0119] Vehicle actuators 706 serve to control aspects of the vehicle's operation. For example, steering, gear selection, acceleration, and / or braking may be controlled.

[0120] The HMI 708 can facilitate communication between the vehicle (e.g., the AD system) and the driver before, during, and after the vehicle is driven. In some implementations, the HMI 708 can be used to generate prompts and receive inputs corresponding to actuation of controls 116 or 116' (FIGS. 1B-1C), 314 or 314' (FIGS. 3A, 3C), and / or 512 or 512' (FIGS. 5A, 5C). For example, a touch screen device can be used.

[0121] Sensors 710 detect or otherwise register characteristics of the vehicle's surroundings. For example, sensors 710 may be used to determine whether there is an available parking spot near the vehicle, whether the vehicle has entered a parking spot, whether the vehicle has come to rest approximately within the parking spot, and / or the vehicle's position and orientation relative to its surroundings.

[0122] The vehicle data component 712 may indicate characteristics of the vehicle (e.g., its size or the type of terrain it is designed to navigate), the nature of the current driving session (e.g., whether a navigation destination is defined), and / or external information (e.g., global positioning system data, traffic data, and / or weather data). Any or all types of information from the vehicle data component 712 may be used by the parking controller in performing operations according to one or more of the examples described herein.

[0123] 8 illustrates an example sequence diagram 800 for the AD system being available to complete an ongoing parking maneuver for a vehicle. The sequence diagram 800 may be used with one or more other examples described elsewhere herein. More or fewer actions may be performed than those shown. Two or more actions may be performed in a different order unless otherwise indicated.

[0124] The sequence diagram 800 illustrates the distribution of actions by, and / or interactions between, two or more of a driver 802, a system perception 804, an HMI 806, a system path planning component 808, or a system vehicle control 810.

[0125] At operation 812, the driver 802 may drive the vehicle at a relatively slow speed. In some implementations, the AD system may be designed to offer parking assistance only in relatively slow speed situations. For example, this may involve traveling at or below the speed limit that applies in the parking stall.

[0126] In operation 814, system perception 804 may evaluate the environment around the vehicle. In some implementations, this may be performed by an AD system that continuously monitors the vehicle's surroundings. For example, this may allow detection of whether any parking spots are available.

[0127] At operation 816, the driver 802 may begin driving the vehicle into the parking spot.

[0128] In operation 818, system perception 804 may detect a parking spot as available for the vehicle.

[0129] At operation 820, the HMI 806 may generate a prompt to the driver that the AD system is ready to take over the parking maneuver.

[0130] At operation 822 , the driver 802 may generate an input on the HMI 806 .

[0131] It may be determined whether the driver has accepted (e.g., confirmed) the proposed assistance at operation 824. If the driver does not accept the prompt (i.e., “No” at operation 824), System Perception 804 may continue to evaluate the environment around the vehicle at operation 814.

[0132] If the driver accepts the prompt (i.e., "yes" at operation 824), other actions may be performed. For example, system perception 804 may continuously evaluate the environment around the vehicle at operation 826. As another example, HMI 806 may represent system status at operation 828. As another example, system path planning component 808 may continuously plan a trajectory for the vehicle (e.g., to park in a given spot) at operation 830. As another example, system vehicle control 810 may control the longitudinal and lateral motion of the vehicle at operation 832. One or more of these actions may continue until the AD system completes parking the vehicle.

[0133] At operation 834, the HMI 806 may indicate that parking is complete.

[0134] 9 illustrates an example sequence diagram 900 for a driver being outside a vehicle that has been parked by an AD system. The sequence diagram 900 may be used with one or more other examples described elsewhere herein. More or fewer actions may be performed than those shown. Two or more actions may be performed in a different order unless otherwise indicated.

[0135] The sequence diagram 900 illustrates the distribution of actions by two or more of a driver 902, a system perception 904, an HMI 906, a system path planning component 908, or a system vehicle control 910, and the interactions between them.

[0136] At operation 912, the driver 902 may drive the vehicle at a relatively slow speed. In some implementations, the AD system may be designed to offer parking assistance only in relatively slow speed situations. For example, this may involve traveling at or below the speed limit that applies in the parking stall.

[0137] At operation 914, system perception 904 may evaluate the environment around the vehicle. In some implementations, this may be performed by an AD system that continuously monitors the vehicle's surroundings. For example, this may allow detection of whether any parking spots are available.

[0138] At operation 916, the driver 902 may begin driving the vehicle into the parking spot.

[0139] In operation 918, system perception 904 may detect a parking spot as available for the vehicle.

[0140] At operation 920, the HMI 906 may generate a prompt to the driver that the AD system is ready to take over the parking maneuver.

[0141] At operation 922 , the driver 902 may generate an input on the HMI 906 .

[0142] It may be determined whether the driver has accepted (e.g., confirmed) the proposed assistance at operation 924. If the driver has not accepted the prompt (i.e., “No” at operation 924), System Perception 904 may continue to evaluate the environment around the vehicle at operation 914.

[0143] If the driver accepts the prompt (i.e., "yes" at operation 924), other actions may be performed. For example, the driver 902 may exit the vehicle at operation 926. For example, this may enable the driver 902 to activate assistance by the AD system using a remote device. Additionally, the system perception 904 may continuously evaluate the environment around the vehicle at operation 928. As another example, the HMI 906 may represent the system status at operation 930. As another example, the system path planning component 908 may continuously plan a trajectory for the vehicle (e.g., to park in a given spot) at operation 932. As another example, the system vehicle control 910 may control the longitudinal and lateral motion of the vehicle at operation 934. One or more of these actions may continue until the AD system completes parking the vehicle.

[0144] At operation 936, the HMI 906 may indicate that parking is complete.

[0145] 10 illustrates an example sequence diagram 1000 for applying machine learning to an AD system that can be used to control parking of a vehicle. The sequence diagram 1000 can be used with one or more other examples described elsewhere herein. More or fewer operations than those shown can be performed. Two or more operations can be performed in a different order unless otherwise indicated.

[0146] The sequence diagram 1000 illustrates the distribution of actions by, or interactions between, two or more of a driver 1002, a system perception 1004, a system learning component 1006, an HMI 1008, a system path planning component 1010, and a system vehicle control 1012.

[0147] At operation 1014, the driver 1002 may park the vehicle in a regular parking spot. In some implementations, the AD system may be designed to distinguish between parking maneuvers performed in commonly occurring situations (e.g., at home or work) and sporadic situations (e.g., at an event). For example, this may enable the AD system to better detect the behavioral patterns of the driver 1002.

[0148] At operation 1016, system perception 1004 may identify the location of the vehicle relative to its immediate surroundings. In some implementations, this may be performed by an AD system that continuously monitors the vehicle's surroundings.

[0149] At operation 1018, the system learning component 1006 may learn from the parking behavior of the driver 1002 over time (e.g., a sequence of multiple observed parking maneuvers) how they tend to park in that particular parking spot. As such, machine learning may be performed and associated with the parking spot and / or the driver 1002.

[0150] At operation 1020, the driver 1002 drives to a regular spot at another time.

[0151] In operation 1022, system perception 1004 may detect that the spot is a normal spot on which machine learning has been performed.

[0152] In operation 1024, the HMI 1008 may generate a prompt to the driver that the AD system is ready to take over the parking maneuver.

[0153] At operation 1026 , the driver 1002 may generate an input on the HMI 1008 .

[0154] It may be determined whether the driver accepted (e.g., confirmed) the proposed assistance at operation 1028. If the driver did not accept the prompt (i.e., "No" at operation 1028), the system learning component 1006 may continue to learn from the driver's behavior at operation 1002.

[0155] If the driver accepts the prompt (i.e., “yes” at operation 1028), other actions may be performed. For example, the driver 1002 may exit the vehicle at operation 1030. For example, this may enable the driver 1002 to activate assistance by the AD system using a remote device. Additionally, the system perception 1004 may continuously evaluate the environment around the vehicle at operation 1032. As another example, the system learning component 1006 may use machine learning to determine a required correction of parking at operation 1034. In some implementations, the correction is determined based on the driver's 1002's past driving behavior. For example, the machine learning may indicate an orientation in which the vehicle should be placed. As another example, the HMI 1008 may represent a system status at operation 1036. As another example, the system path planning component 1010 may continuously plan a trajectory for the vehicle (e.g., to park in a given spot according to the learned behavior) at operation 1038. As another example, the system vehicle control 1012 may control the longitudinal and lateral movement of the vehicle in operation 1040. One or more of these operations may continue until the AD system has completed parking the vehicle.

[0156] At operation 1042, the HMI 1008 may indicate that parking is complete.

[0157] 11 illustrates an example of a vehicle 1100. The vehicle 1100 may be used with one or more other examples described elsewhere herein. The vehicle 1100 may include an ADAS / AD system 1102 and vehicle control 1104. The ADAS / AD system 1102 may be implemented using some or all of the components described with reference to FIG. 13 below. The ADAS / AD system 1102 includes sensors 1106 and a planning algorithm 1108. Other aspects that the vehicle 1100 may include, including but not limited to other components of the vehicle 1100 in which the ADAS / AD system 1102 may be implemented, are omitted here for brevity.

[0158] The sensor 1106 is described herein as also including appropriate circuitry and / or executable programming for processing the sensor output and performing detection based on that processing. The sensor 1106 may include a radar 1110. In some implementations, the radar 1110 may include any object detection system based at least in part on radio waves. For example, the radar 1110 may be oriented in a forward direction relative to the vehicle and may be used to detect at least a distance to one or more other objects (e.g., another vehicle). The radar 1110 may detect the surroundings of the vehicle 1100 by sensing the presence of objects relative to the vehicle 1100.

[0159] The sensor 1106 may include a lidar 1112. In some implementations, the lidar 1112 may include any object detection system based at least in part on laser light. For example, the lidar 1112 may be oriented in any direction relative to the vehicle and may be used to detect at least the distance to one or more other objects (e.g., another vehicle). The lidar 1112 may detect the surroundings of the vehicle 1100 by sensing the presence of objects relative to the vehicle 1100.

[0160] The sensors 1106 may include a camera 1114. In some implementations, the camera 1114 may include any image sensor whose signal is taken into account by the vehicle 1100. For example, the camera 1114 may be oriented in any direction relative to the vehicle and may be used to detect vehicles, lanes, lane markings, curbs, and / or road signs. The camera 1114 may detect the surroundings of the vehicle 1100 by visually registering conditions related to the vehicle 1100.

[0161] The sensors 1106 may include an ultrasonic sensor 1116. In some implementations, the ultrasonic sensor 1116 may include any transmitter, receiver, and / or transceiver used in detecting at least the proximity of an object based on ultrasonic waves. For example, the ultrasonic sensor 1116 may be positioned on or near the exterior of the vehicle. The ultrasonic sensor 1116 may detect the surroundings of the vehicle 1100 by sensing the presence of an object relative to the vehicle 1100.

[0162] Any of the sensors 1106 alone, or two or more of the sensors 1106 collectively, may detect whether the ADAS / AD system 1102 is controlling the motion of the vehicle 1100, its surroundings. In some implementations, at least one of the sensors 1106 may generate an output that is taken into account in providing prompts to the driver and / or in controlling the motion of the vehicle 1100. For example, the outputs of two or more sensors (e.g., the outputs of the radar 1110, the lidar 1112, and the camera 1114) may be combined. In some implementations, one or more other types of sensors may be included in the sensors 1106 additionally or instead.

[0163] The planning algorithm 1108 may plan the ADAS / AD system 1102 to perform one or more actions, or no action, in response to monitoring the surroundings of the vehicle 1100 and / or driver inputs. Outputs of one or more of the sensors 1106 may be taken into account. In some implementations, the planning algorithm 1108 may perform motion planning and / or plan a trajectory for the vehicle 1100.

[0164] Vehicle control 1104 may include steering control 1118. In some implementations, the ADAS / AD system 1102 and / or another driver of the vehicle 1100 controls the trajectory of the vehicle 1100 by adjusting the steering angle of at least one wheel by manipulating the steering control 1118. The steering control 1118 may be configured to control the steering angle through a mechanical connection between the steering control 1118 and an adjustable wheel, or may be part of a steer-by-wire system.

[0165] Vehicle control 1104 may include gear control 1120. In some implementations, the ADAS / AD system 1102 and / or another operator of the vehicle 1100 uses gear control 1120 to select among multiple operating modes of the vehicle (e.g., drive mode, neutral mode, or park mode). For example, gear control 1120 may be used to control an automatic transmission in the vehicle 1100.

[0166] Vehicle control 1104 may include traffic light control 1122. In some implementations, traffic light control 1122 may control one or more traffic lights that may be generated by vehicle 1100. For example, traffic light control 1122 may control the turn signals and / or the horn of vehicle 1100.

[0167] Vehicle controls 1104 may include brake controls 1124. In some implementations, brake controls 1124 may control one or more types of braking systems designed to slow the vehicle, stop the vehicle, and / or keep the vehicle stationary when stopped. For example, brake controls 1124 may be actuated by the ADAS / AD system 1102. As another example, brake controls 1124 may be actuated by the driver using a brake pedal.

[0168] Vehicle control 1104 may include a vehicle dynamics system 1126. In some implementations, the vehicle dynamics system 1126 may control one or more functions of the vehicle 1100 in addition to, in the absence of, or instead of driver control. For example, when the vehicle is stopped on a hill, the vehicle dynamics system 1126 may hold the vehicle stationary if the driver does not activate the brake control 1124 (e.g., by pressing the brake pedal).

[0169] Vehicle control 1104 may include acceleration control 1128. In some implementations, acceleration control 1128 may control one or more types of propulsion motors of the vehicle. For example, acceleration control 1128 may control an electric motor and / or an internal combustion motor of the vehicle 1100.

[0170] The vehicle 1100 may include a user interface 1130 . The user interface 1130 may include an audio interface 1132. In some implementations, the audio interface 1132 may include one or more speakers positioned within the passenger compartment. For example, the audio interface 1132 may operate, at least in part, with an infotainment system within the vehicle.

[0171] In some implementations, the audio interface 1132 may provide speech recognition functionality. For example, the audio interface 1132 may receive an audio signal from a microphone and process the audio signal to convert the audio to text. The vehicle 1100 may interpret the text as one or more user commands and perform one or more actions in response to the commands. For example, the commands may correspond to the instruction "park" (e.g., in FIGS. 1A-1C), "complete my parking" (e.g., in FIGS. 3A-3D), or "touch up my parking" (e.g., in FIGS. 5A-5D). The audio interface 1132 may be implemented in a passenger compartment of a vehicle (e.g., any or all of the vehicles described herein) and / or in a portable electronic device.

[0172] The user interface 1130 may include a visual interface 1134. In some implementations, the visual interface 1134 may include at least one display device in a passenger compartment of the vehicle 1100. For example, the visual interface 1134 may include a touch screen device and / or an instrument cluster display.

[0173] In some implementations, the visual interface 1134 may provide gesture recognition capabilities. For example, the visual interface 1134 may receive a video signal from a camera and process the video signal to interpret gestures captured by the camera. The vehicle 1100 may apply the interpreted gestures as one or more user commands and perform one or more actions in response to the commands. For example, the commands may correspond to the instruction "park" (e.g., in FIGS. 1A-1C), "complete my parking" (e.g., in FIGS. 3A-3D), or "touch up my parking" (e.g., in FIGS. 5A-5D). The audio interface 1132 may be implemented in a passenger compartment of a vehicle (e.g., any or all of the vehicles described herein) and / or in a portable electronic device.

[0174] 12A-12C show additional examples relating to parking. The examples may be used with one or more other examples described elsewhere herein. The example in FIG. 12A will be described with reference to a parking stall 1200, which may have one or more entrances or exits (not shown) for vehicles. Parking spots within the parking stall 1200 may be arranged in one or more rows. Row 1202A includes parking spots defined by a (e.g., painted or structural) divider 1204 such that vehicles are positioned essentially diagonally to the direction of travel along row 1202A. Similarly, row 1202B includes parking spots where vehicles are positioned essentially diagonally to the direction of travel along row 1202B. Other approaches may be used. By diagonally positioned, the parking spots are neither perpendicular nor parallel to the direction of travel.

[0175] Vehicle 100 is traveling within parking stall 1200, as indicated diagrammatically by arrow 1206. Vehicle 100 is traveling in the lane formed between rows 1202A-1202B. Vehicle 100 is currently essentially controlled by the driver. For example, the driver is currently steering vehicle 100 using the steering wheel, gear selector, accelerator, and brake pedals, searching for an available parking spot within parking stall 1200.

[0176] The AD system of the vehicle 100 continuously monitors the surroundings of the vehicle 100 using at least one sensor to detect whether any parking spots are available. The use of sensors is illustrated here diagrammatically by a signal 1208 near the vehicle 100, but the sensor does not necessarily generate any radiation or other emissions to probe the surroundings. Here, the AD system of the vehicle 100 may have detected a parking spot 1210 in row 1202B and may have prompted the occupant of the vehicle 100 using a control 116'' in the UI 114. The parking spot 1210 may have been detected before the vehicle 100 arrived at the parking spot 1210. However, in this example, the vehicle 100 continued to drive in the parking stall 1200 past the location of the parking spot 1210.

[0177] Continuous monitoring of the surroundings of vehicle 100 again resulted in the detection of parking spot 1214 located in row 1202A. Parking spot 1214 is now detected before vehicle 100 arrives at parking spot 1214. For example, vehicle 100 does not have to pull up beside or begin to enter parking spot 1214 for detection to occur. Rather, the system detects parking spot 1214 when sensors generate a signal that positively indicates that parking spot 1214 is suitable for vehicle 100 and is not currently occupied.

[0178] As mentioned above, at this point in the example, vehicle 100 may be controlled only by the driver. Nevertheless, for any parking spot that the AD system detects, the AD system may plan a trajectory for vehicle 100. This may ensure that the AD system may provide a fluid handover from the driver at any time if the driver so chooses. Here, for example, if the driver wishes to park vehicle 100 in parking spot 1214, the AD system plans a trajectory for vehicle 100 to enter there.

[0179] The AD system may generate a prompt to the driver regarding one or more detected parking spots. For example, the control 116'' may be a GUI button labeled "PARK" suggesting to the driver to let the AD system handle parking of the vehicle 100 in the parking spot 1214. In some implementations, the UI 114 may show the parking spot 1214 in which the AD system is currently suggesting to the driver to park the vehicle 100. For example, the UI 114 may present a map of at least a portion of the parking section 1200 in which the parking spot 1214 is shown; or the UI 114 may show the parking spot 1214 by AR technology applied to a captured image of the surroundings of the vehicle 100. Other approaches may be used.

[0180] The driver may accept the prompt by activating control 116''. Control 116'' may be available to the driver before vehicle 100 arrives at parking spot 1214. For example, vehicle 100 does not have to pull in beside or begin to enter parking spot 1214 for control 116'' to be active. If the driver accepts the prompt by activating control 116'', the AD system may take over control of some or all of vehicle 100 and steer vehicle 100 into parking spot 1214. This may involve the AD system receiving inputs generated by the driver using control 116'' in response to the prompt, such inputs being generated before the vehicle enters parking spot 1214. The AD system may then control the vehicle in response to the inputs to park the vehicle in parking spot 1214. For example, the AD system may perform this maneuver using a trajectory for vehicle 100 that the AD system generates upon detecting parking spot 1214. This functionality may ensure that convenient parking assistance is provided to the driver, both to identify available parking spots and to take over steering of the vehicle 100 to an available parking spot if the driver so desires at any point during the operation. To name just a few examples, the driver may actuate controls 116'' using the GUI of the vehicle 100, the GUI of the portable electronic device, by giving audio commands (e.g., using the audio interface 1132 in FIG. 11 or the portable electronic device) and / or by making gestures (e.g., using the visual interface 1134 in FIG. 11 or the portable electronic device).

[0181] If there is a handover to the AD system, the driver can exit the vehicle 100 before or after control is granted to the AD system, or can remain in the vehicle during the maneuver. For example, after the control 116'' is available, the driver can stop the vehicle 100, exit the vehicle to the parking stall 1200, and activate the control 116'' using a remote control device (e.g., a portable electronic device such as a smartphone having an app that presents the control 116'' for activation). As another example, after stopping the vehicle 100, the driver can open the door of the vehicle 100 and then activate the control 116'' using a GUI inside the vehicle 100. The AD system can receive the driver-generated input and can wait to perform the parking maneuver until it detects that the door of the vehicle 100 is closed (at which point the driver may have exited the vehicle 100). Other approaches can be used.

[0182] To name just two examples, if the driver of the vehicle 100 does not actuate the control 116'', the driver may manually park in either the parking spot 1210 or 1214 or may forgo parking in the parking bay 1200. If the driver begins to park manually (e.g., in either the parking spot 1210 or 1214), one or more other options may become available to the driver. In some implementations, "finish my parking" assistance may be offered, for example, along the lines of the examples described with reference to FIGS. 3A-3D. In some implementations, "touch up my parking" assistance may be offered, for example, along the lines of the examples described with reference to FIGS. 5A-5D.

[0183] The example in Fig. 12B will be described with reference to a road 1216. Parking spots may be arranged along the edge of the road 1216, where the road 1216 curves to the right in this illustration, as indicated by the right edge 1218. The vehicle 100 is traveling along the road 1216, as indicated diagrammatically by the arrow 1220. The vehicle 100 is traveling to the left of the row of cars parked along the curved shape of the right edge 1218. The vehicle 100 is currently essentially controlled by the driver. For example, the driver is currently maneuvering the vehicle 100 using the steering wheel, gear selector, accelerator, and brake pedals in search of an available parking spot along the road 1216.

[0184] The AD system of the vehicle 100 continuously monitors the surroundings of the vehicle 100 using at least one sensor to detect whether any parking spots are available. The use of a sensor is illustrated here diagrammatically by a signal 1222 near the vehicle 100, although the sensor does not necessarily generate any radiation or other emissions to probe the surroundings.

[0185] As mentioned above, at this point in the example, vehicle 100 may be controlled only by the driver. Nevertheless, for any parking spot that the AD system detects, the AD system may plan a trajectory for vehicle 100. This may ensure that the AD system may provide a fluid handover from the driver at any time if the driver so chooses. Here, for example, if the driver wishes to park vehicle 100 in parking spot 1224, the AD system plans a trajectory for vehicle 100 to enter there.

[0186] Here, the AD system of the vehicle 100 may have detected a parking spot 1224 along the right edge 1218, and the control 116 in the UI 114 (3)The parking spot 1224 may have been detected before the vehicle 100 arrived at the parking spot 1224. The AD system may generate a prompt to the driver regarding one or more detected parking spots. For example, the control 116 (3) may be a GUI button labeled "PARK" suggesting to the driver to let the AD system handle parking of the vehicle 100 in the parking spot 1224. In some implementations, the UI 114 may show the parking spot 1224 where the AD system is currently suggesting to the driver to park the vehicle 100. For example, the UI 114 may present a map of at least a portion of the road 1216 on which the parking spot 1224 is shown; or the UI 114 may show the parking spot 1224 by AR technology applied to a captured image of the surroundings of the vehicle 100. Other approaches may be used.

[0187] The driver is in control 116 (3) Before the vehicle 100 reaches the parking spot 1224, the driver may accept the prompt by activating control 116. (3) For example, control 116 may be available. (3) The vehicle 100 does not have to pull up beside or begin to enter the parking spot 1224 for the control 116 to be activated. (3) If the driver accepts the prompt by activating control 116, the AD system may take over control of some or all of the vehicle 100 and steer the vehicle 100 into the parking spot 1224. This is because, in response to the prompt, the driver may actuate control 116. (3)11, such input is generated before the vehicle enters the parking spot 1224. The AD system can then control the vehicle in response to the input to park the vehicle in the parking spot 1224. For example, the AD system can perform this maneuver using a trajectory for the vehicle 100 that the AD system generates upon detecting the parking spot 1224. This functionality can ensure that convenient parking assistance is provided to the driver, both to identify an available parking spot and to take over steering the vehicle 100 to an available parking spot if the driver so desires at any point during operation. To name just a few examples, the driver can control the GUI of the vehicle 100, the GUI of the portable electronic device 116, by giving audio commands (e.g., using the audio interface 1132 in FIG. 11 or the portable electronic device) and / or by making gestures (e.g., using the visual interface 1134 in FIG. 11 or the portable electronic device). (3) can be activated.

[0188] If there is a handover to the AD system, the driver can exit the vehicle 100 before or after control is given to the AD system, or can remain in the vehicle during the maneuver. (3) After the vehicle 100 is available, the driver stops the vehicle 100, exits the vehicle onto the road 1216, and activates a remote control device (e.g., control 116 for actuation). (3) 116) using a portable electronic device such as a smartphone having an app that presents (3) As another example, after stopping the vehicle 100, the driver can open the door of the vehicle 100 and then activate the control 116 using the GUI inside the vehicle 100. (3)The AD system may receive driver generated input and may wait to execute a parking maneuver until it detects that the doors of the vehicle 100 are closed (at which point the driver may have exited the vehicle 100). Other approaches may be used.

[0189] To name just two examples, the driver of the vehicle 100 may control 116 (3) If the driver does not activate the parking spot 1224, the driver may manually park in the parking spot 1224 or may forgo parking along the road 1216. If the driver begins to park manually (e.g., at the parking spot 1224), one or more other options may become available to the driver. In some implementations, "finish my parking" assistance may be offered, for example, along the lines of the examples described with reference to FIGS. 3A-3D. In some implementations, "touch up my parking" assistance may be offered, for example, along the lines of the examples described with reference to FIGS. 5A-5D.

[0190] The example in Figure 12C will be described with reference to a garage 1226. The vehicle 100 is traveling towards the garage 1226, as indicated diagrammatically by arrow 1228. The vehicle 100 is now essentially controlled by the driver. For example, the driver is now steering the vehicle 100 using the steering wheel, gear selector, accelerator and brake pedals while approaching the garage 1226.

[0191] The AD system of the vehicle 100 continuously monitors the surroundings of the vehicle 100 using at least one sensor to detect whether any parking spots are available. The use of sensors is illustrated here diagrammatically by signals 1230 near the vehicle 100, but the sensors do not necessarily generate any radiation or other emissions to probe the surroundings. As another example, the vehicle 100 may use geolocation techniques (e.g., satellite and / or other wireless navigation) to detect that a garage 1226 is nearby, where the garage 1226 is known to be a frequent parking spot for the vehicle 100 (e.g., by being the driver's home garage).

[0192] As mentioned above, at this point in the example, vehicle 100 may be controlled only by the driver. Nevertheless, for any parking spot that the AD system detects, the AD system may plan a trajectory for vehicle 100. This may ensure that the AD system may provide a fluid handover from the driver at any time if the driver so chooses. Here, for example, if the driver wishes to park vehicle 100 in garage 1226, the AD system plans a trajectory for vehicle 100 to enter there.

[0193] Here, the AD system of the vehicle 100 may have detected a parking spot 1232 in the garage 1226 and a control 116 in the UI 114 (4) The parking spot 1232 may have been detected (e.g., based on sensor output and / or geolocation) prior to the vehicle 100 arriving at the parking spot 1232. The AD system may generate a prompt to the driver regarding one or more detected parking spots. For example, control 116 (4)may be a GUI button labeled "PARK" suggesting to the driver to let the AD system handle parking of the vehicle 100 in the parking spot 1232. In some implementations, the UI 114 may show the parking spot 1232 where the AD system is currently suggesting to the driver to park the vehicle 100. For example, the UI 114 may present a map of at least a portion of the garage 1226 in which the parking spot 1232 is shown; or the UI 114 may show the parking spot 1232 by AR technology applied to a captured image of the surroundings of the vehicle 100. Other approaches may be used.

[0194] The driver is in control 116 (4) Before the vehicle 100 reaches the parking spot 1232, the driver may accept the prompt by activating control 116. (4) For example, control 116 may be available. (4) The vehicle 100 does not have to pull up beside or begin to enter the parking spot 1232 for the control 116 to be activated. (4) If the driver accepts the prompt by activating control 116, the AD system may take over control of some or all of the vehicle 100 and steer the vehicle 100 into the parking spot 1232. This is because, in response to the prompt, the driver may actuate control 116. (4)11, such input is generated before the vehicle enters the parking spot 1232. The AD system can then control the vehicle in response to the input to park the vehicle in the parking spot 1232. For example, the AD system can perform this maneuver using a trajectory for the vehicle 100 that the AD system generates upon detecting the parking spot 1232. This functionality can ensure that convenient parking assistance is provided to the driver, both to identify an available parking spot and to take over steering the vehicle 100 to an available parking spot if the driver so desires at any point during operation. To name just a few examples, the driver can control the GUI of the vehicle 100, the GUI of the portable electronic device 116, by giving audio commands (e.g., using the audio interface 1132 or the portable electronic device in FIG. 11) and / or by making gestures (e.g., using the visual interface 1134 or the portable electronic device in FIG. 11). (4) can be activated.

[0195] If there is a handover to the AD system, the driver can exit the vehicle 100 before or after control is given to the AD system, or can remain in the vehicle during the maneuver. (4) After the vehicle 100 is available, the driver stops the vehicle 100, exits the vehicle into the garage 1226, and accesses a remote control device (e.g., control 116 for actuation). (4) 116) using a portable electronic device such as a smartphone having an app that presents (4) As another example, after stopping the vehicle 100, the driver can open the door of the vehicle 100 and then activate the control 116 using the GUI inside the vehicle 100. (4)The AD system may receive driver generated input and may wait to execute a parking maneuver until it detects that the doors of the vehicle 100 are closed (at which point the driver may have exited the vehicle 100). Other approaches may be used.

[0196] To name just two examples, the driver of the vehicle 100 may control 116 (4) If the driver does not activate the parking spot 1232, the driver can manually park in the parking spot 1232 or can forgo parking in the garage 1226. If the driver begins to manually park in the parking spot 1232, one or more other options may become available to the driver. In some implementations, "finish my parking" assistance may be offered, for example, along the lines of the examples described with reference to FIGS. 3A-3D. In some implementations, "touch up my parking" assistance may be offered, for example, along the lines of the examples described with reference to FIGS. 5A-5D.

[0197] FIG. 13 illustrates an example architecture of a computing device 1300 that may be used to implement aspects of the disclosure, including any of the systems, apparatus, and / or techniques described herein or any other systems, apparatus, and / or techniques that may be utilized in various possible embodiments.

[0198] The computing device illustrated in FIG. 13 may be used to execute the operating system, application programs, and / or software modules (including software engines) described herein.

[0199] The computing device 1300 includes at least one processing device 1302 (e.g., processor), such as a central processing unit (CPU) in some embodiments. A variety of processing devices are available from a variety of manufacturers, such as Intel or Advanced Micro Devices. In this example, the computing device 1300 also includes a system memory 1304 and a system bus 1306 that couples various system components including the system memory 1304 to the processing device 1302. The system bus 1306 is one of a number of types of bus structures that may be used, including, but not limited to, a memory bus or memory controller; a peripheral bus; and a local bus, using any of a variety of bus architectures.

[0200] Examples of computing devices that may be implemented using computing device 1300 include a desktop computer, a laptop computer, a tablet computer, a mobile computing device (such as a smartphone, a touchpad mobile digital device, or other mobile device), or other device configured to process digital instructions.

[0201] The system memory 1304 includes a read-only memory 1308 and a random access memory 1310. A basic input / output system 1312, containing the basic routines that act to transfer information within the computing device 1300, such as during start-up, may be stored in the read-only memory 1308.

[0202] In some embodiments, computing device 1300 also includes a secondary storage device 1314, such as a hard disk drive for storing digital data. The secondary storage device 1314 is connected to the system bus 1306 by a secondary storage interface 1316. The secondary storage device 1314 and its associated computer-readable media provide non-volatile and non-transitory storage of computer-readable instructions (including application programs and program modules), data structures, and other data for computing device 1300.

[0203] Although the exemplary environment described herein employs a hard disk drive as the secondary storage device, other embodiments use other types of computer-readable storage media. Examples of these other types of computer-readable storage media include magnetic cassettes, flash memory cards, digital video disks, Bernoulli cartridges, compact disk read-only memories, digital versatile disk read-only memories, random access memories, or read-only memories. Some embodiments include non-transitory media. For example, a computer program product may be tangibly embodied in a non-transitory storage medium. Furthermore, such computer-readable storage media may include local storage or cloud-based storage.

[0204] A number of program modules may be stored in secondary storage device 1314 and / or system memory 1304, including operating system 1318, one or more application programs 1320, other program modules 1322 (such as the software engines described herein), and program data 1324. Computing device 1300 may utilize any suitable operating system, such as Microsoft Windows®, Google Chrome® OS, Apple® OS, Unix®, or Linux® and variants, and any other operating system suitable for a computing device. Other examples may include Microsoft®, Google®, or Apple® operating systems used in a tablet computing device, or any other suitable operating system.

[0205] In some embodiments, a user provides input to the computing device 1300 via one or more input devices 1326. Examples of input devices 1326 include a keyboard 1328, a mouse 1330, a microphone 1332 (e.g., for voice and / or other audio input), a touch sensor 1334 (such as a touchpad or touch-sensitive display), and a gesture sensor 1335 (e.g., for gesture input). In some implementations, the input devices 1326 provide presence, proximity, and / or motion-based detection. In some implementations, a user may walk into their home, which may trigger an input to the processing device. For example, the input devices 1326 may then facilitate an automated experience for the user. Other embodiments include other input devices 1326. The input devices may be connected to the processing device 1302 through an input / output interface 1336 coupled to the system bus 1306. These input devices 1326 may be connected by any number of input / output interfaces (such as a parallel port, a serial port, a game port, or a universal serial bus). In some contemplated embodiments, wireless communication between the input devices 1326 and the input / output interfaces 1336 is also possible, including infrared, BLUETOOTH® wireless technology, 802.11a / b / g / n, cellular, ultra-wideband (UWB), ZigBee®, or other radio frequency communication systems, to name just a few.

[0206] In this exemplary embodiment, a display device 1338, such as a monitor, liquid crystal display device, light emitting diode display device, projector, or touch-sensitive display device, is also connected to the system bus 1306 via an interface, such as a video adapter 1340. In addition to the display device 1338, computing device 1300 may also include various other peripheral devices (not shown), such as speakers or printers.

[0207] The computing device 1300 may be connected to one or more networks via a network interface 1342. The network interface 1342 may provide for wired and / or wireless communication. In some implementations, the network interface 1342 may include one or more antennas for transmitting and / or receiving wireless signals. When used in a local area networking environment or a wide area networking environment (such as the Internet), the network interface 1342 may include an Ethernet interface. Other possible embodiments use other communication devices. For example, some embodiments of the computing device 1300 include a modem for communicating across the network.

[0208] Computing device 1300 may include at least some form of computer-readable media. Computer-readable media includes any available media that can be accessed by computing device 1300. By way of example, computer-readable media include computer-readable storage media and computer-readable communication media.

[0209] Computer-readable storage media include volatile and nonvolatile, removable and non-removable media implemented in any device configured to store information, such as computer-readable instructions, data structures, program modules, or other data. Computer-readable storage media include, but are not limited to, random access memory, read-only memory, electrically erasable programmable read-only memory, flash memory, or other memory technology, compact disc read-only memory, digital versatile disk, or other optical storage, magnetic cassettes, magnetic tapes, magnetic disk storage, or other magnetic storage devices, or any other medium that can be used to store the desired information and that can be accessed by the computing device 1300.

[0210] Typically, computer-readable communication media embodies computer-readable instructions, data structures, program modules, or other data in a modulated data signal, such as a carrier wave or other transport mechanism, and includes any information delivery media. The term "modulated data signal" refers to a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. By way of example, computer-readable communication media includes wired media, such as a wired network or direct-wired connection, and wireless media, such as acoustic, radio frequency, infrared and other wireless media. Combinations of any of the above are also included within the scope of computer-readable media.

[0211] The computing device illustrated in FIG. 13 is also an example of a programmable electronic apparatus that may include one or more such computing devices, and when multiple computing devices are included, such computing devices may be coupled together to a suitable data communications network to collectively perform various functions, methods, or operations disclosed herein.

[0212] The terms "substantially" and "about" used throughout this specification are used to describe and report small variations, such as those due to processing variations. For example, they can refer to less than or equal to ±5%, such as less than or equal to ±2%, such as less than or equal to ±1%, such as less than or equal to ±0.5%, such as less than or equal to ±0.2%, such as less than or equal to ±0.1%, such as less than or equal to ±0.05%. Also, as used herein, indefinite articles such as "a" or "an" mean "at least one".

[0213] It should be understood that all combinations of the above concepts, and additional concepts discussed in more detail below (to the extent such concepts are not mutually inconsistent), are contemplated as being part of the inventive subject matter disclosed herein. In particular, all combinations of claimed subject matter appearing at the end of this disclosure are contemplated as being part of the inventive subject matter disclosed herein.

[0214] A number of implementations have been described. Nevertheless, it will be understood that various modifications may be made without departing from the spirit and scope of the present specification.

[0215] Additionally, the logic flows depicted in the figures do not require the particular order shown, or sequential order, to achieve desirable results. Additionally, other processes may be provided or processes may be eliminated from the described flows, and other components may be added to or removed from the described systems. Accordingly, other implementations are within the scope of the following claims.

[0216] While certain features of the described implementations have been shown as described herein, many modifications, substitutions, changes, and equivalents will now occur to those skilled in the art. It should therefore be understood that the appended claims are intended to cover all such modifications and changes that fall within the scope of the implementations. It should be understood that they have been presented by way of example only, and not by way of limitation, and that various changes in form and details may be made. Any part of the apparatus and / or method described herein may be combined in any combination, except in mutually exclusive combinations. The implementations described herein may include various combinations and / or subcombinations of the functions, components, and / or features of the different implementations described. [Item 1] a driver assistance (AD) system using sensors to continuously monitor the surroundings of the vehicle controlled by the driver; the AD system detecting available parking spots using the sensors; In response to the AD system detecting the parking spot, planning a trajectory for the vehicle to park at the parking spot; and generating a prompt to the driver to have the AD system handle parking of the vehicle at the parking spot in response to the AD system detecting the parking spot, the prompt being executed before the vehicle reaches the parking spot. A computer-implemented method comprising: [Item 2] 2. The computer-implemented method of claim 1, wherein when the AD system detects the parking spot, the parking spot is substantially parallel to a driving path of the vehicle. [Item 3] 2. The computer-implemented method of claim 1, wherein when the AD system detects the parking spot, the parking spot is substantially perpendicular to a driving path of the vehicle. [Item 4] 2. The computer-implemented method of claim 1, wherein when the AD system detects the parking spot, the parking spot is substantially oblique to a driving path of the vehicle. [Item 5] receiving driver-generated input in response to the prompt, the input being generated before the vehicle enters the parking spot; and the AD system controlling the vehicle in response to the input to park the vehicle in the parking spot. Item 1. The computer-implemented method of item 1, further comprising: [Item 6] 6. The computer-implemented method of claim 5, wherein the driver exits the vehicle before the AD system parks the vehicle in the parking spot. [Item 7] The driver begins to move the vehicle into the parking spot without accepting the prompt by the AD system, and the method comprises: 2. The computer-implemented method of claim 1, further comprising continuing to generate the prompts to the driver during at least a portion of a vehicle maneuver in which the driver is parking the vehicle in the parking spot. [Item 8] 8. The computer-implemented method of claim 7, wherein continuing to generate the prompt indicates that the AD system is available to complete the parking of the vehicle in the parking spot at any point in time. [Item 9] receiving driver-generated input in response to the AD system continuing to generate the prompts, the input being generated before the vehicle comes to rest in the parking spot; and the AD system controlling the vehicle in response to the input to park the vehicle in the parking spot. 8. The computer-implemented method of claim 7, further comprising: [Item 10] The driver has parked the vehicle in the parking spot without accepting the prompt by the AD system, the vehicle has a first orientation within the parking spot, and the method includes: executing a motion plan, the AD system to move the vehicle to a second orientation within the parking spot; receiving, by the AD system, an input for modifying the first orientation generated by a user; and 8. The computer-implemented method of claim 7, further comprising: in response to the input, the AD system controlling the vehicle to assume the second orientation within the parking spot according to the motion plan. [Item 11] a driver assistance (AD) system using a sensor to determine that a vehicle controlled by a driver has begun to enter a parking spot; in response to the AD system determining that the vehicle has begun to enter the parking spot, generating a prompt to the driver to cause the AD system to handle parking of the vehicle in the parking spot; receiving, by the AD system, input generated by the driver in response to the prompt; and the AD system controlling the vehicle in response to the input to finish parking the vehicle in the parking spot. A computer-implemented method comprising: [Item 12] Item 12. The computer-implemented method of item 11, wherein the prompt includes at least one of a visual message or an audio message. [Item 13] Item 12. The computer-implemented method of item 11, wherein the input is generated using a touch screen of the vehicle. [Item 14] Item 12. The computer-implemented method of item 11, wherein generating the prompt indicates that the AD system is available to complete the parking of the vehicle in the parking spot at any point in time. [Item 15] Item 12. The computer-implemented method of item 11, wherein the driver exits the vehicle before the AD system controls the vehicle to finish parking the vehicle in the parking spot. [Item 16] Item 12. The computer-implemented method of item 11, wherein controlling the vehicle to finish parking the vehicle in the parking spot includes parking the vehicle in an orientation within the parking spot, the orientation being obtained using a machine learning component that analyzes previous instances in which the vehicle has been parked. [Item 17] a driving assistance (AD) system determining, using a sensor, that the vehicle is stationary in a first orientation relative to a parking spot; executing a motion plan, the AD system to move the vehicle to a second orientation relative to the parking spot; generating a prompt to a user to cause the AD system to modify the first orientation in response to the AD system determining that the vehicle is stationary; receiving, by the AD system, input generated by the user in response to the prompt; and and in response to the input, the AD system controls the vehicle to assume the second orientation relative to the parking spot according to the motion plan. A computer-implemented method comprising: [Item 18] 20. The computer-implemented method of claim 17, wherein the user is located outside the vehicle while generating the input. [Item 19] Item 18. The computer-implemented method of item 17, wherein the user is located outside the vehicle while the AD system controls the vehicle to assume the second orientation within the parking spot. [Item 20] The vehicle is initially brought into the parking spot at the first orientation under driver control, and the method includes: the AD system using the sensor to determine that the vehicle has begun to enter the parking spot before determining that the vehicle is stationary; and 18. The computer-implemented method of claim 17, further comprising: in response to the AD system determining that the vehicle has begun to enter the parking spot, generating a prompt to the driver to have the AD system handle parking of the vehicle in the parking spot; and the driver not accepting the prompt before parking the vehicle in the first orientation in the parking spot. [Item 21] Item 18. The computer-implemented method of item 17, wherein the AD system obtains the second orientation using a machine learning component that analyzes previous instances in which the vehicle was parked.

Claims

1. a driver assistance (AD) system using sensors to continuously monitor the surroundings of a vehicle controlled by a driver; the AD system detecting available parking spots using the sensors; in response to the AD system detecting the parking spot, planning a trajectory for the vehicle to park at the parking spot; 1. A computer-implemented method comprising: in response to the AD system detecting the parking spot, generating a prompt to the driver to cause the AD system to handle parking of the vehicle at the parking spot using the planned trajectory, the prompt being executed before the vehicle reaches the parking spot; The driver begins to move the vehicle into the parking spot without accepting the prompt by the AD system, and the computer-implemented method further comprises: The computer-implemented method further comprising: continuing to generate a further prompt, different from the prompt, to the driver during at least a portion of a vehicle maneuver in which the driver is parking the vehicle in the parking spot, the further prompt notifying the driver that a parking maneuver utilizing the AD system is available to take over a manual parking maneuver by the driver.

2. The further prompt: As the parking maneuver using the AD system that takes over the manual parking maneuver by the driver, the AD system is available to complete the manual parking maneuver in progress by the driver; or The computer-implemented method of claim 1 , further comprising notifying the driver that the AD system is available to correct the completed parking maneuver.

3. 2. The computer-implemented method of claim 1, wherein when the AD system detects the parking spot, the parking spot is substantially parallel to a driving path of the vehicle.

4. 2. The computer-implemented method of claim 1, wherein when the AD system detects the parking spot, the parking spot is substantially perpendicular to a driving path of the vehicle.

5. 2. The computer-implemented method of claim 1, wherein when the AD system detects the parking spot, the parking spot is substantially oblique to a driving path of the vehicle.

6. receiving driver-generated input in response to the prompt, the input being generated before the vehicle enters the parking spot; and the AD system responding to the input by controlling the vehicle to park the vehicle in the parking spot. The computer-implemented method of claim 1 , further comprising:

7. The computer-implemented method of claim 6 , wherein the driver exits the vehicle before the AD system parks the vehicle in the parking spot.

8. 2. The computer-implemented method of claim 1, wherein continuing to generate further prompts indicates that the AD system is available to complete the parking of the vehicle in the parking spot at any time.

9. A driver assistance (AD) system using sensors to continuously monitor the surroundings of a vehicle controlled by a driver; the AD system detecting available parking spots using the sensors; in response to the AD system detecting the parking spot, planning a trajectory for the vehicle to park at the parking spot; 1. A computer-implemented method comprising: in response to the AD system detecting the parking spot, generating a prompt to the driver to have the AD system handle parking of the vehicle at the parking spot using the planned trajectory, the prompt being executed before the vehicle reaches the parking spot; The driver begins to move the vehicle into the parking spot without accepting the prompt by the AD system, and the computer-implemented method further comprises: continuing to generate the prompts to the driver during at least a portion of a vehicle maneuver during which the driver is parking the vehicle in the parking spot; The computer-implemented method, wherein continuing to generate the prompt indicates that the AD system is available to complete the parking of the vehicle in the parking spot at any point in time.

10. receiving, by the AD system, driver-generated input in response to the AD system continuing to generate the further prompts, the input being generated before the vehicle comes to rest in the parking spot; and the AD system responding to the input by controlling the vehicle to park the vehicle in the parking spot. The computer-implemented method of claim 1 , further comprising:

11. The driver completes parking the vehicle in the parking spot without accepting the prompt by the AD system, the vehicle having a first orientation within the parking spot, and the computer-implemented method further comprises: executing a motion plan with the AD system to move the vehicle to a second orientation within the parking spot; receiving an input by the AD system to modify the first orientation generated by a user; and 2. The computer-implemented method of claim 1, further comprising: in response to the input, the AD system controlling the vehicle to assume the second orientation within the parking spot according to the motion plan.

12. A driver assistance (AD) system using sensors to continuously monitor the surroundings of a vehicle controlled by a driver; the AD system detecting available parking spots using the sensors; in response to the AD system detecting the parking spot, planning a trajectory for the vehicle to park at the parking spot; 1. A computer-implemented method comprising: in response to the AD system detecting the parking spot, generating a prompt to the driver to have the AD system handle parking of the vehicle at the parking spot using the planned trajectory, the prompt being executed before the vehicle reaches the parking spot; The driver begins to move the vehicle into the parking spot without accepting the prompt by the AD system, and the computer-implemented method further comprises: continuing to generate the prompts to the driver during at least a portion of a vehicle maneuver during which the driver is parking the vehicle in the parking spot; The driver completes parking the vehicle in the parking spot without accepting the prompt by the AD system, the vehicle having a first orientation within the parking spot, and the computer-implemented method further comprises: executing a motion plan with the AD system to move the vehicle to a second orientation within the parking spot; receiving an input by the AD system to modify the first orientation generated by a user; and The computer-implemented method further comprising: in response to the input, the AD system controlling the vehicle to assume the second orientation within the parking spot according to the motion plan.

13. determining, using a sensor, that a vehicle under full manual control by a driver has begun to enter a parking spot; generating a prompt to the driver to have the AD system handle parking of the vehicle in the parking spot in response to the AD system determining that the vehicle has begun to enter the parking spot, where generating the prompt indicates that the AD system is available to complete the parking of the vehicle in the parking spot at any time; receiving, by the AD system, input generated by the driver in response to the prompt; and the AD system responding to the input by controlling the vehicle to finish parking the vehicle in the parking spot. A computer-implemented method comprising:

14. The computer-implemented method of claim 13 , wherein the prompt comprises at least one of a visual message or an audio message.

15. The computer-implemented method of claim 13 , wherein the input is generated using a touch screen of the vehicle.

16. 14. The computer-implemented method of claim 13, wherein the driver exits the vehicle before the AD system controls the vehicle to finish parking the vehicle in the parking spot.

17. 14. The computer-implemented method of claim 13, wherein controlling the vehicle to finish parking the vehicle in the parking spot comprises parking the vehicle in an orientation within the parking spot, the orientation being obtained using a machine learning component that analyzes previous instances in which the vehicle has been parked.

18. determining, using a sensor, that a vehicle under full manual control by a driver has begun to enter a parking spot; generating a first prompt to the driver to have the AD system handle parking of the vehicle in the parking spot in response to the AD system determining that the vehicle has begun to enter the parking spot, where generating the first prompt indicates that the AD system is available to complete the parking of the vehicle in the parking spot at any time, and the driver does not accept the first prompt before parking the vehicle; determining, by the AD system using the sensor and after generating the first prompt, that the vehicle is stationary in a first orientation of the vehicle relative to the parking spot, where the vehicle is brought into the first orientation within the parking spot under control of the driver; executing a motion plan with the AD system to move the vehicle to a second orientation of the vehicle relative to the parking spot; generating a second prompt to a user to cause the AD system to modify the first orientation of the vehicle in response to the AD system determining that the vehicle is stationary; receiving, by the AD system, input generated by the user in response to the second prompt; and and in response to the input, the AD system controls the vehicle to assume the second orientation of the vehicle relative to the parking spot according to the motion plan. A computer-implemented method comprising:

19. 20. The computer-implemented method of claim 18, wherein the user is located outside the vehicle while generating the input.

20. 20. The computer-implemented method of claim 18, wherein the user is located outside the vehicle while the AD system controls the vehicle to assume the second orientation within the parking spot.

21. 20. The computer-implemented method of claim 18, wherein the AD system obtains the second orientation using a machine learning component that analyzes previous instances in which the vehicle was parked.

22. 20. The computer-implemented method of claim 18, further comprising providing a seamless transition between (i) an initial offer by the first prompt to have the AD system handle the parking of the vehicle in the parking spot and (ii) a subsequent offer by the second prompt to have the AD system correct the first orientation of the vehicle.

Citation Information

Patent Citations

  • Device for vehicle, vehicle control system, and vehicle control method

    JP2016007959A

  • Information processing device, information processing method, and program

    JP2017140890A

  • Vehicle device, vehicle control system, and vehicle control method

    JP2018131201A

  • Parking support system

    JP2021095113A

  • Driving assistance device

    WO2017072894A1