Vehicle-mounted traffic assistance

Abstract

A system comprising a processor; and a memory comprising instructions for execution by the processor to: guide a vehicle based on determining a first lane mark and a second lane mark, wherein a lane mark is a mathematical description of a road lane mark applied to a road to mark a traffic lane; determine a missing first lane mark or a second lane mark; and guide the vehicle for a determined period of time based on the remaining first lane mark or the second lane mark.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the field of autonomous vehicles, and more specifically to systems and methods for piloting a vehicle using lane markings. BACKGROUND

[0002] A vehicle can be equipped to operate in both an autonomous piloting mode and a passenger piloting mode. The vehicle can be equipped with computing devices, networks, sensors, and controllers to acquire information about the vehicle's environment and pilot the vehicle based on the information. Safe and comfortable piloting of the vehicle can depend on acquiring accurate and timely information about the vehicle's environment. The computing devices, networks, sensors, and controllers can be equipped to analyze their performance, detect when information is not being acquired in an accurate and timely manner, and take corrective action, including notifying the vehicle's passengers to relinquish autonomous control or bring the vehicle to a stop. SUMMARY

[0003] A vehicle can be equipped to operate in both an autonomous piloting mode and a passenger piloting mode. For semi-autonomous or fully autonomous modes, it is meant the mode of operation in which the vehicle can be piloted by a computing device as part of a vehicle information system with sensors and controllers. The vehicle can be occupied or unoccupied, but in either case, the vehicle can be piloted without passenger assistance. For purposes of this disclosure, autonomous mode is defined as follows: each of vehicle propulsion (e.g., via a powertrain system including an internal combustion engine and / or electric motor), braking, and steering is controlled by one or more vehicle computers; in semi-autonomous mode, the vehicle computer controls one or more of vehicle propulsion, braking, and steering. In a non-autonomous vehicle, none of these are controlled by a computer.

[0004] Disclosed herein is a method comprising piloting a vehicle based on determining a first lane mark and a second lane mark, wherein the lane mark is a mathematical description of a road lane marking applied to a roadway to mark a traffic lane. Determining a missing first lane mark or a second lane mark and piloting the vehicle for a determined period of time can be based on a remaining first lane mark or second lane mark and determining a steerable path polynomial confidence, wherein the steerable path polynomial confidence is a probability that the vehicle will follow the path accurately, wherein the lane mark is missing due to a roadway entrance or exit ramp. Determining the first lane mark and the second lane mark can include processing one or more acquired images with a Hough transform to determine a lane mark confidence.

[0005] The steerable path polynomial confidence can be based on determining a lane marker confidence, determining a vehicle orientation relative to the steerable path polynomial, determining a steerable path polynomial curvature, and determining inertial navigation data. The steerable path polynomial confidence can be based on determining a vehicle orientation relative to the steerable path polynomial, determining inertial navigation data, and determining a steerable path polynomial curvature. The vehicle orientation is determined based on the inertial navigation data. The steerable path polynomial curvature is determined based on the remaining first lane marker or second lane marker, the vehicle orientation, and the inertial navigation data. The time period can be based on the steerable path polynomial curvature and the lane marker confidence.

[0006] The time period can be based on the steerable path polynomial and a vehicle speed or a predetermined time. The vehicle speed can be based on the inertial navigation data. The time period can be based on a lateral acceleration a limit applied to the steerable path polynomial. The lane marker confidence can be determined by comparing a Hough transform result to the input video data. The lane marker confidence can be determined by comparing a position of the lane marker to a position of the steerable path polynomial to determine if the lane marker is parallel to the steerable path polynomial and at an expected distance from the steerable path polynomial. If the lane marker is not parallel to the steerable path polynomial and is not at the expected distance from the steerable path polynomial, then the lane marker confidence can be low.

[0007] A computer readable medium is also disclosed, the computer readable medium storing program instructions for performing some or all of the above method steps. A computer programmed to perform some or all of the above method steps is also disclosed, the computer comprising a computer device programmed to guide a vehicle based on determining a first lane marker and a second lane marker, wherein the lane marker is a mathematical description of a road lane marker applied to a road to mark a traffic lane. The computer can also be programmed to determine a missing first lane marker or second lane marker, and the time period for guiding the vehicle can be based on the remaining first lane marker or second lane marker and determining a steerable path polynomial confidence, wherein the steerable path polynomial confidence is a probability that the vehicle will accurately follow the path, wherein the lane marker is missing due to a road entrance or exit ramp. The computer can also be programmed to determine a first lane marker and a second lane marker, the determining a first lane marker and a second lane marker comprising processing one or more acquired images with a Hough transform to determine a lane marker confidence.

[0008] The steerable path polynomial confidence can be based on determining a lane marker confidence, determining a vehicle orientation relative to the steerable path polynomial, determining a steerable path polynomial curvature, and determining inertial navigation data. The steerable path polynomial confidence can be based on determining a vehicle orientation relative to the steerable path polynomial, determining inertial navigation data, and determining a steerable path polynomial curvature. The vehicle orientation is determined based on the inertial navigation data. The steerable path polynomial curvature is determined based on the remaining first lane marker or second lane marker, the vehicle orientation, and the inertial navigation data. The time period can be based on the steerable path polynomial curvature and the lane marker confidence.

[0009] The computer can also be programmed to determine a time period based on the steerable path polynomial and a vehicle speed or a predetermined time. The vehicle speed can be based on the inertial navigation data. The time period can be based on a lateral acceleration a limit applied to the steerable path polynomial. The lane marker confidence can be determined by comparing a Hough transform result to the input video data. The lane marker confidence can be determined by comparing a location of the lane marker to a location of the steerable path polynomial to determine if the lane marker is parallel to the steerable path polynomial and at an expected distance from the steerable path polynomial. The lane marker confidence can be low if the lane marker is not parallel to the steerable path polynomial and is not at the expected distance from the steerable path polynomial. BRIEF DESCRIPTION OF DRAWINGS

[0010] Figure 1 is a block diagram of an example vehicle.

[0011] Figure 2 is a diagram of an example traffic scene image with lane markers.

[0012] Figure 3 is a diagram of an example traffic scene image with lane markers.

[0013] Figure 4 is a diagram of an example traffic scene with lane markers.

[0014] Figure 5 is a diagram of an example traffic scene with lane markers.

[0015] Figure 6 is a flowchart of an example process of guiding a vehicle.

[0016] Figure 7 is a flowchart of an example process of determining a time extension for guiding a vehicle.

[0017] Figure 8 is a flowchart of an example process of determining a time extension for guiding a vehicle. DETAILED DESCRIPTION

[0018] Figure 1 is a diagram of a vehicle information system 100 that includes a vehicle 110 that is operable in an autonomous ("autonomous" by itself refers to "fully autonomous" in this disclosure) mode and a passenger-guided (also referred to as non-autonomous) mode. The vehicle 110 also includes one or more computing devices 115 for performing computations during autonomous operation for guiding the vehicle 110. The computing devices 115 can receive information about operation of the vehicle from sensors 116.

[0019] The computing devices 115 include a processor and a memory such as are known. In addition, the memory includes one or more forms of computer-readable media, and stores instructions that are executable by the processor to perform various operations including as disclosed herein. For example, the computing devices 115 can include programming to operate one or more of vehicle brakes, propulsion (e.g., to control acceleration of the vehicle 110 by controlling one or more of an internal combustion engine, an electric motor, a hybrid engine, etc.), steering, climate control, interior and / or exterior lights, etc., as well as to determine whether and when the computing devices 115 (as opposed to a human operator) control such operations.

[0020] The computing devices 115 can include more than one computing device (e.g., controllers (e.g., a powertrain controller 112, a brake controller 113, a steering controller 114, etc.) included in the vehicle 110 for monitoring and / or controlling various vehicle components, etc.) or be coupled to the more than one computing device, e.g., via a vehicle communication bus as further described below. The computing devices 115 are generally arranged for communication over a vehicle communication network (e.g., including a bus in the vehicle 110 such as a controller area network (CAN) bus, etc.); the vehicle 110 network can additionally or alternatively include wired or wireless communication mechanisms such as are known, e.g., Ethernet or other communication protocols.

[0021] Via the vehicle network, the computing devices 115 can transmit messages to and / or receive messages from various devices in the vehicle (e.g., controllers, actuators, sensors including the sensors 116, etc.). Alternatively or additionally, in cases where the computing devices 115 actually include multiple devices, the vehicle communication network can be used to communicate between the devices represented as the computing devices 115 in this disclosure. In addition, as mentioned above, various controllers or sensing elements such as the sensors 116 can provide data to the computing devices 115 via the vehicle communication network.

[0022] Furthermore, the computing device 115 may be configured to communicate with a remote server computer 120 (e.g., a cloud server) via a network 130 through a vehicle-to-infrastructure (V-to-I) interface 111, the network 130 of which may utilize various wired and / or wireless networking technologies (e.g., cellular, etc.) as described below. (And wired and / or wireless packet networks). The computing device 115 can be configured to communicate with other vehicles 110 via a V-to-I interface 111 using a vehicle-to-vehicle (V-to-V) network formed between nearby vehicles based on a specific infrastructure or through an infrastructure-based network. The computing device 115 also includes, for example, known non-volatile memory. The computing device 115 can record information by storing it in the non-volatile memory for later retrieval and transmission to a server computer 120 or a user mobile device 160 via the vehicle communication network and the vehicle-to-infrastructure (V-to-I) interface 111.

[0023] As mentioned above, typically included in the instructions stored in memory and executable by the processor of computing device 115 are instructions programmed to operate (e.g., braking, steering, propulsion, etc.) one or more vehicle 110 components without human intervention. Using data received in computing device 115 (e.g., sensor data from sensor 116, server computer 120, etc.), computing device 115 can make various determinations and / or control various vehicle 110 components and / or operations without driver operation of vehicle 110. For example, computing device 115 may include programming to regulate vehicle 110 operating behaviors (such as speed, acceleration, deceleration, steering, etc.) and strategic behaviors (such as distance and / or time between vehicles, lane changes, minimum clearance between vehicles, minimum left-turn crossing distance, arrival time at a specific location, and minimum arrival time for crossing an intersection (without indicator lights).

[0024] A controller (as used herein) includes a computing device typically programmed to control a particular vehicle subsystem. Examples include a powertrain controller 112, a brake controller 113, and a steering controller 114. A controller may be an electronic control unit (ECU), such as those known, which may include additional programming as described herein. A controller may be communicatively connected to a computing device 115 and receive instructions from the computing device 115 to actuate subsystems according to those instructions. For example, the brake controller 113 may receive instructions from the computing device 115 to operate the brakes of vehicle 110.

[0025] The one or more controllers 112, 113, 114 of the vehicle 110 can include known electronic control units (ECUs) or the like, including, by way of non-limiting examples, one or more powertrain controllers 112, one or more brake controllers 113, and one or more steering controllers 114. Each of the controllers 112, 113, 114 can include a respective processor and memory, as well as one or more actuators. The controllers 112, 113, 114 can be programmed and connected to a vehicle 110 communication bus, such as a controller area network (CAN) bus or a local interconnect network (LIN) bus, to receive instructions from the computer 115 and control the actuators based on the instructions.

[0026] The sensors 116 can include known various devices to provide data via the vehicle communication bus. For example, a radar fixed to a front bumper (not shown) of the vehicle 110 can provide a distance from the vehicle 110 to a next vehicle in front of the vehicle 110, a global positioning system (GPS) sensor disposed in the vehicle 110 can provide geographic coordinates of the vehicle 110. The distance provided by the radar and / or other sensors 116 and / or the geographic coordinates provided by the GPS sensor can be used by the computing device 115 to operate the vehicle 110 autonomously or semi-autonomously.

[0027] The vehicle 110 is generally a ground-based autonomous vehicle 110 having three or more wheels (e.g., a passenger car, a light-duty truck, or the like). The vehicle 110 includes one or more sensors 116, a V-to-l interface 111, a computing device 115, and one or more controllers 112, 113, 114.

[0028] The sensors 116 can be programmed to collect data related to the vehicle 110 and the environment in which the vehicle 110 is operating. By way of example, but not limitation, the sensors 116 can include, for example, altimeters, video cameras, LIDAR, radar, ultrasonic sensors, infrared sensors, pressure sensors, accelerometers, gyroscopes, temperature sensors, pressure sensors, Hall sensors, optical sensors, voltage sensors, current sensors, mechanical sensors such as switches, etc. The sensors 116 can be used to sense the environment in which the vehicle 110 is operating, for example, the sensors 116 can detect phenomena such as weather conditions (rain, outside temperature, etc.), road grade, road location (e.g., using road edges, lane markings, etc.), or the location of target objects such as nearby vehicles 110. The sensors 116 can also be used to collect data including dynamic vehicle 110 data related to the operation of the vehicle 110 such as speed, yaw rate, steering angle, engine speed, brake pressure, oil pressure, power levels applied to the controllers 112, 113, 114 in the vehicle 110, connectivity between components, and the accuracy and timeliness of the performance of components of the vehicle 110.

[0029] Figure 2 is a graph of a video image 218 of a traffic scene 200 having a road 202 on which the vehicle 110 is being guided. The computing device 115 can be configured to receive image data from a video camera 202 as one of the sensors 116. The image data can include an image such as, for example, the traffic scene image 200. The video image 218 includes a visual depiction of road lane markings 204, 206. The road lane markings 204, 206 are paint, pigment, or synthetic laminates applied to the road as geometric shapes such as, for example, solid or dashed lines and arrows in contrasting colors including, for example, white and yellow, to mark traffic lanes. A passenger guiding the vehicle 110 can see the road lane markings and thereby safely and efficiently guide the vehicle 110 to use the road to reach a destination by following the traffic lanes. A video camera sensor 116 in the vehicle 110 can acquire the video image 218 of the traffic scene 200 including the visual depiction of the road markings 204, 206. The computing device 115 can use machine vision techniques to determine lane markers 208, 210 associated with the road lane markings 204, 206 in the video image 218 of the traffic scene 200. The lane markers 208, 210 are mathematical descriptions of the geometric shapes represented by the visual depiction of the road lane markings 204, 206. The computing device 115 can combine the determined lane markers 208, 210 with steerable path parameters including a steerable path polynomial 214 to form a lane model mathematically describing the traffic lanes on the road 202 on which the vehicle 110 can safely and accurately be guided to reach a predetermined destination.

[0030] The steerable path parameters can include a steerable path polynomial 214, which is an n-degree polynomial function, where n is, for example, between 3 and 5 inclusive, that describes the predicted motion of the vehicle 110 in the plane of the road that is parallel to the location on the vehicle 110 and extends in the direction corresponding to the current direction of the vehicle 110 and includes the lateral and longitudinal accelerations applied to the vehicle 110 by the brakes, steering, and powertrain. The lane model can also include lane markings 208, 210, where the road markings 204, 206 are mathematically represented as an m-degree polynomial function, where m is between 1 and 2 inclusive, that represents a polynomial function that best fits the road lane markings 208, 210 in the video image 218.

[0031] For example, since the road lane markings 208, 210 can include a finite number of geometric shapes, such as lines, arrows, letters, numbers, and glyphs, a finite number of sizes, the Hough transform can identify these geometric shapes in the video image 218 data by first thresholding the pixel data and then projecting the video image 218 data onto a plane that is parallel to the road 202, where thresholding includes comparing the pixel data of the video image 218 to a predetermined value to determine a "1" or "0" value output. The thresholded and projected video image 218 data can be processed by a finite number of Hough transforms that map the thresholded projected video image 218 data into a parametric geometric space, allowing the computing device 115 to identify and locate the geometric shapes in the video image 218 by determining clusters of similar parameter values in the multi-dimensional parameter space by means of, for example, maximum likelihood estimation. Maximum likelihood estimation is a statistical processing technique that identifies a geometric shape as a cluster of similar points in a three-dimensional space by applying statistical measures to the points. The identified geometric shapes can be inverse transformed from the multi-dimensional parameter space back into image space and can be the root mean square difference from the input projected and thresholded video image 218 data that is computed to produce a goodness of fit measure.

[0032] The lane model can include a steerable path polynomial 214 and lane markers 208, 210 that can allow the computing device 115 to safely guide the vehicle 110 to a predetermined destination. The computing device 115 can associate a confidence level with the lane model based on the confidence level associated with the lane markers 208, 210 and the lane hints 216. The confidence level can be ranked as high or low, with high meaning a high probability that the lane markers 208, 210 accurately represent the road lane markers 204, 206 and low meaning a low probability that the lane markers 208, 210 accurately represent the road lane markers 204, 206. The confidence level can also be at an intermediate level and can be ranked as low / medium and high / medium. An example of a goodness of fit value for the lane markers 208, 210 can be experimentally produced to determine a relationship between the goodness of fit value and the confidence level. For example, this predetermined relationship between the goodness of fit value and the confidence level can be applied to the goodness of fit value determined for the Hough transform results to determine a confidence level for the Hough transform results, including the lane markers 208, 210.

[0033] For example, the lane marker 208, 210 confidence level can be determined based on the confidence level associated with the Hough transform results and the expected orientation relative to the steerable path polynomial 214. The confidence level about the lane markers 208, 210 determined in this way can be combined with the steerable path polynomial 214 curvature and the total confidence level about the vehicle 110 position discussed below with respect to Figures 3 to 5 the steerable path polynomial 214 to form a total confidence level for the lane model.

[0034] Figure 3is a diagram of a traffic scene image 300. The traffic scene image 300 includes an image of a road 302 with a highway off-ramp 304, which is a frequent occurrence on roads 302. The highway off- and on-ramps can cause the computing device 115 to determine that the confidence level for the lane model, which can be based on the confidence levels of two or more lane marks 306, 312, is insufficient to determine that the turnable path polynomial 316 has a high or moderate confidence for safely guiding the vehicle 110. In the traffic scene image 300, the lane marks 308 corresponding to the road lane marks 306 are determined by the computing device 115 to have a high confidence level due to the Hough transform results being in good agreement with the visual representation of the road lane marks 306 in the video image 318 data and the lane marks 308 being properly positioned with respect to the turnable path polynomial 316. The lane marks 312 (dashed lines) corresponding to the road lane marks 310 have a low confidence level. Although the Hough transform results are in good agreement with the visual representation of the road lane marks 310 in the video image 318 data, the lane marks 312 are not properly positioned with respect to the turnable path polynomial 316 due to the lane marks not being parallel to the turnable path polynomial 316 at the expected distance of a portion of the length of the turnable path polynomial 316. The low confidence level is associated with the lane marks 312 and the lane marks 312 can be determined to be missing from the lane model due to the lane marks 312 not being properly positioned with respect to the turnable path polynomial 316.

[0035] In Figure 3 the example, the computing device 115 is guiding the vehicle with the lane model based on two or more lane marks 308, 312 having a high confidence and the lane marks 312 are determined to be missing due to changes in the video image 318 of the road 302 caused by the motion of the vehicle 110. This can cause the computing device 115 to stop computing a new turnable path polynomial 316 and thereby place limits on the time and distance traveled in guiding the vehicle 110. In the example of a passenger readying to assist in guiding the vehicle 110, the computing device 115 can hand over control of the vehicle 110 to the passenger before the vehicle 110 reaches the end of the computed turnable path polynomial 316 because the computing device 115 is unable to safely guide the vehicle 110 to the intended destination without the turnable path polynomial 316. The computing device 115 can indicate the hand over of control to the passenger by alerting the member with a sound, like a chime, or recorded or synthesized voice, a visual alert, a haptic alert, etc.

[0036] In Figure 3In the example traffic scenario 300 shown, there can be a high probability that the missing lane marker 312 will soon reappear when the vehicle 110 travels through the exit ramp 304 and thereby acquires video image 318 data that includes both high-confidence lane markers 308, 312, due to the missing lane marker 312 being associated with the exit ramp 304. For example, although the lane marker 312 is missing for a short period of time while passing through the exit ramp 304, the computing device 115 can anticipate continuation of the high-confidence lane markers 308, 312 by adding an extension 314 to the steerable path polynomial 316 under certain conditions to allow the computing device 115 to guide the vehicle 110. The conditions for adding the extension 314 to the steerable path polynomial 316 include having at least one high-confidence lane marker 308, the vehicle 110 being positioned on or near the steerable path polynomial 316, and having a relatively straight (non-curved) steerable path polynomial 316. These conditions are discussed below with respect to Figure 4 and Figure 5 .

[0037] Figure 4 is a diagram of a traffic scenario 400 in which a computing device 115 in a vehicle 110 on a road 402 having an exit ramp 404 with lane markers 406, 408 has determined a lane marker 410 with high confidence and a lane marker 412 with low or no confidence, respectively, based on the road lane markers 406, 408. When the vehicle 110 is at point po in the traffic scenario 400, the road lane marker 420 is not visible to the sensors 116 of the vehicle 110. To avoid handing over control to the occupant, the computing device 115 can determine a curvature of a steerable path polynomial 418 on which to guide the vehicle 110 based on having at least one lane marker 410 with high or moderate confidence, and determine a vehicle 110 position based on inertial navigation data from the sensors 116, the computing device 115 adds an extension 422 to the steerable path polynomial 418 to extend availability of autonomous guidance from a point pi to a point p2 on the road 402 to allow the vehicle 110 to be guided by the computing device 115 until the road lane marker 420 enters the field of view of the sensors 116 before reaching the point p2.

[0038] When road lane marker 420 enters the field of view of sensor 116 as vehicle 110 is directed along steerable path polynomial 418, computing device 115 can determine a new lane marker associated with road lane marker 420 with a high degree of confidence, and thus determine a lane model with a high or moderate degree of confidence, allowing computing device 115 to continue to safely guide vehicle 110 to the predetermined destination without interruption despite passing exit ramp 404 and missing one of the two or more high or moderate degree of confidence lane markers 410, 412 required for a high or moderate degree of confidence lane model for a period of time. Adding extension 422 to steerable path polynomial 418 allows computing device 115 to guide vehicle 110 through missing lane marker 412 without handing over control of vehicle 110 to an occupant or alerting an occupant.

[0039] Extension 422 applied to a steerable path polynomial can be determined by computing device 115 as a period of time or a distance. For example, in traffic scenario 400, vehicle 110 will follow steerable path polynomial 418, traveling a distance di between points po and pi at a speed s in time to. Extension 422 extends steerable path polynomial 418 along extension 422 a distance d2 from point pi to point p2. Vehicle 110 will travel a distance d2 between points pi and p2 in time ti. Extension 422 can be determined by computing device 115 as distance d2 or time ti, and converted from one to the other based on determining the speed of vehicle 110 with inertial navigation data from sensor 116, for example, in either case.

[0040] Figure 5 is a diagram of traffic scenario 500 in which computing device 115 in vehicle 110 on road 502 with exit ramp 504 has determined a lane model including steerable path polynomial 518 based on lane markers 510, 512, which in turn are determined based on road lane markers 506, 508, in which lane marker 510 is associated with the lane model with a high degree of confidence and lane marker 512 (dashed line) is associated with road lane marker 508 with a low degree of confidence, as lane marker 512 is not positioned parallel to and properly spaced from steerable path polynomial 518. In this case, steerable path polynomial 518 has a non-zero radius of curvature r as the predicted lateral acceleration for steerable path polynomial 518 is not zero.

[0041] The steerable path polynomial 518 can include non-zero lateral acceleration. For example, guiding the vehicle 110 along the steerable path polynomial 518 from point po to point p2 can include non-zero lateral acceleration during the time periods from po to pi and from pi to p2, where the steerable path polynomial 518 has a constant right lateral acceleration equal to a, passing through the center of a circle 524 tangent to the steerable path polynomial 518 with a radius r generates a centripetal force equal to the mass of the vehicle 110. The lateral acceleration a and the speed of the vehicle 110 combine with the longitudinal acceleration to determine the position, speed, and direction of the vehicle 110 on the road 502. The lateral acceleration a can have a constraint determined by occupant comfort in addition to the traction constraint that avoids the vehicle 110 from slipping and subsequently losing control of the vehicle 110. High lateral acceleration a can be uncomfortable to the occupants and the computing device 115 can be programmed to avoid high lateral acceleration a unless in an emergency situation, where avoiding a collision can depend on, for example, high lateral or longitudinal acceleration or deceleration.

[0042] The lateral acceleration is based on the radius of curvature r and the speed, both of which can be determined by the computing device 115 using data from the sensors 116, including inertial navigation data. The inertial navigation data can be based on accelerometer data that measures the acceleration of the vehicle 110 in three-dimensional space with high sensitivity and accuracy. Integrating the acceleration with respect to time yields velocity and integrating again with respect to time yields displacement in three-dimensional space. Tracking the acceleration in three-dimensional space allows the computing device 115 to determine the radius of curvature r of the steerable path polynomial 518 associated with the lane model and determine the position of the vehicle 110 relative to the steerable path polynomial 518 and the road 502.

[0043] The computing device 115 can determine a high confidence value for the lane marker 510 based on the machine vision techniques discussed above with respect to Figure 2 The computing device 115 can report the high confidence value. In the example where the computing device 115 determines that the lane marker 510 has a medium or high confidence and determines that the missing lane marker 512, the computing device 115 can add an extension 522 to the steerable path polynomial 518 to allow the computing device 115 to guide the vehicle 110 until the road lane marker 520 is visible to the sensors 116 of the vehicle 110, and the computing device 115 can, for example, determine two high confidence lane markers 510, 512. The computing device 115 can determine that the lane marker 512 is missing based on the machine vision program returning a low confidence value for the lane marker 512 associated with the current lane model as discussed above with respect to Figure 3

[0044] ​The computing device 115 can determine a radius of curvature r of the steerable path polynomial 518 and an extension 522 associated with the lane model based on a speed of the vehicle 110, determine whether a predicted lateral acceleration a will be greater than a predetermined value. If the predicted lateral acceleration a is greater than the predetermined value, the extension 522 will not be applied to the steerable path polynomial 518. The predetermined value can be determined experimentally by testing the accuracy of the vehicle 110 traveling while laterally accelerating. Since a high lateral acceleration a can decrease the confidence that the vehicle 110 will accurately travel to the desired point, a high lateral acceleration can decrease the confidence that applying the extension 522 to the steerable path polynomial will allow the computing device 115 to safely and accurately guide the vehicle 110 to reach, for example, the point p2 on the road 502.

[0045] Applying the extension 522 to the steerable path polynomial 518 can also be based on an orientation of the vehicle 110 with respect to the steerable path polynomial 518. The vehicle 110 position can vary due to normal variations in the road 502 and the vehicle 110 while guiding the vehicle 110 to be precisely positioned on the steerable path polynomial 518 at a given time. Guiding the vehicle 110 along a path defined by the steerable path polynomial 518 can be performed by the computing device 115 using techniques based on closed loop feedback control theory that determine a trajectory of the vehicle 110 (including a position, orientation, speed, lateral acceleration, and longitudinal acceleration of the vehicle 110) and compare the determined trajectory to a trajectory predicted by the steerable path parameters including the steerable path polynomial 518. The trajectory of the vehicle 110 can be determined based on inertial navigation data as discussed above. The trajectory including the position of the vehicle 110 with respect to the steerable path polynomial 518 can determine, among other parameters, a predicted lateral acceleration needed to return the vehicle 110 to the steerable path polynomial 518. If the computing device 115 determines that the lateral acceleration a needed to return the vehicle 110 to a position on the steerable path polynomial 518 is greater than a predetermined value, the computing device 115 can alert an occupant and hand over control to the occupant. The predetermined value can be based on experimental results determining a relationship between the lateral acceleration a and occupant comfort and based on traction limits beyond which the vehicle 110 can skid or lose control.

[0046] Figure 6 is a process 600 for guiding a vehicle based on determining a lane model and a confidence of steerable path parameters for the lane model Figures 1 to 5The process 600 can be implemented by a processor of the computing device 115, obtains input information from the sensors 116, and executes instructions and sends control signals via the controllers 112, 113, 114, for example. The process 600 includes a number of steps that are performed in the order disclosed. The process 600 also includes implementations with fewer steps or can include steps performed in a different order.

[0047] The process 600 begins at step 602, where the computing device 115 in the vehicle 110 determines first lane marks 208 and second lane marks 210 with high confidence and includes the lane marks 208, 210 in a lane model when guiding or assisting in guiding the vehicle 110, the lane model also having steerable path parameters including a steerable path polynomial 214.

[0048] At step 604, the computing device 115 determines that one or more of the lane marks 312 are missing by determining that the confidence associated with the lane model 312 is low. Such a determination can be made by comparing the confidence associated with the lane marks 312 to a predetermined threshold as discussed above with respect to Figure 3 step 604.

[0049] At step 606, the computing device 115 determines the confidence level associated with the remaining first lane marks 208 or second lane marks 210 and proceeds to step 608 if the confidence level is moderate or high. If the confidence level is low, the process 600 proceeds to step 612.

[0050] At step 608, the computing device 115 determines a confidence value based on the steerable path parameters including the steerable path polynomial 418 and inertial navigation data obtained via the sensors 116 determining, for example, the vehicle 110 speed and predicted lateral acceleration. If the steerable path parameter confidence is moderate or high, the extension 422 is added to the steerable path polynomial 418 and the process 600 proceeds to step 610. If the steerable path parameter confidence is low, the process 600 proceeds to step 612.

[0051] At step 610, the computing device 115 guides the vehicle 110 along the steerable path polynomial 418 and the extension 422. After step 610, the process 600 ends.

[0052] At step 612, the computing device 115 alerts an occupant of the vehicle 110 and hands over control of guiding the vehicle 110 to the occupant. After step 612, the process 600 ends.

[0053] Figure 7is a process 700 for determining an extendable portion 422 of a steerable path polynomial 418 based on lane model confidence and steerable path parameter confidence, with respect to Figures 1 to 5 the flowchart described in the background. For example, the process 700 can be implemented by a processor of the computing device 115, obtaining input information from the sensors 116, and executing instructions and sending control signals via the controllers 112, 113, 114. The process 700 includes a number of steps performed in the order disclosed. The process 700 also includes implementations with fewer steps or can include steps performed in a different order.

[0054] The process 700 begins at step 702, where the computing device 115 determines a steerable path parameter including a steerable path polynomial 518.

[0055] At step 704, the computing device 115 determines whether the steerable path parameter is valid and whether the steerable path parameter includes a steerable path polynomial 518 with sufficient longitudinal range. The steerable path parameter is valid if the computing device 115 can determine that the steerable path polynomial 518 directs the motion of the vehicle 110 from the point po to the point pi in a safe and accurate manner. Directing the vehicle 110 from the point po to the point pi in a safe and accurate manner requires that the steerable path polynomial 518 and the vehicle trajectory remain within predetermined limits determined by the passenger comfort and traction limits as discussed above with respect to Figure 5 .

[0056] The steerable path polynomial 518 has sufficient longitudinal range if the distance from the point po to the point pi is greater than a predetermined minimum value. For example, the sufficient longitudinal range can be defined in terms of a reaction time needed for the passenger to control the vehicle 110 after the computing device 115 alerts the passenger that control of the vehicle 110 is needed. The predetermined minimum value can be expressed in distance or time. For example, the longitudinal range can be defined as the time in seconds it would take to travel the distance from the point po to pi at the current vehicle 110 speed. If the longitudinal range is greater than the predetermined minimum value (e.g., three seconds), the longitudinal range is determined to be sufficient and the process 700 proceeds to step 708, otherwise, if the steerable path polynomial 518 is invalid or does not have sufficient longitudinal range, the process 700 proceeds to step 706.

[0057] At step 706, the computing device 115 determines whether the remaining first lane mark 506 or second lane mark 508 has high confidence. If the remaining first lane mark 506 or second lane mark 508 has high confidence, the process 700 proceeds to step 712, otherwise the process 700 proceeds to step 710.

[0058] At step 712, the computing device 115 sets the extension time period to low / medium. After this step, the process 700 ends.

[0059] At step 710, the computing device 115 determines whether the remaining first lane mark 506 or second lane mark 508 has a medium confidence. If the remaining first lane mark 506 or second lane mark 508 has a medium confidence, the process 700 proceeds to step 714, otherwise the process 700 proceeds to step 722.

[0060] At step 714, the computing device sets the extension time period to minimum extension. After this step, the process 700 ends.

[0061] At step 708, the computing device 115 determines whether the remaining first lane mark 506 or second lane mark 508 has a high confidence. If the remaining first lane mark 506 or second lane mark 508 has a high confidence, the process 700 proceeds to step 718, otherwise the process 700 proceeds to step 716.

[0062] At step 716, the computing device 115 determines whether the remaining first lane mark 506 or second lane mark 508 has a medium confidence. If the remaining first lane mark 506 or second lane mark 508 has a medium confidence, the process 700 proceeds to step 720, otherwise the process 700 proceeds to step 722.

[0063] At step 718, the computing device sets the extension time period to maximum extension. After this step, the process 700 ends.

[0064] At step 720, the computing device sets the extension time period to high / medium extension. After this step, the process 700 ends.

[0065] At step 722, the computing device sets the extension time period to no extension. After this step, the process 700 ends.

[0066] Figure 8 is a process 800 for determining whether to add an extension 522 to a steerable path polynomial 418 based on a steerable path parameter confidence and inertial navigation data. The process 800 is described in relation to the flowcharts described in Figures 1 to 5 FIGS. 1-7. For example, the process 800 can be implemented by a processor of the computing device 115, obtain input information from the sensors 116, and execute instructions and send control signals via the controllers 112, 113, 114. The process 800 includes a plurality of steps performed in the order disclosed. The process 800 also includes implementations with fewer steps or can include steps performed in a different order.

[0067] The process 800 begins at step 802, where the computing device 115 determines steerable path parameters, including, for example, the steerable path polynomial 518. The steerable path polynomial 518 can have a non-zero radius of curvature r as discussed above with respect to Figure 5

[0068] At step 804, the computing device determines inertial navigation data from the sensors 116, including the position, orientation, velocity, lateral acceleration, and longitudinal acceleration of the vehicle 110.

[0069] At step 804, the computing device 115 can determine inertial navigation data and thereby determine a trajectory including the position, orientation, velocity, lateral acceleration, and longitudinal acceleration of the vehicle 110 using input from the sensors 116, like the accelerometers discussed above with respect to Figures 3 to 5 The inertial navigation data can allow the computing device 115 to predict the lateral acceleration of the vehicle 110 based on the current trajectory of the vehicle 110 and the expected trajectory based on the steerable path polynomial 518.

[0070] At step 806, the computing device 115 determines whether the lane marker 510 confidence is good. As discussed above with respect to Figures 3 to 5 The computing device 115 can determine the lane marker 510 confidence by examining output from a machine vision program that determines the lane marker 510 based on processing a visual representation of, for example, the lane marker 506, to determine the lane marker 510. The machine vision program can determine a probability that the lane marker 510 correctly represents the visual representation of the lane marker 506 and is associated with a lane model that describes a lane of the roadway on which the vehicle 110 is being guided. In this example, a lane marker 510 confidence equal to medium or high is determined to be good. If the lane marker 510 confidence is determined to not be good, the process 800 branches to step 818. If the lane marker confidence is determined to be good, the process 800 branches to step 808.

[0071] At step 808, the computing device 115 can determine whether the steerable path parameter confidence is good. The steerable path parameter confidence can be determined by determining a probability that the steerable path parameters accurately represent the steerable path polynomial 214, 316, 418, 518 that is currently guiding the vehicle 110 in which it is located. This probability can be determined, for example, by a machine vision technique that determines the steerable path polynomial 214, 316, 418, 518. If the steerable path parameter confidence is medium or low, the process 800 can branch to step 818. If the steerable path parameter confidence is high, the process 800 branches to step 810. ​

[0072] At step 810, the computing device 115 can determine whether the position of the vehicle 110 determined by the inertial navigation data is on or within a predetermined distance (e.g., half the width of the vehicle 110) from the steerable path polynomial 518. If the position of the vehicle 110 is not on or within the predetermined distance from the steerable path polynomial 518, the process 800 branches to step 818. If the computing device 115 determines that the position of the vehicle 110 is on or within the predetermined distance from the steerable path polynomial 518, the process 800 branches to step 814.

[0073] At step 814, the computing device compares the steerable path polynomial 518 to the inertial navigation data to determine whether the lateral acceleration predicted by the steerable path polynomial 518 is close to the value of the lateral acceleration reported by the inertial navigation data. The values should be nearly equal. In examples where the difference between the values is greater than a predetermined threshold, e.g., 20%, the process 800 branches to step 818. In examples where the difference between the values is less than 20%, the process 800 branches to step 812.

[0074] At step 812, the computing device determines that the radius of curvature associated with the steerable path polynomial 518 currently being used to steer the vehicle 110 is within a predetermined limit. The computing device 115 can determine that the vehicle 110 can be safely guided on the steerable path polynomial 518. The predetermined limit can be based on limits of lateral acceleration a generated as a function of radius of curvature r with respect to passenger comfort and traction limits as discussed above with respect to FIG. 5. The lateral acceleration a and thus the radius of curvature r based on the steerable path polynomial 518 can be limited to values that produce the desired safety and accuracy. In examples where the computing device 115 determines that the radius of curvature r is greater than the predetermined limit, the process 800 branches to step 818, otherwise the process 800 branches to step 816. Figure 5

[0075] At step 816, the computing device 115 determines that the steerable path polynomial 518 can have an added extension 522. The process 800 can determine the amount of extension 522 from the minimum extension 522 through the low / medium and high / medium extensions 522 to the maximum extension 522. After this step, the process 800 ends.

[0076] At step 818, the computing device 115 determines that no extension 522 of the steerable path polynomial will be available for the vehicle 110. After this step, the process 800 ends.

[0077] ​Computing devices such as those discussed herein typically each include instructions executable by one or more processing units to cause the device to perform a process or set of processes. Instructions for the processes described above and elsewhere herein can be stored in memory, which can be a single storage device or spread across multiple storage devices. Single storage devices can be memory alone or can be some combination of memory and storage. Single storage devices can be volatile, nonvolatile, or a combination of volatile and non-volatile. In some embodiments, the processing unit(s) can be configured to execute instructions stored in the memory to perform all or a subset of the processes described above and elsewhere herein. For example, the processing unit(s) can be configured to execute instructions stored in the memory to perform the processes described above and elsewhere herein.

[0078] Computer-executable instructions can be compiled or interpreted from computer programs created using a variety of programming languages and / or technologies, including, without limitation, and either alone or in combination, Java TM , C, C++, Visual Basic, Java Script, Perl, HTML, etc. In general, a processor (e.g., a microprocessor) receives instructions, from a memory, a computer-readable medium, etc., and executes these instructions, thereby performing one or more processes, including one or more of the processes described herein. Such instructions and other data can be stored in a file or files on a computer-readable medium. A file can be a collection of data stored on a computer-readable medium. Files usually are located in directories or folders on the computer-readable medium. Files can be of any type, including, but not limited to, audio files, video files, image files, executable files, document files, etc.

[0079] A computer-readable medium includes any medium that participates in providing data (e.g., instructions) that can be read by a computer. Such a medium can take many forms, including but not limited to, non-volatile media, volatile media, etc. Non-volatile media includes, for example, optical or magnetic disks and other persistent memory. Volatile media includes dynamic random access memory (DRAM), which typically constitutes a main memory. Common forms of computer-readable media include, for example, a floppy disk, a flexible disk, hard disk, magnetic tape, any other magnetic medium, a CD-ROM, DVD, any other optical medium, punch cards, paper tape, any other physical medium with patterns of holes, a RAM, a PROM, and EPROM, a FLASH-EPROM, any other memory chip or cartridge, or any other medium from which a computer can read.

[0080] Unless expressly defined otherwise herein, all terms used in the claims are intended to have their ordinary and customary meaning as understood by one of ordinary skill in the art. In particular, use of the singular articles, such as “a,” “the,” “said,” etc., should be understood in the context of the claims to refer to one or more of the indicated elements, unless the claims expressly limit to the contrary.

[0081] The term “exemplary” is used herein in the sense of serving as an example, not an ideal. For instance, a reference to “exemplary widget” should not be understood as a reference to the only widget, but rather to one of several possible widgets.

[0082] The adverb "approximately" modifying a value or result means that the shape, structure, measurement, value, determination, calculation result, etc. can be off by a margin of error due to imperfections in materials, machining, manufacturing, sensor measurements, calculations, processing time, communication time, etc. from the exact described geometry, distance, measurement, value, determination, calculation result, etc.

[0083] In the drawings, like reference numerals refer to like elements throughout. Additionally, some or all of the elements can be modified. With respect to the media, processes, systems, methods, etc. described herein, it should be understood that, although the steps of such processes, etc. have been described as occurring in a certain order, such processes can be practiced with the described steps performed in an order other than the order described herein. It further should be understood that certain steps can be performed simultaneously, can be added, or certain steps can be omitted. In other words, the descriptions of processes herein are provided for the purpose of illustrating certain embodiments, and should in no way be construed so as to limit the claimed application.

[0084] According to the present invention, there is provided a method having: guiding a vehicle based on determining a first lane mark and a second lane mark, wherein a lane mark is a mathematical description of a road lane mark applied to a road to mark a traffic lane; determining a missing first lane mark or a second lane mark; and guiding the vehicle for a determined period of time based on the remaining first lane mark or second lane mark and determining a steerable path polynomial confidence, wherein a steerable path polynomial confidence is a probability that the vehicle will accurately follow a path.

[0085] According to one embodiment, the lane mark is missing due to a road entrance or exit ramp.

[0086] According to one embodiment, determining the first lane mark and the second lane mark includes processing one or more acquired images with a Hough transform to determine a lane mark confidence.

[0087] According to one embodiment, the above invention features further based on the lane mark confidence, determining an orientation of the vehicle relative to a steerable path polynomial, determining a steerable path polynomial curvature, and determining inertial navigation data to determine the steerable path polynomial confidence.

[0088] According to one embodiment, the above invention features further based on determining the orientation of the vehicle relative to the steerable path polynomial, determining inertial navigation data, and determining a steerable path polynomial curvature to determine a steerable path polynomial confidence.

[0089] According to one embodiment, the orientation of the vehicle is determined based on inertial navigation data.

[0090] According to one embodiment, determining the steerable path polynomial curvature is based on the remaining first lane mark or second lane mark, the orientation of the vehicle, and inertial navigation data.

[0091] According to one embodiment, the time period is based on the steerable path polynomial curvature and the lane mark confidence.

[0092] According to one embodiment, the above invention features further include determining the time period based on the steerable path polynomial and a vehicle speed or a predetermined time.

[0093] According to one embodiment, the vehicle speed is based on inertial navigation data.

[0094] According to the present invention, there is provided a system having a processor; and a memory including instructions executed by the processor to: guide a vehicle based on determining first lane marks and second lane marks, wherein a lane mark is a mathematical description of a road lane mark imposed on a road to mark a traffic lane; determine a missing first lane mark or second lane mark; and guide the vehicle a determined time period based on remaining first lane marks or second lane marks and determining a steerable path polynomial confidence, wherein a steerable path polynomial confidence is a probability that the vehicle will accurately follow a path.

[0095] According to one embodiment, the system is programmed to determine that the lane mark is missing due to a road entrance or exit ramp.

[0096] According to one embodiment, the system is programmed to determine the first lane mark and the second lane mark by processing one or more acquired images with a Hough transform to determine a lane mark confidence.

[0097] According to one embodiment, the system is programmed to determine the steerable path polynomial confidence based on the lane mark confidence, determining an orientation of the vehicle relative to a steerable path polynomial, determining a steerable path polynomial curvature, and determining inertial navigation data.

[0098] According to one embodiment, the above invention features further include the system is programmed to determine a steerable path polynomial confidence based on determining the orientation of the vehicle relative to the steerable path polynomial, determining inertial navigation data, and determining a steerable path polynomial curvature.

[0099] According to one embodiment, the system is programmed to determine the orientation of the vehicle based on inertial navigation data.

[0100] According to one embodiment, the system is programmed to determine a steerable path polynomial curvature is based on the remaining first lane marker or second lane marker, the orientation of the vehicle, and inertial navigation data.

[0101] According to one embodiment, the system is programmed to determine the time period is based on the steerable path polynomial curvature and the lane marker confidence.

[0102] According to one embodiment, the system is programmed to determine the time period based on the steerable path polynomial and a vehicle speed or a predetermined time.

[0103] According to one embodiment, the system is programmed to determine the vehicle speed based on inertial navigation data.

Claims

1. A method comprising: Vehicles are guided based on the determination of a first lane marking and a second lane marking, wherein the first lane marking and the second lane marking are mathematical descriptions of road lane markings applied to the road to mark traffic lanes; Identify the missing first or second lane marking; as well as The curvature of a steerable path polynomial guiding a vehicle is determined based on at least one of the first and second lane markings with high or moderate confidence. The vehicle's position is determined based on sensors. An extension is added to the steerable path polynomial to extend the availability of autonomous guidance until the missing first or second lane marking enters the sensor's field of view before reaching the end of the extension.

2. The method of claim 1, wherein the first lane marking or the second lane marking is missing due to a road entrance or exit ramp.

3. The method of claim 1, wherein determining the first lane marking and the second lane marking comprises processing one or more acquired images using a Hough transform to determine lane marking confidence.

4. The method of claim 3, further comprising determining the confidence level of the steerable path polynomial based on the lane marking confidence level, determining the orientation of the vehicle relative to the steerable path polynomial, determining the curvature of the steerable path polynomial, and determining inertial navigation data.

5. The method of claim 4, further comprising determining a steerable path polynomial confidence level based on determining the orientation of the vehicle relative to the steerable path polynomial, determining inertial navigation data, and determining the curvature of the steerable path polynomial.

6. The method of claim 5, wherein the orientation of the vehicle is determined based on inertial navigation data.

7. The method of claim 6, wherein determining the steerable path polynomial curvature is based on the remaining first or second lane markings, the vehicle's orientation, and inertial navigation data.

8. The method of claim 7, wherein the extension applied to the steerable path polynomial is determined as a time period or distance, the time period being based on the curvature of the steerable path polynomial and the lane marking confidence level.

9. The method of claim 8, further comprising determining the time period based on the steerable path polynomial and vehicle speed or a predetermined time.

10. The method of claim 9, wherein the vehicle speed is based on inertial navigation data.

11. The method of claim 10, wherein the time period is based on the lateral acceleration applied to the steerable path polynomial. a limit.

12. The method of claim 3, wherein the lane marking confidence is determined by comparing the Hough transform result with the input video data.

13. The method of claim 12, wherein the lane marking confidence is determined by comparing the position of the lane marking with the position of the steerable path polynomial to determine whether the lane marking is parallel to the steerable path polynomial and at a expected distance from the steerable path polynomial.

14. The method of claim 13, wherein the lane marking confidence is low if the first lane marking or the second lane marking is not parallel to the steerable path polynomial and is not at a expected distance from the steerable path polynomial.

15. A system comprising a computer programmed to perform the method as described in any one of claims 1 to 14.

Images