Lane Control System

Abstract

A lane control system (10) for an automated vehicle includes a camera (14), a laser radar sensor (20), and a controller (24). The camera (14) is used to capture an image (16) of a road (18) on which a host vehicle (12) is traveling. The laser radar sensor (20) is used to detect a discontinuity (22) on the road (18). The controller (24) communicates with the camera (14) and the laser radar sensor (20) and is used to define an area of ​​interest (26) in the image (16), construct a road model (32) of the road (18) based on the area of ​​interest (26), determine that the host vehicle (12) is approaching the discontinuity (22), and adjust the area of ​​interest (26) in the image (16) based on the discontinuity (22).

Description

[0001] This invention application is a divisional application of the invention patent application with application number 201711276245.7, application date December 6, 2017, and name “Visual Sensing Compensation”. Technical Field

[0002] The present invention generally relates to a lane control system for an automated vehicle, and more particularly to a system that can compensate for visual sensing. Background Art

[0003] As is known, lane keeping assist and / or lane centering methods are applied to vehicles traveling on the road. These methods rely on a continuous information feed from a vision system mounted on the vehicle. Data loss from the vision system may cause various lane keeping assist and lane centering methods to perform degraded or fail to perform their intended functions. Summary of the invention

[0004] According to one embodiment, the present invention provides a lane control system for use on an automated vehicle. The lane control system includes a camera, a lidar sensor, and a controller. The camera is used to capture an image of a road on which a host vehicle is traveling. The lidar sensor is used to detect a road discontinuity. The controller communicates with the camera and the lidar sensor and is used to define a region of interest in the image, construct a road model of the road based on the region of interest, determine that the host vehicle is approaching the discontinuity, and adjust the region of interest in the image based on the discontinuity.

[0005] Further characteristics and advantages of the invention will become more apparent on reading the following detailed description of a preferred embodiment, given by way of non-limiting example only and with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0006] The present invention will now be described in detail by way of example with reference to the following drawings, in which:

[0007] Figure 1 is a schematic diagram of a lane control system of an embodiment;

[0008] Figure 2 is a schematic diagram of an embodiment; and

[0009] Figure 3 It is a schematic diagram of an embodiment. DETAILED DESCRIPTION

[0010] The roads that the host vehicle 12 travels on are rarely smooth and often contain irregularities such as potholes, debris overpasses, and discontinuities caused by sloped road conditions that may transiently affect the suspension and / or trajectory of the host vehicle 12. A lane control system 10 is described herein that can anticipate such transient events and maintain control of the host vehicle 12 during transient events that would otherwise interrupt vision system data.

[0011] Figure 1 A non-limiting example of a lane control system 10 suitable for an automated vehicle (such as a host vehicle 12) is shown, hereinafter referred to as system 10. The term "automated vehicle" mentioned herein does not mean that the host vehicle 12 is required to be fully automated or autonomous. It is conceivable that the teachings provided in this case are also applicable to the following situations: the host vehicle 12 relies on pure manual operation by a person, and the automated control only provides lane keeping assist (LKA) or lane centering (LC) to the person, and can operate the brakes of the host vehicle 12 to prevent the host vehicle 12 from entering the driving path of the approaching vehicle.

[0012] The system 10 includes a camera 14 for capturing an image 16 of a road 18 along which the host vehicle 12 is traveling. Cameras 14 suitable for use on the host vehicle 12 are commercially available and are recognized by those skilled in the art, such as the APTINA MT9V023 from Micron Technology, Inc. of Boise, Idaho, USA. The camera 14 can be mounted on the front of the host vehicle 12, or mounted inside the host vehicle 12 in a position that allows the camera 14 to view the area surrounding the host vehicle 12 through the windshield of the host vehicle 12. The camera 14 is preferably a video recorder 14 or a camera 14 that can capture images 16 of the road 18 and its surrounding area at a sufficient frame rate, such as ten frames per second.

[0013] The system 10 also includes a lidar sensor 20 for detecting discontinuities 22 on the road 18. The discontinuities 22 may be any size measurable by the lidar sensor 20 (e.g., from a few millimeters to hundreds of millimeters), and may include, but are not limited to, potholes, road debris, dips, bumps, or road surface transitions. The discontinuities 22 may span the entire width of the road 18 or only a portion of the road 18. Based on the reflectivity of the discontinuities 22 and the unobstructed line of sight between the discontinuities 22 and the lidar sensor 20, the lidar sensor 20 may detect the discontinuities 22 at a range of more than 200 meters (200 m).

[0014] The system 10 also includes a controller 24 that communicates with the camera 14 and the lidar sensor 20. The controller 24 may include a processor (not specifically shown) such as a microprocessor or other control circuits, such as analog and / or digital control circuits including application specific integrated circuits (ASICs) for processing data that are well known to those skilled in the art. The controller 24 may include a memory (not specifically shown) for storing one or more routines, thresholds, and captured data, including non-volatile memory, such as an electrically erasable programmable read-only memory (EEPROM). The one or more routines are used by the processor to perform the following steps: Based on the signals received by the controller 24 from the lidar sensor 20 and the camera 14, determine whether the discontinuity 22 detected on the road 18 will be in the expected path of the host vehicle 12.

[0015] The controller 24 may define a region of interest 26 ( Figure 2 ) to identify features of the road 18, including but not limited to lane markings 28, road edges 30, vanishing points, and other unique features of the road 18 that can be used for LKA and / or LC as can be understood by those skilled in the art. Figure 2 A region of interest 26 is shown within the image 16 captured by the camera 14. The lower edge of the region of interest 26 may be located anywhere in front of the host vehicle 12, preferably 1.5 seconds in front of the host vehicle, regardless of the speed of the host vehicle 12 (not shown).

[0016] The controller 24 is also configured to construct a road model 32 of the road 18 based on the features detected in the region of interest 26, such as Figure 2 As the host vehicle 12 moves along the road 18, the road model 32 may be updated with new information from the camera 14 and the lidar sensor 20. The road model 32 may be updated at a rate equal to the camera 14 frame rate or at a slower rate to satisfy all computational constraints. For example, a Kalman filter may be used to track the edges of the road and is recognized by those skilled in the art. As will be appreciated by those skilled in the art of autonomous control, the controller 24 may also assume some or all of the vehicle control 34( Figure 1 ).

[0017] The controller 24 is also configured to determine that the host vehicle 12 is approaching the discontinuity 22 in anticipation of the above-described transient events. Once the lidar sensor 20 detects the discontinuity 22, the controller 24 can track the discontinuity 22 and determine an arrival time to the discontinuity 22 based on the speed of the host vehicle 12 and the distance to the discontinuity 22 (not specifically shown). The arrival time can be updated at a rate equal to the clock speed (not shown) of the controller 24 or at a slower rate to meet all computational constraints, or can also vary based on the speed of the host vehicle 12. The arrival time can be stored in the memory of the controller 24 and can be associated with the trajectory of the discontinuity 22.

[0018] The controller 24 is also configured to adjust the region of interest 26 in the image 16 in anticipation of the host vehicle 12 approaching the discontinuity 22 and create an adjusted region of interest 36 based on the discontinuity 22 ( Figure 3 The adjusted region of interest 36 may be used to compensate for any data loss and / or sensor noise present in the image 16, such as momentary changes in the position of the road marking 28 in the image 16, when the host vehicle 12 travels over and reacts to the discontinuity 22 resulting from an interruption in the camera 14's field of view. The interruption in the camera 14's field of view may result in non-optimal steering control, where the steering command from the controller 24 fluctuates. Figure 3 The adjusted region of interest 36 is shown in comparison to the transient region of interest 38, which shows the view of the camera 14 at some point during the host vehicle 12's arrival at the discontinuity 22, and the host vehicle 12 reacts to the discontinuity 22 (e.g., a bump, vibration, rotation, etc.) by, for example, turning the camera 14 upward and to the left. Figure 3 In the non-limiting example shown, when the host vehicle 12 reaches the discontinuity 22, the adjusted region of interest 36 is shifted rightward and downward relative to the transient region of interest 38 to align with the previous region of interest 40 to compensate for the reaction that will occur when the host vehicle 12 reaches the discontinuity 22. The previous region of interest 40 is defined as the last region of interest 26 used to update the road model 32 before the host vehicle 12 reaches the discontinuity 22. As the host vehicle 12 traverses the discontinuity 22, the road model may be updated using information from the adjusted region of interest 36, eliminating the interruption that would occur in updating the road model 32 if the transient region of interest 38 were relied upon, thereby allowing the controller 24 to steer the host vehicle 12 as if the discontinuity 22 were not present. Depending on the size of the discontinuity 22, the duration of the transient event may last for several seconds, during which time the road model 32 continues to be updated based on the adjusted region of interest 36 until the transient response of the host vehicle drops below a user-defined threshold and a stable reaction of the host vehicle 12 to the road 18 is reestablished.

[0019] Utilizing the adjusted region of interest 36 to update the road model 32 is beneficial because it requires fewer computational resources than some prior art techniques, such as tracking a reference object in the image 16 and adjusting the region of interest 26 based on the movement of the reference object. Such tracking of the reference object typically requires a significant amount of computational resources because, when the discontinuity 22 is large enough, it causes the controller 24 to search the entire image 16 to relocate the reference object. In contrast, the system 10 predicts how the region of interest 26 will move in future images 16 due to the discontinuity 22, thereby eliminating the need to search the entire image 16 for the reference object, reducing the image processing burden on the controller 24.

[0020] The controller 24 may use a transformation such as an affine transformation and / or a perspective transformation to create the adjusted region of interest 36, wherein the image 16 is rotated to account for the change in angle of the camera 14 field of view and determine the real-world coordinates of the lane marking 28. The controller 24 may determine the type of transformation required based on the discontinuity 22. That is, if the discontinuity 22 in the road 18 is detected to be a sudden vertical drop, the controller 24 may use, for example, an affine transformation. If the discontinuity 22 is detected to be a slope, the controller 24 may use, for example, a perspective transformation.

[0021] The controller 24 may use a dynamic model 42 of the host vehicle 12 ( Figure 1 ) to predict the reaction of the vehicle 12 to the discontinuity 22. The dynamic model 42 estimates the dynamic response of the host vehicle 12 to various inputs, including but not limited to suspension inputs, steering inputs, speed inputs, wheel speed inputs, and cargo load inputs. The dynamic model 42 may also include components that can be understood by those skilled in the art to describe the motion of the host vehicle 12 under various conditions, such as aerodynamics, geometry, mass, motion, tires, and specific components of the road 18.

[0022] Thus, the present invention provides a lane control system 10, a camera 14, a lidar sensor 20, and a controller 24 for the lane control system 10. The lane control system 10 is an improvement over other lane control systems because it can anticipate discontinuities 22 in the road 18 and compensate for erroneous vision system inputs.

[0023] Although the present invention has been described with preferred embodiments, the scope of protection of the present invention is not limited thereto, but is subject to the scope set forth in the appended claims. In addition, the use of the terms first, second, upper, lower, etc. does not represent any order of importance, position, or direction, but only uses the terms first and second to distinguish one element from another. In addition, the use of the terms one, an, etc. does not represent a limitation on quantity, but only indicates that there is at least one of the items.

Claims

1. A lane control system for a host vehicle, the system comprising a controller, the controller being configured to: acquiring an image of a road from a camera of the host vehicle; defining a region of interest in the image; detecting a discontinuity in the road using a lidar sensor of the host vehicle; In response to determining that the host vehicle is approaching the discontinuity, adjusting the region of interest in the image by applying a transformation to the region of interest to compensate for missing data or noise of the image; as well as A road model is constructed based on the adjusted region of interest in the image.

2. The system of claim 1, wherein: The controller is also configured to control the host vehicle based on the road model.

3. The system of claim 1, wherein: The transformation includes at least one of an affine transformation or a perspective transformation.

4. The system of claim 1 or 3, wherein: The controller is configured to apply the transformation by rotating the adjusted region of interest to account for angular changes in the field of view of the camera.

5. The system of claim 1 or 3, wherein: The controller is configured to: based on the discontinuity, selecting a type of transformation to apply to the region of interest in the image to create the adjusted region of interest within the image; as well as The selected transform type is applied to the region of interest.

6. The system of claim 5, wherein: The controller is configured to select an affine transformation in response to determining that the discontinuity comprises a vertical drop in the lane.

7. The system of claim 5, wherein: The controller is configured to select a perspective transform in response to determining that the discontinuity comprises a slope in the lane.

Images