Vehicle system and method for autonomous operation using unclassified hazard detection

Abstract

Systems and methods are provided for detecting and mitigating unclassified hazards of a driver assistance system of a vehicle. The system includes: a data storage device including map data; one or more sensors configured to generate sensor data indicative of one or more sensing objects in the vicinity of the vehicle; and a controller configured, by one or more processors thereof: to receive map data and sensor data; filtering the sensor data to ignore at least one of the sensed objects located outside a boundary area, the boundary area including a portion of an adjacent lane contiguous with a current lane in which the vehicle is operating; determining whether there is an unclassified hazard in an adjacent lane; and in response to determining that there is an unclassified hazard in the adjacent lane, keeping the vehicle operating in the current lane.

Description

Technical Field

[0001] The technical field relates generally to vehicle systems and, more particularly, to autonomous operation of a vehicle configured to detect unclassified hazards to facilitate safe automatic lane changes. Background Art

[0002] An autonomous vehicle is a vehicle that is capable of sensing its environment and navigating with little or no user input. Autonomous vehicles use sensing devices such as radar, lidar, image sensors, etc. to sense their environment. Autonomous vehicle systems also use information from global positioning system (GPS) technology, navigation systems, vehicle-to-vehicle communications, vehicle-to-infrastructure technology, and / or drive-by-wire systems to navigate the vehicle.

[0003] Vehicle automation has been classified into numerical levels ranging from zero to five, with zero corresponding to no automation with full human control and five corresponding to full automation with no human control. Various automatic driver assistance systems (such as cruise control, adaptive cruise control, and parking assist systems) correspond to lower levels of automation, while true "driverless" vehicles correspond to higher levels of automation.

[0004] Due to the huge number of different variables in the real environment, autonomous vehicle control systems may encounter environments or scenarios that may require assistance. For example, traffic, road conditions and other obstacles or scenarios that hinder autonomous operation may be encountered. In lower-level automation systems (e.g., level three or below), such scenarios may require the driver or other vehicle occupants to manually control or operate the vehicle in certain situations, which imposes a burden on the occupants, which is somewhat contrary to the intention of automation. Therefore, it is desirable to provide a vehicle control system and method that can autonomously resolve scenarios and obtain satisfactory solutions for how to autonomously operate the vehicle while reducing the burden on the driver to improve the user experience. In conjunction with the accompanying drawings and the aforementioned technical field and background technology, other desired features and characteristics of the present invention will become apparent from the subsequent detailed description and the appended claims. Summary of the invention

[0005] A method for detecting and mitigating unclassified hazards for a driver assistance system of a vehicle is provided. In one embodiment, the method includes: obtaining map data from a data storage device, the map data including lane geometry information related to a road; obtaining sensor data including one or more sensed objects in the vicinity of the vehicle from one or more sensors of the vehicle; filtering the sensor data using one or more processors to ignore at least one of the one or more sensed objects that is located outside a boundary region, the boundary region including a portion of an adjacent lane that is adjacent to a current lane in which the vehicle is operating, the boundary region having a size determined at least in part by the map data and a current position of the vehicle; determining, using the one or more processors, whether an unclassified hazard exists in the adjacent lane based at least in part on a number of the one or more sensed objects located within the boundary region; and maintaining the vehicle operating in the current lane in response to determining that an unclassified hazard exists in the adjacent lane using the one or more processors.

[0006] In various embodiments, the method includes preventing, with one or more processors, an autonomous lane change by a driver assistance system in response to determining that an unclassified hazard exists in an adjacent lane.

[0007] In various embodiments, the method includes generating, using one or more processors, a normalized boundary region having an area smaller than an area of ​​the boundary region based on normalized errors of the one or more sensors.

[0008] In various embodiments, the method includes, by one or more processors: determining a confidence score based at least in part on a number of one or more sensed objects located within a boundary area; determining a lane score based on the confidence score; and comparing the lane score to a lane score threshold, wherein in response to the lane score being greater than the lane score threshold, maintaining the vehicle operating in the current lane.

[0009] In various embodiments, the method includes determining a confidence score based on at least one of: a detection time of one or more sensing objects, an object-to-object position correlation of one or more sensing objects, a number of one or more sensing objects located within a boundary area, and a pattern map defined by one or more sensing objects.

[0010] In various embodiments, the method includes obtaining visual data including a portion of an adjacent lane from one or more cameras onboard the vehicle, wherein a size of the boundary area is based at least in part on the map data, a current position of the vehicle, and the visual data.

[0011] In various embodiments, the method includes enabling, by one or more processors, an autonomous lane change in response to determining that an unclassified hazard is not present in an adjacent lane.

[0012] In various embodiments, the method includes dynamically adjusting a shape of the boundary region to relate to a geometry of an adjacent lane.

[0013] In various embodiments, the one or more sensors include a radar sensor and the sensor data includes radar point cloud data.

[0014] A system for detecting and mitigating unclassified hazards for a driver assistance system of a vehicle is provided. In one embodiment, the system includes: a data storage device including map data, the map data including information related to a road including lane geometry; one or more sensors of the vehicle configured to generate sensor data indicating one or more sensed objects in the vicinity of the vehicle; and a controller in operable communication with the data storage device and the one or more sensors. The controller is configured to, by one or more processors thereof: receive the map data and the sensor data; filter the sensor data to ignore at least one of the one or more sensed objects that is located outside a boundary region, the boundary region including a portion of an adjacent lane adjacent to a current lane in which the vehicle is operating, the boundary region having a size determined at least in part by the map data and a current position of the vehicle; determine whether an unclassified hazard exists in the adjacent lane based at least in part on the number of the one or more sensed objects located within the boundary region; and maintain the vehicle operating in the current lane in response to determining that an unclassified hazard exists in the adjacent lane.

[0015] In various embodiments, the controller is configured to, by one or more processors, prevent an autonomous lane change by the driver assistance system in response to determining that an unclassified hazard exists in an adjacent lane.

[0016] In various embodiments, the controller is configured to generate, by the one or more processors, a normalized boundary region having an area smaller than an area of ​​the boundary region based on normalized errors of the one or more sensors.

[0017] In various embodiments, the controller is configured to, by one or more processors: determine a confidence score based at least in part on a number of one or more sensed objects located within a boundary area; determine a lane score based on the confidence score; and compare the lane score to a lane score threshold, wherein in response to the lane score being greater than the lane score threshold, maintain the vehicle operating in the current lane.

[0018] In various embodiments, the controller is configured to, by one or more processors, determine a confidence score based on at least one of: a detection time of one or more sensing objects, an object-to-object position correlation of one or more sensing objects, a number of one or more sensing objects located within a boundary area, and a pattern map defined by one or more sensing objects.

[0019] In various embodiments, the controller is configured to, by one or more processors: obtain visual data including a portion of an adjacent lane from one or more cameras onboard the vehicle, wherein a size of the boundary area is based at least in part on map data, a current position of the vehicle, and the visual data.

[0020] In various embodiments, the controller is configured to enable, by one or more processors, an autonomous lane change in response to determining that an unclassified hazard is not present in an adjacent lane.

[0021] In various embodiments, the controller is configured to dynamically adjust, by one or more processors, the shape of the boundary region to correlate with the geometry of adjacent lanes.

[0022] In various embodiments, the one or more sensors include a radar sensor and the sensor data includes radar point cloud data.

[0023] A vehicle is provided, comprising: a data storage device including map data, the map data including information related to a road including lane geometry; one or more radar sensors of the vehicle configured to generate radar sensor data indicating one or more radar objects in the vicinity of the vehicle; and a controller in operable communication with the data storage device and the one or more radar sensors. The controller is configured to, by one or more processors thereof: receive the map data and radar point cloud data; filter the radar point cloud data to ignore at least one of the one or more radar objects that is outside a boundary region, the boundary region including a portion of an adjacent lane adjacent to a current lane in which the vehicle is operating, the boundary region having a size determined at least in part by the map data and a current position of the vehicle; wherein a shape of the boundary region is dynamically adjusted to correlate with the lane geometry, a normalized boundary region including an area less than an area of ​​the boundary region is generated based on a normalized error of the one or more radar sensors, and a confidence score is determined based on at least one of the following: the one or more radar objects The invention relates to a method of controlling a vehicle to detect a lane difference between an adjacent lane and an adjacent lane. The method comprises: determining a lane score based on the confidence score, comparing the lane score to a lane score threshold, determining whether an unclassified hazard exists in an adjacent lane based on the lane score, wherein the unclassified hazard exists in response to the lane score being greater than the lane score threshold; and operating a vehicle based on the determination of whether an unclassified hazard exists in an adjacent lane, including: preventing an autonomous lane change in response to a determination that an unclassified hazard exists in an adjacent lane, and enabling an autonomous lane change in response to a determination that an unclassified hazard does not exist in an adjacent lane.

[0024] In various embodiments, a controller of a vehicle is configured to, by one or more processors, obtain visual data including a portion of an adjacent lane from one or more cameras onboard the vehicle, wherein a size of the boundary area is based at least in part on map data, a current position of the vehicle, and the visual data. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Exemplary embodiments will be described below with reference to the accompanying drawings, wherein like reference numerals refer to like elements, and wherein:

[0026] Figure 1 is a functional block diagram illustrating an autonomous vehicle control system for a vehicle according to various implementations;

[0027] Figure 2 is suitable for various implementations by Figure 1 A block diagram of an automated driving system (ADS) implemented by an autonomous vehicle control system of a vehicle;

[0028] Figure 3 According to one or more aspects described herein Figure 1 The autonomous vehicle control system includes a Figure 2 Data flow diagram of an autonomous vehicle control system for an unclassified hazard detection system used together with an ADS;

[0029] Figure 4 Depicts one or more aspects described herein. Figure 1 The autonomous vehicle control system is suitable for Figure 2 A flowchart of the unclassified hazard detection process implemented by the ADS; and

[0030] Figures 5 to 10 Depicts one or more aspects described herein. Figure 4 An exemplary scenario of an example implementation of an unclassified hazard detection process.

[0031] Fig.11 including an example of a method according to one or more aspects described herein Figure 4 A graph of an exemplary relationship between a confidence level determined by an unclassified hazard detection process and a corresponding lane score;

[0032] Fig.12 and Fig.13 Depicts one or more aspects described herein. Figure 4 An exemplary scenario of an example implementation of an unclassified hazard detection process. DETAILED DESCRIPTION

[0033] The following detailed description is merely exemplary in nature and is not intended to limit application and use. In addition, there is no intention to be bound by any express or implied theory presented in the aforementioned technical field, background technology, invention content or the following detailed description. As used herein, the term module refers to any hardware, software, firmware, electronic control component, processing logic and / or processor device alone or in any combination, including but not limited to: application-specific integrated circuits (ASICs), electronic circuits, processors (shared, dedicated or groups) and memories that execute one or more software or firmware programs, combinational logic circuits and / or other suitable components that provide the described functions.

[0034] Embodiments of the present disclosure may be described herein in terms of functional and / or logical block components and various processing steps. It should be understood that such block components may be implemented by any number of hardware, software, and / or firmware components configured to perform specified functions. For example, embodiments of the present disclosure may employ various integrated circuit components, such as memory elements, digital signal processing elements, logic elements, lookup tables, etc., that may implement various functions under the control of one or more microprocessors or other control devices. In addition, those skilled in the art will appreciate that embodiments of the present disclosure may be practiced in conjunction with any number of systems, and the systems described herein are merely exemplary embodiments of the present disclosure.

[0035] For the sake of brevity, conventional techniques related to signal processing, data transmission, signaling, control, and other functional aspects of the system (and the various operating components of the system) may not be described in detail herein. In addition, the connecting lines shown in the various figures included herein are intended to represent example functional relationships and / or physical connections between the various elements. It should be noted that many alternative or additional functional relationships or physical connections may exist in the embodiments of the present disclosure.

[0036] Now refer to Figure 1 According to one or more implementations, as described in more detail below, the autonomous vehicle control system 100 determines a plan for autonomously operating the vehicle 10 along a route in a manner that accounts for unclassified objects or obstacles detected by the onboard sensors 28, 40. In this regard, a control module onboard the vehicle 10 calibrates different types of onboard sensors 28, 40 relative to each other and / or the vehicle 10, thereby allowing data from those different types of onboard sensors 28, 40 to be spatially or otherwise associated with each other based on the calibration for purposes of object detection, object classification, and resulting autonomous operation of the vehicle 10. In various implementations, the vehicle control system 100 includes an unclassified hazard detection system that is configured to detect unclassified hazards by at least partially correlating map data with radar point cloud data to facilitate a safe autonomous lane change maneuver.

[0037] In various embodiments, the vehicle 10 can be any of many different types of automobiles, such as, for example, a sedan, van, truck, or sport utility vehicle (SUV), and in some embodiments, can be two-wheel drive (2WD) (i.e., rear-wheel drive or front-wheel drive), four-wheel drive (4WD) or all-wheel drive (AWD), and / or various other types of vehicles. In certain embodiments, the vehicle 10 can also include an electric bicycle and / or one or more other types of vehicles. Additionally, in various embodiments, it should also be understood that the vehicle 10 can include any number of other types of mobile platforms.

[0038] like Figure 1 As depicted, the exemplary vehicle 10 generally includes a chassis 12, a body 14, front wheels 16, and rear wheels 18. The body 14 is disposed on the chassis 12 and substantially surrounds the components of the vehicle 10. The body 14 and chassis 12 may together form a frame and substantially surround the components of the vehicle 10. The wheels 16-18 are each rotatably coupled to the chassis 12 near a respective corner of the body 14.

[0039] In an exemplary implementation, the vehicle 10 is an autonomous vehicle or is otherwise configured to support one or more autonomous operating modes, and the control system 100 is incorporated into the vehicle 10. In an exemplary implementation, the vehicle 10 is a so-called Level 2 automation system. Level 2 stands for "partial driving automation" and refers to a specific driving mode performance of an autonomous driving system that controls steering, acceleration, and braking in specific scenarios while the driver remains alert and actively supervises the autonomous driving system at all times, and can provide support to the driver to control the main driving task.

[0040] As shown, the vehicle 10 generally includes a propulsion system 20, a transmission system 22, a steering system 24, a braking system 26, a sensor system 28, an actuator system 30, at least one data storage device 32, at least one controller 34, and a communication system 36. In various implementations, the propulsion system 20 may include an internal combustion engine, an electric motor such as a traction motor, and / or a fuel cell propulsion system. The transmission system 22 is configured to transfer power from the propulsion system 20 to the wheels 16, 18 according to a selectable speed ratio. According to various embodiments, the transmission system 22 may include a step-ratio automatic transmission, a continuously variable transmission, or other suitable transmission. The braking system 26 is configured to provide braking torque to the wheels 16, 18. In various implementations, the braking system 26 may include friction brakes, wire control brakes, regenerative braking systems (such as motors), and / or other suitable braking systems. The steering system 24 affects the position of the wheels 16, 18. Although drawn as including a steering wheel for exemplary purposes, in some embodiments contemplated within the scope of the present disclosure, the steering system 24 may not include a steering wheel 24a.

[0041] The sensor system 28 includes one or more sensing devices 40a-40n that sense observable conditions of the external environment and / or the internal environment of the vehicle 10. The sensing devices 40a-40n may include, but are not limited to, radar, lidar, global positioning system, optical camera, thermal camera, ultrasonic sensor, and / or other sensors. The actuator system 30 includes one or more actuator devices 42a-42n that control one or more vehicle features, such as, but not limited to, the propulsion system 20, the transmission system 22, the steering system 24, and the braking system 26. In various implementations, the vehicle features may also include interior and / or exterior vehicle features, such as, but not limited to, doors, trunks, and cabin features, such as air, music, lighting, etc. (not numbered).

[0042] The data storage device 32 stores data for automatically controlling the vehicle 10. In various implementations, the data storage device 32 stores a defined map of the navigable environment. In various implementations, the defined map may be predefined by a remote system and obtained from the remote system. For example, the defined map may be assembled by the remote system and communicated to the vehicle 10 (wirelessly and / or in a wired manner) and stored in the data storage device 32. As can be appreciated, the data storage device 32 may be part of the controller 34, separate from the controller 34, or part of the controller 34 and part of a separate system. The storage device 32 may be any suitable type of storage device, including various different types of direct access storage and / or other memory devices. In an exemplary embodiment, the storage device 32 includes a program product from which a computer-readable memory device may receive instructions for executing one or more embodiments of one or more processes of the present disclosure (such as, for example, described below in conjunction with the present disclosure). Figure 4 In another exemplary embodiment, the program product may be directly stored in and / or otherwise accessed by the memory device and / or one or more other disks and / or other memory devices.

[0043] The controller 34 includes at least one processor 44, a communication bus 45, and a computer-readable storage device or medium 46. The processor 44 performs the computational and control functions of the controller 34. The processor 44 can be any customized or commercially available processor, a central processing unit (CPU), a graphics processing unit (GPU), an auxiliary processor among several processors associated with the controller 34, a semiconductor-based microprocessor (in the form of a microchip or chipset), a macroprocessor, any combination thereof, or any device generally used to execute instructions. For example, the computer-readable storage device or medium 46 can include volatile and non-volatile storage in a read-only memory (ROM), a random access memory (RAM), and a keyless memory (KAM). KAM is a permanent or non-volatile memory that can be used to store various operating variables when the processor 44 is powered off. The computer readable storage device or medium 46 may be implemented using any of a number of known memory devices such as PROM (Programmable Read Only Memory), EPROM (Electrical PROM), EEPROM (Electrically Erasable PROM), flash memory, or any other electrical, magnetic, optical, or combination memory device capable of storing data, some of which represents executable instructions used by the controller 34 in controlling the vehicle 10. The bus 45 is used to transfer programs, data, status, and other information or signals between the various components of the vehicle 10. The bus 45 may be any suitable physical or logical means of connecting computer systems and components. This includes, but is not limited to, direct hardwired connections, fiber optics, infrared, and wireless bus technologies.

[0044] The instructions may include one or more separate programs, each of which includes an ordered list of executable instructions for implementing logical functions. When executed by the processor 44, the instructions receive and process signals from the sensor system 28, execute logic, calculations, methods and / or algorithms for automatically controlling components of the vehicle 10, and generate control signals to the actuator system 30 based on the logic, calculations, methods and / or algorithms to automatically control components of the vehicle 10. Although Figure 1 Only one controller 34 is shown, but embodiments of the vehicle 10 may include any number of controllers 34 that communicate via any suitable communication medium or combination of communication media and collaborate to process sensor signals, execute logic, calculations, methods and / or algorithms, and generate control signals that automatically control features of the vehicle 10.

[0045] As can be appreciated, the controller 34 may be different in other ways. Figure 1. For example, the controller 34 may be coupled to or may otherwise utilize one or more remote computer systems and / or other control systems, such as as part of one or more of the vehicle devices and systems described above. It should be understood that while the exemplary embodiment is described in the context of a fully functional computer system, those skilled in the art will recognize that the mechanisms of the present disclosure are capable of being distributed as a program product, and that one or more types of non-transitory computer-readable signal-bearing media are used to store the program and its instructions and implement its distribution, such as a non-transitory computer-readable medium that carries the program and contains computer instructions stored therein for causing a computer processor (such as, processor 44) to execute and implement the program. Such a program product may take a variety of forms, and the present disclosure applies equally regardless of the specific type of computer-readable signal-bearing medium used to implement the distribution. Examples of signal-bearing media include: recordable media, such as floppy disks, hard drives, memory cards, and optical disks; and transmission media, such as digital and analog communication links. It should be understood that cloud-based storage and / or other technologies may also be utilized in certain embodiments. Similarly, it should be understood that the computer system of the controller 34 may also be different from the computer system of the controller 34 in other ways. Figure 1 In the embodiments depicted in FIG. 1 , a computer system such as controller 34 may be coupled to or may otherwise utilize one or more remote computer systems and / or other control systems.

[0046] In various implementations, one or more instructions of the controller 34 are embodied in the control system 100 (e.g., in the data storage element 46), and when executed by the processor 44, the instructions cause the processor 44 to obtain data captured or generated from the imaging and ranging devices (e.g., of the sensors 28, 40) and utilize the captured environmental data to determine commands for autonomously operating the vehicle 10, as described in more detail below.

[0047] Still refer to Figure 1 In an exemplary implementation, the communication system 36 is configured to wirelessly communicate information to and from other entities 48 via a communication network, such as, but not limited to, other vehicles ("V2V" communication), infrastructure ("V2I" communication), remote systems, and / or personal devices. In an exemplary implementation, the communication system 36 is a wireless communication system configured to communicate via a wireless local area network (WLAN) using the IEEE 802.11 standard or by using cellular data communications. However, additional or alternative communication methods such as dedicated short-range communication (DSRC) channels are also considered to be within the scope of the present disclosure. A DSRC channel refers to a one-way or two-way short-range to medium-range wireless communication channel designed specifically for automotive use, and a corresponding set of protocols and standards.

[0048] The communication network utilized by the communication system 36 may include a wireless carrier system such as a cellular telephone system including a plurality of cell towers (not shown), one or more mobile switching centers (MSCs) (not shown), and any other networking components required to connect the wireless carrier system to the land communication system, and the wireless carrier system may implement any suitable communication technology, including, for example: digital technology, such as CDMA (e.g., CDMA2000), LTE (e.g., 4G LTE or 5G LTE), GSM / GPRS or other current or emerging wireless technologies. Additionally or alternatively, a second wireless carrier system in the form of a satellite communication system may be utilized to provide one-way or two-way communications using one or more communication satellites (not shown) and uplink transmission stations (not shown), including but not limited to satellite radio services, and / or satellite telephone services, etc. Some implementations may utilize land communication systems, such as traditional land-based telecommunications networks, including the public switched telephone network (PSTN) for providing hard-wired telephones, packet-switched data communications, and Internet infrastructure. One or more portions of the terrestrial communication system may be implemented using standard wired networks, fiber optic or other optical networks, cable networks, power lines, other wireless networks such as wireless local area networks (WLANs), or networks providing broadband wireless access (BWA), or any combination thereof.

[0049] Now refer to Figure 2 According to various implementations, the controller 34 implements an autonomous driving system (ADS) 70. That is, appropriate software and / or hardware components of the controller 34 (e.g., the processor 44 and the computer-readable storage device 46) are utilized to provide an autonomous driving system 70 for use in conjunction with the vehicle 10, for example, to automatically control the various actuators 30 to control vehicle acceleration, steering, and braking, respectively, without human intervention.

[0050] In various implementations, the instructions for the autonomous driving system 70 may be organized by function or system. Figure 2 As shown, the autonomous driving system 70 may include a sensor fusion system 74, a positioning system 76, a guidance system 78, and a vehicle control system 80. As may be appreciated, in various implementations, instructions may be organized into any number of systems (e.g., combined, further divided, etc.), as the present disclosure is not limited to this example.

[0051] In various implementations, the sensor fusion system 74 integrates and processes the sensor data and predicts the presence, location, classification, and / or path of objects and environmental characteristics of the vehicle 10. In various implementations, the sensor fusion system 74 may combine information from multiple sensors including, but not limited to, cameras, lidar, radar, and / or any number of other types of sensors.

[0052] The positioning system 76 processes the sensor data and other data to determine the location of the vehicle 10 relative to the environment (e.g., local location relative to a map, precise location relative to a lane of a road, vehicle heading, speed, etc.). The guidance system 78 processes the sensor data and other data to determine a path for the vehicle 10 to follow given the current sensor data and vehicle attitude. The vehicle control system 80 then generates control signals for controlling the vehicle 10 based on the determined path. In various implementations, the controller 34 implements machine learning techniques to assist in the functions of the controller 34, such as feature detection / classification, obstacle mitigation, route traversal, mapping, sensor integration, ground truth determination, etc.

[0053] In one or more implementations, the guidance system 78 includes a motion planning module that generates a motion plan for controlling the vehicle 10 as the vehicle traverses along the route. The motion planning module includes a longitudinal solver module that generates a longitudinal motion plan output, which is used to control the vehicle 10 along the route in a general direction of travel, for example, by accelerating or decelerating the vehicle 10 at one or more locations along the route in the future to maintain a desired speed or rate. The motion planning module also includes a lateral solver module that generates a lateral motion plan output, which is used to control the lateral movement of the vehicle 10 along the route to change the general direction of travel, for example, by steering the vehicle 10 at one or more locations along the route in the future (e.g., keeping the vehicle 10 in the center of the lane, changing lanes, etc.). The longitudinal plan output and the lateral plan output correspond to the command (or planned) path output provided to the vehicle control system 80 for controlling the vehicle actuator 30 to achieve the movement of the vehicle 10 along the route corresponding to the longitudinal plan and the lateral plan.

[0054] During normal operation, the longitudinal solver module attempts to optimize the vehicle speed (or rate) in the direction of travel, the vehicle acceleration in the direction of travel, and the derivative of the vehicle acceleration in the direction of travel (alternatively referred to herein as the longitudinal jerk of the vehicle), while the lateral solver module attempts to optimize one or more of the steering angle, the rate of change of the steering angle, and the acceleration or second derivative of the steering angle (alternatively referred to herein as the lateral jerk of the vehicle). In this regard, the steering angle can be related to the curvature of the path or route, and any of the steering angle, the rate of change of the steering angle, and the acceleration or second derivative of the steering angle, either alone or in combination, can be optimized by the lateral solver module.

[0055] In an exemplary implementation, the guidance system 78 supports a hands-free autonomous mode of operation that, while enabled and operating, controls steering, acceleration, and braking to provide lane centering while attempting to maintain a driver-selected speed and / or following distance (or gap time) relative to other vehicles using current sensor data (or obstacle data) provided by the sensor fusion system 74 and current vehicle pose provided by the positioning system 76. In the autonomous mode of operation, the guidance system 78 includes otherwise implementing a lane change coordinator that analyzes route information (if available) in addition to data or other information from the sensor fusion system 74, the positioning system 76, and potentially other modules or systems to determine whether to initiate and execute a lane change from a current lane of travel to an adjacent lane of travel, for example, based on the presence or absence of slower moving traffic in the current lane of travel ahead of the vehicle 10 (e.g., to pass or overtake another vehicle), whether the current lane is ending or merging into an adjacent lane, and / or whether a lane change is required to maintain travel along a desired route, etc. In this regard, the lane change coordinator can automatically determine when to initiate a lane change and automatically configure the lateral solver module and / or the motion planning module to generate a corresponding lateral plan to change lanes in a desired manner and provide the lateral plan to the vehicle control system 80, which automatically generates corresponding control signals for autonomously controlling the vehicle actuator 30 to manipulate the vehicle 10 and perform the lane change.

[0056] Now refer to Figure 3 , and continue to refer to Figure 1 to Figure 2 , a data flow diagram illustrating how Figure 1 The controller 34 realizes the Figure 2 Elements of the unclassified hazard detection system 200 of the ADS 70. As can be appreciated, various embodiments of the unclassified hazard detection system 200 according to the present disclosure may include any number of modules embedded within the controller 34, which modules may be combined together and / or further divided to similarly implement the systems and methods described herein. In addition, the inputs to the unclassified hazard detection system 200 may be received from other control modules (not shown) associated with the vehicle 10, and / or determined / modeled by other sub-modules (not shown) within the controller 34. In addition, the inputs may also be pre-processed, such as sub-sampling, noise reduction, normalization, feature extraction, missing data reduction, etc. In various embodiments, the unclassified hazard detection system 200 includes a positioning module 210, a boundary area module 212, a hazard detection module 214, and a control module 216.

[0057] In various embodiments, the positioning module 210 receives as input map data 220 retrieved from the data storage device 32 or the computer-readable storage device or medium 46. The map data 220 includes various data indicating various characteristics of the road, including road geometry and road curvature. In various embodiments, the positioning module 210 receives as input sensor data 222 generated by the sensor system 28. The sensor data 222 includes various data indicating camera objects, including lane markings and radar objects.

[0058] The positioning module 210 performs positioning on known road geometry. In various embodiments, positioning may include determining the location of the vehicle 10 based on GPS signals. In various embodiments, positioning may include determining the lane of the road in which the vehicle 10 is located based on lane markings. In various embodiments, positioning may include determining the approximate width of the lane of the road based on the lane markings. The positioning module 210 generates positioning data 224, which includes various data indicating, for example, the location of the vehicle 10, the lane in which the vehicle 10 is located, the approximate width of the lane of the road, and / or the location of radar objects on the road (i.e., any radar objects located outside the road can be eliminated or ignored).

[0059] In various embodiments, the boundary region module 212 receives as input the positioning data 224 generated by the positioning module 210. The boundary region module 212 determines a region of interest in a lane of a road, such as a lane that is laterally adjacent to the lane in which the vehicle 10 is located, and establishes a bounding box or region having boundaries associated with the lateral extent of the region of interest. In various embodiments, the region of interest corresponds to a portion of the road that represents a portion of the road that will be physically entered during an autonomous lane change by the vehicle 10, wherein the presence of a permanent object (e.g., a concrete barrier, a crack in the road, etc.) within the portion of the road may indicate that the vehicle 10 is at risk if the autonomous lane change is performed. In various embodiments, the size or lateral dimension (e.g., width) of the boundary region may be determined at least in part by a predetermined standard deviation of noise attributable to a radar system of the vehicle 10 that generates the sensor data 222. In various embodiments, the boundary region may have a dynamic shape that continuously adjusts based at least in part on the curvature of the road as determined based on the map data 220. The boundary region module 212 eliminates or ignores at least one, some, or all of the radar objects that are located outside the boundaries of the boundary region. In various embodiments, the boundary region may be normalized one or more times to provide a narrower boundary region to create an adjusted distribution of radar objects. In various embodiments, the boundary region may be normalized in response to the lateral position of the image along the image representing the inferred hazard (e.g., Fig. 9). The boundary region module 212 generates boundary region data 226, which includes various data indicating the established boundary region and the number and positions of radar objects within the boundary region.

[0060] In various embodiments, the hazard detection module 214 receives as input the boundary area data 226 generated by the boundary area module 212. The hazard detection module 214 dynamically determines a confidence level associated with the possibility of a hazard existing in the boundary area based at least in part on the number of radar objects located within the boundary area at a given time. In various embodiments, the confidence level can be expressed as a numerical value or a percentage (e.g., 0% to 100% confidence that a hazard exists). The hazard detection module 214 then generates and dynamically updates the lane score of the desired lane based at least in part on the confidence level. In various embodiments, the lane score can be expressed as a numerical value (e.g., between 100 and -100). In various embodiments, the lane score can be expressed as a word, phrase, and / or category (e.g., between "highly expected" and "highly undesirable"). In various embodiments, the lane score can be determined as a numerical value and converted to a word, phrase, and / or category. For example, if the confidence level is 100% (i.e., very high or substantially certain), the lane score can be -100 (very low), and -100 can be converted to "highly undesirable". In various embodiments, the relationship between the confidence level and the lane score may be weighted and / or exponential.Hazard detection module 214 generates hazard detection data 228 that includes various data indicating lane scores for one or more lanes of a road, such as a desired lane.

[0061] In various embodiments, the control module 216 receives as input the hazard detection data 228 generated by the hazard detection module 214. The control module 216 generates instructions configured to allow and / or restrict lane change maneuvers based on the lane score. In various embodiments, the instructions are configured to enable and / or disable autonomous lane change. The control module 216 generates control data 230 including various data indicating the instructions. The control module 216 can transmit the control data 230 to the vehicle control system 80. In some embodiments, the control data 230 can cause the vehicle control system 80 and its lane change system to operate normally in response to a sufficiently high lane score (i.e., low confidence that there is a hazard), and can suspend, disable or prevent the automatic lane change of the lane change system in response to a sufficiently low lane score (i.e., high confidence that there is a hazard) at least in the direction of the inferred hazard. For example, in response to a sufficiently low lane score, the control data 230 can cause the operation of the vehicle 10 to remain in the current lane in which the vehicle 10 is traveling until the lane score rises to such an extent that the autonomous lane change can be performed safely. Notably, in some embodiments, preventing a lane change to an adjacent lane with a low lane score may not prevent the vehicle 10 from autonomously changing lanes to another lane with a sufficiently high lane score.

[0062] In various embodiments, the unclassified hazard detection system 200 may operate continuously during operation of the vehicle 10, continuously during operation of a lane change system, or in response to a request for an autonomous lane change by a lane change system.

[0063] Figure 4 An exemplary implementation of an unclassified hazard detection method 300 is depicted, which is suitable for being implemented by one or more control modules onboard a vehicle (e.g., by the unclassified hazard detection system 200 in conjunction with the ADS 70 supported by the controller 34 in the vehicle 10) to autonomously operate one or more actuators onboard the vehicle to change lanes in a manner affected by the detection of an unclassified hazard in a lateral lane. For illustrative purposes, the following description may refer to the above combined Figures 1 to 3 Although portions of the unclassified hazard detection method 300 may be performed by different elements of a vehicle system, for purposes of illustration, the present subject matter is primarily described in the context of the unclassified hazard detection method 300 being primarily performed by the ADS 70 and the unclassified hazard detection system 200 implemented by the controller 34 associated with the vehicle 10.

[0064] Will refer to Figures 5 to 13 The non-limiting examples presented in are used to illustrate various aspects of the method 300. In these examples, a vehicle 510 is traveling on a road 500 that includes five lanes, including a first lane 501, a second lane 502, a third lane 503, a fourth lane 504, and a fifth lane 505 from right to left. Figures 5 to 7 and Fig.12 , vehicle 510 is located in the third lane 503, and Figures 8 to 10 and Fig.13 , vehicle 510 is located in the first lane 501 .

[0065] In an exemplary implementation, the unclassified hazard detection method 300 may begin at 310. At 312, the method 300 may include receiving external object related sensor data from one or more sensing devices 28, 40 of the vehicle 10, such as radar objects (i.e., targets / returns) from a radar system. In various embodiments, the radar objects may be stored as radar point clouds.

[0066] At 314, the method 300 may include receiving map data including road characteristics such as road geometry and road curvature from a data storage device (e.g., the data storage device 32 or the computer-readable storage device or medium 46). Optionally, at 314, the method 300 may include receiving sensor data related to road characteristics from one or more sensing devices 28, 40 of the vehicle 10, such as video from a camera system. In various embodiments, the sensor data related to road characteristics may be analyzed to determine a current lane in the road in which the vehicle 10 is located based on lane markings. In various embodiments, the sensor data related to road characteristics may be analyzed to determine an approximate width of a lane of the road based on lane markings. In various embodiments, the method 300 may include receiving lane markings as determined by another system of the vehicle 10, an identification of a current lane in which the vehicle 10 is traveling, and / or a lane width at 314. In various embodiments, the method 300 may include establishing fused lane boundaries based at least in part on the GPS location of the vehicle 10, map-based road characteristics and curvature, and camera-based perceived road boundaries.

[0067] At 316, method 300 may include establishing a boundary region having a boundary associated with a lateral extent of the region of interest in a road lane, such as a lane laterally adjacent to a current lane in which vehicle 10 is located. In various embodiments, the size of the boundary region may be determined at least in part by a predetermined standard deviation of noise attributable to a radar system of vehicle 10 that generated the radar object. In various embodiments, the boundary region may have a dynamic shape that continuously adjusts based at least in part on road curvature.

[0068] For example, in Figure 5 In the embodiment, a first boundary area defined by boundary 512a is established on the lateral left side of vehicle 510, and Figure 85 , a second boundary region defined by boundary 512 b is established on the lateral left side of vehicle 510. It is noteworthy that the shapes of the first and second boundary regions have been adjusted based on the curvature of road 500. For example, the second boundary region is curved to the right due to the curvature of first lane 501 and second lane 502 of road 500. Radar object 514 is shown in the area adjacent to vehicle 510 on road 500.

[0069] At 318, method 300 may include filtering the radar objects to eliminate or ignore any radar objects located outside the boundary region. At 320, method 300 may include normalizing the boundary region one or more times to provide a narrower boundary region and thereby create an adjusted distribution of radar objects. In various embodiments, one or more radar objects located outside the normalized boundary region may be eliminated or ignored.

[0070] For example, in Figure 6 and Fig. 9 middle, Figure 5 and Figure 8 Radar objects 514 have been filtered out so that only certain radar objects 516 located within boundary region 512 remain. In these examples, there are four remaining radar objects 516a within the first boundary region and four remaining radar objects 516b within the second boundary region. Figure 7 and Fig.10 , the boundary region 512 is normalized based on the position of the remaining radar targets 516 to provide a normalized boundary region 520 .

[0071] At 322, method 300 may include determining a confidence level associated with the possibility of a hazard existing in the boundary region based at least in part on the number of radar objects located within the boundary region at a given time. The confidence level may be dynamically increased or decreased depending on the number of radar objects within the boundary region and / or the grouping of radar objects within the boundary region. For example, at 324, an increase in the number of radar objects within the normalized boundary region, a pattern diagram of radar objects within the normalized boundary region, and / or a retention (i.e., recurrence) of radar objects within the normalized boundary region over time may result in an increase in the confidence level. Conversely, at 326, a decrease in the number of radar objects within the normalized boundary region and / or a lack of retention (i.e., recurrence) of radar objects within the normalized boundary region over time may result in a decrease in the confidence level. In various embodiments, the confidence level may increase exponentially. For example, the confidence level may increase at an increasing rate in response to, for example, continued retention of radar objects within the normalized boundary region. In various embodiments, the confidence level may decrease or decay relatively slowly over time in response to, for example, a reduced presence of a radar object within a normalized boundary region, rather than decreasing in a sharp drop or abrupt manner.

[0072] At 328, method 300 may include determining or establishing a lane score based at least in part on the confidence level. In various embodiments, the lane score may be dynamically updated over time based on the confidence level. The lane score may have various relationships with the confidence level, such as, but not limited to, an exponential relationship. In various embodiments, the lane score may be a composite score based on the confidence level and one or more additional factors. In various embodiments, the lane score may increase exponentially. For example, the lane score may increase at an increasing rate in response to, for example, a retention of the confidence level. In various embodiments, the lane score may decrease or decay over time at a predetermined rate in response to, for example, a retention decrease in the confidence level, rather than decreasing in a sharp drop or jump manner.

[0073] For example, Fig.11 A first (upper) graph illustrating increasing confidence levels over time and a second (lower) graph illustrating lane scores as confidence levels based on the first graph decrease over time are presented.

[0074] Fig.11 The relationship between the confidence level of the first graph and the lane score of the second graph presented is non-limiting. In the first graph, the y-axis is the confidence level percentage (%) and the x-axis is time (t) (in seconds). In the second graph, the y-axis is the lane score without unit and the x-axis is time (t) (in seconds).

[0075] At 330, the lane score may be compared to a preprogrammed lane score threshold. At 332, method 300 may include pausing or preventing a lane change maneuver in response to the lane score exceeding the lane score threshold. At 334, method 300 may include enabling or allowing a lane change maneuver in response to the lane score being less than the lane score threshold. Method 300 may end at 336.

[0076] In various embodiments, the boundary region may be normalized to provide a normalized line in response to a repeating pattern of radar objects in laterally similar locations along an extended longitudinal region that may represent the lateral location of an inferred hazard. Fig.12 A first normalized line 522a adjacent to the vehicle 510 is presented, and Fig.13 A second normalized line 522b adjacent to the vehicle 510 is presented.

[0077] The systems and methods disclosed herein provide various benefits over certain existing systems and methods. For example, various autonomous driving systems perform object and / or hazard detection by consuming sensor input to classify objects. Although useful in many cases, classifying objects may require specific sensors and / or may require a large amount of computing resources. In addition, problems may arise when there are objects that are not within one of the preprogrammed classification categories. The systems and methods disclosed herein solve these problems by providing detection and mitigation of unclassified hazards by correlating map data and sensor data (e.g., radar objects) to detect, determine and / or mitigate unclassified hazards present on the road and suspending and / or enabling one or more autonomous functions (e.g., automatic lane change maneuvers) to avoid such hazards. By combining the implicit knowledge of the road contained in the map data with the sensor data from the vehicle, the systems and methods disclosed herein are able to provide multi-domain, multi-sensor fusion interpretations based on, for example, a single sensor data (e.g., radar). In various implementations, the systems and methods do not need to classify the hazards. For example, if the systems and methods are operating to, for example, enable or suspend an automated lane change maneuver, knowledge of the likelihood of a potential unclassified hazard in a lateral lane is sufficient to determine whether the vehicle needs to change from a safe position (i.e., current lane) to a potentially unsafe position (e.g., desired lane). By detecting potential hazards without classifying them, the systems and methods disclosed herein can facilitate safe operation of the vehicle while reducing the type or number of sensors and / or computational requirements required for operation of the ADS 70.

[0078] Although at least one exemplary embodiment has been given in the foregoing detailed description, it should be understood that there are a large number of variations. It should also be understood that the exemplary embodiment or multiple exemplary embodiments are only examples and are not intended to limit the scope, applicability or configuration of the present disclosure in any way. On the contrary, the foregoing detailed description will provide a convenient roadmap for implementing the exemplary embodiment or multiple exemplary embodiments for those skilled in the art. It should be understood that various changes can be made to the functions and arrangements of the elements without departing from the scope of the present disclosure as set forth in the attached claims and their legal equivalents.

Claims

1. A method for detecting and mitigating unclassified hazards of a driver assistance system of a vehicle, the method comprising: Obtaining map data from a data storage device, the map data comprising information associated with a road including lane geometry; obtaining sensor data including one or more sensed objects in a vicinity of the vehicle from one or more sensors of the vehicle; filtering, with one or more processors, the sensor data to ignore at least one of the one or more sensed objects that is outside a boundary region, the boundary region comprising a portion of an adjacent lane adjoining a current lane in which the vehicle is operating, the boundary region having a size determined at least in part by the map data and a current position of the vehicle; determining, with the one or more processors, whether an unclassified hazard is present in the adjacent lane based at least in part on a number of the one or more sensed objects located within the boundary region; as well as In response to determining that the unclassified hazard is present in the adjacent lane, utilizing the one or more processors, maintaining the vehicle operating in the current lane.

2. The method according to claim 1, further comprising: In response to determining that the unclassified hazard is present in the adjacent lane, an autonomous lane change by the driver assistance system is prevented, utilizing the one or more processors.

3. The method according to claim 1, further comprising: By the one or more processors: determining a confidence score based at least in part on a number of the one or more sensed objects located within the boundary region; determining a lane score based on the confidence score; as well as The lane score is compared to a lane score threshold, wherein in response to the lane score being greater than the lane score threshold, the vehicle is maintained operating in the current lane.

4. The method according to claim 1, further comprising: The shape of the boundary region is dynamically adjusted to relate to the geometry of the adjacent lanes.

5. The method according to claim 1, wherein: The one or more sensors include a radar sensor and the sensor data includes radar point cloud data.

6. A system for detecting and mitigating unclassified hazards for a driver assistance system of a vehicle, the system comprising: a data storage device, the data storage device comprising map data, the map data comprising information related to the road including lane geometry; one or more sensors of the vehicle, the one or more sensors configured to generate sensor data indicative of one or more sensed objects in a vicinity of the vehicle; as well as a controller in operable communication with the data storage device and the one or more sensors, the controller being configured by one or more processors of the controller to: receiving the map data and the sensor data; filtering the sensor data to ignore at least one of the one or more sensed objects that is outside a boundary region, the boundary region comprising a portion of an adjacent lane adjoining a current lane in which the vehicle is operating, the boundary region having a size determined at least in part by the map data and a current position of the vehicle; determining whether an unclassified hazard exists in the adjacent lane based at least in part on a number of the one or more sensed objects located within the boundary region; as well as In response to determining that the unclassified hazard exists in the adjacent lane, maintaining the vehicle operating in the current lane.

7. The system according to claim 6, wherein: The controller is configured to, by the one or more processors, prevent an autonomous lane change by the driver assistance system in response to determining that the unclassified hazard is present in the adjacent lane.

8. The system according to claim 6, wherein: The controller is configured by the one or more processors to: determining a confidence score based at least in part on a number of the one or more sensed objects located within the boundary region; determining a lane score based on the confidence score; as well as The lane score is compared to a lane score threshold, wherein in response to the lane score being greater than the lane score threshold, the vehicle is maintained operating in the current lane.

9. The system according to claim 6, wherein: The controller is configured to dynamically adjust, by the one or more processors, a shape of the boundary region to correlate with a geometry of the adjacent lane.

10. The system according to claim 6, wherein: The one or more sensors include a radar sensor and the sensor data includes radar point cloud data.

Images