System and method for obstacle proximity detection

Abstract

Embodiments for assisting in steering a vehicle in response to detecting an object in the vicinity of the vehicle are described. The described embodiments include: receiving an image of a lane indicator corresponding to a lane of travel of the vehicle from an image capture device; determining reference values ​​corresponding to a position of the lane indicator relative to the vehicle based on the image; associating each reference value with a value for locating a point in a reference system related to a direction of travel of the vehicle; receiving position information of an object in an environment external to the vehicle from a radar sensor; determining an assigned lane of the object based on the values ​​and the position information; detecting an impending change of the lane of travel of the vehicle to the assigned lane of the object; and generating a steering control value based on the impending change.

Description

Technical Field

[0001] The present disclosure relates to vehicles, and more particularly to systems and methods for detecting obstacles in the vicinity of a vehicle. Background Art

[0002] Over the past few decades, advances in occupant safety in vehicles have played a major role in reducing the number of deaths and injuries. These advances include passive safety features (e.g., seat belts, airbags, chassis structure design, etc.) as well as active safety features (e.g., electronic stability control, anti-lock braking systems, adaptive cruise control, automatic braking systems, etc.). Active safety features are critical to avoiding collisions or reducing the severity of collisions.

[0003] Typically, vehicles are provided with various functions to monitor the conditions of other moving vehicles in the vicinity of the vehicle. These functions can enable the vehicle to detect the presence of other vehicles and obstacles. Such functions can also warn the operator of the vehicle or control the operation of the vehicle in response to various conditions of other vehicles or obstacles. Summary of the invention

[0004] The present disclosure generally relates to systems and methods for detecting obstacles near a vehicle. One aspect of the disclosed embodiments includes a long-range object detection system for a vehicle. The long-range object detection system includes a processor and a memory. The memory includes instructions that, when executed by the processor, cause the processor to: receive an image of a lane indicator corresponding to a lane of travel of the vehicle from an image capture device; based on the image, determine reference values ​​corresponding to a position of the lane indicator relative to the vehicle; associate each reference value with a value for locating a point in a reference system associated with a direction of travel of the vehicle; receive position information of an object in an environment external to the vehicle from a radar sensor; determine an assigned lane of the object based on the value and the position information; detect an impending change of the lane of travel of the vehicle to the assigned lane of the object; generate a steering control value based on the impending change; and selectively control the steering of the vehicle based on the steering control value.

[0005] Another aspect of the disclosed embodiment includes a method for assisting vehicle steering in response to detecting an object near the vehicle. The method includes receiving an image of a lane indicator corresponding to a lane of travel of the vehicle from an image capture device. The method also includes determining, based on the image, a reference value corresponding to a position of the lane indicator relative to the vehicle. The method also includes associating each reference value with a value for locating a point in a reference system associated with a direction of travel of the vehicle. The method also includes receiving position information of an object in an environment external to the vehicle from a radar sensor. The method also includes determining an assigned lane for the object based on the value and the position information; and detecting an impending change from the lane of travel of the vehicle to the assigned lane of the object. The method also includes generating a steering control value based on the impending change. The method also includes selectively controlling the steering of the vehicle based on the steering control value.

[0006] Another aspect of the disclosed embodiment includes a long-range object detection system for a vehicle. The long-range object detection system includes a radar sensor and an image capture device. The radar sensor is configured to detect an object in an environment outside the vehicle. The image capture device is configured to capture an image representing an environment within a field of view of the image capture device. The long-range object detection system also includes a processor; and a memory. The memory includes instructions that, when executed by the processor, cause the processor to: receive an image from the image capture device, wherein the image is a lane indicator corresponding to a lane of travel of the vehicle; based on the image, determine reference values ​​corresponding to a position of the lane indicator relative to the vehicle; associate each reference value with a value for locating a point in a reference system associated with a direction of travel of the vehicle; receive position information of an object from the radar sensor; determine an assigned lane of the object based on the value and the position information; detect an impending change of the lane of travel of the vehicle to the assigned lane of the object; generate a steering control value based on the impending change; and selectively control the steering of the vehicle based on the steering control value.

[0007] These and other aspects of the disclosure are disclosed in the following detailed description of the embodiments, the appended claims and the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] The present disclosure is best understood by the following detailed description when read in conjunction with the accompanying drawings. It is emphasized that, according to common practice, the various features of the drawings are not drawn to scale. Instead, the sizes of the various features are arbitrarily expanded or reduced for clarity.

[0009] Figure 1 A vehicle in accordance with the principles of the present disclosure is generally shown.

[0010] Figure 2 Generally illustrated is the use of the position of a lane indicator ahead of a vehicle to infer the position of a lane indicator behind the vehicle on a curved road in accordance with the principles of the present disclosure.

[0011] Figure 3 A long-range object detection system in accordance with the principles of the present disclosure is generally shown.

[0012] Figure 4 Points on a lane indicator within a vehicle's reference frame are generally shown in accordance with the principles of the present disclosure.

[0013] Figure 5 Vehicle lane designation assignments according to the principles of the present disclosure are generally shown.

[0014] Figures 6 to 8 A long-range object detection system in accordance with the principles of the present disclosure is generally shown.

[0015] Fig. 9 is a flow chart generally illustrating a method of long-range object detection according to the principles of the present disclosure. DETAILED DESCRIPTION

[0016] The following discussion is directed to various embodiments of the present disclosure. Although one or more of these embodiments may be preferred, the disclosed embodiments should not be interpreted or otherwise used as limiting the scope of the present disclosure, including the claims. In addition, it will be understood by those skilled in the art that the following description has a wide range of applications, and the discussion of any embodiment is intended only to be an exemplary discussion of that embodiment, and is not intended to imply that the scope of the present disclosure, including the claims, is limited to that embodiment.

[0017] Over the past few decades, advances in occupant safety in vehicles have played a major role in reducing the number of deaths and injuries. These advances include passive safety features (e.g., seat belts, airbags, chassis structure design, etc.) as well as active safety features (e.g., electronic stability control, anti-lock braking systems, adaptive cruise control, automatic braking systems, etc.). Active safety features are critical to avoiding collisions or reducing the severity of collisions.

[0018] Typically, vehicles are provided with various functions to monitor the conditions of other moving vehicles in the vicinity of the vehicle. These functions can enable the vehicle to detect the presence of other vehicles and obstacles. Such functions can also warn the operator of the vehicle or control vehicle maneuvers in response to various conditions of other vehicles or obstacles.

[0019] While these features provide the operator with more safety and improved functionality, a constant hazard to the operator is the possibility of collision with an obstacle in the vehicle's blind spot, which is an area around the vehicle that the operator cannot directly observe. In conventional techniques, the challenge is to reliably detect the vehicle and / or objects in the vicinity of the vehicle. One particular challenge is assigning a detected object to a specific adjacent lane.

[0020] The systems and methods described herein can be configured to address the above-mentioned problems by assisting in steering a vehicle in response to detecting an object near the vehicle. For example, an image of a lane indicator corresponding to a lane of travel of the vehicle is received from an image capture device, and a reference value is determined based on the image. The reference value corresponds to the position of the lane indicator relative to the vehicle, and each reference value is associated with a value for locating a point within a reference system related to the direction of travel of the vehicle. Position information of an object in the external environment of the vehicle is also received from a radar sensor, and an assigned lane of the object is determined based on the value and the position information. In addition, an impending change of the vehicle's lane of travel to the object's assigned lane is detected, a steering control value is generated based on the impending change, and the steering of the vehicle is selectively controlled based on the steering control value.

[0021] To explore the foregoing in more detail, we will now describe Figure 1 . Figure 1 A vehicle 10 is generally shown in accordance with the principles of the present disclosure. Vehicle 10 may include any suitable vehicle, such as a car, truck, sport utility vehicle, minivan, crossover, any other passenger vehicle, any suitable commercial vehicle, or any other suitable vehicle. Although vehicle 10 is illustrated as a passenger vehicle having wheels and used on a road, the principles of the present disclosure may be applied to other vehicles, such as airplanes, ships, trains, drones, or other suitable vehicles.

[0022] The vehicle 10 includes a vehicle body 12 and a hood 14. A passenger compartment 18 is at least partially defined by the vehicle body 12. Another portion of the vehicle body 12 defines an engine compartment 20. The hood 14 is movably attached to a portion of the vehicle body 12 such that the hood 14 provides access to the engine compartment 20 when the hood 14 is in a first position or an open position and covers the engine compartment 20 when the hood 14 is in a second position or a closed position. In some embodiments, the engine compartment 20 may be located at the rear of the vehicle 10 (compared to what is generally shown).

[0023] The passenger compartment 18 may be disposed rearward of the engine compartment 20, but in embodiments where the engine compartment 20 is disposed in a rearward portion of the vehicle 10, the passenger compartment 18 may be disposed forward of the engine compartment 20. The vehicle 10 may include any suitable propulsion system, including an internal combustion engine, one or more electric motors (e.g., an electric vehicle), one or more fuel cells, a hybrid (e.g., a hybrid vehicle) propulsion system including a combination of an internal combustion engine, one or more electric motors, and / or any other suitable propulsion system.

[0024] In some embodiments, the vehicle 10 may include a gasoline engine or a gasoline fuel engine, such as a spark ignition engine. In some embodiments, the vehicle 10 may include a diesel fuel engine, such as a compression ignition engine. The engine compartment 20 houses and / or surrounds at least some components of the propulsion system of the vehicle 10. Additionally or alternatively, a propulsion control device (e.g., an accelerator actuator (e.g., an accelerator pedal), a brake actuator (e.g., a brake pedal), a steering wheel, and other such components) are disposed in the passenger compartment 18 of the vehicle 10. The propulsion control device may be actuated or controlled by a driver of the vehicle 10 and may be directly connected to corresponding components of the propulsion system, such as a throttle, brake, axle, vehicle transmission, etc. In some embodiments, the propulsion control device may communicate a signal to a vehicle computer (e.g., drive-by-wire), which in turn may control corresponding propulsion components of the propulsion system. Thus, in some embodiments, the vehicle 10 may be an autonomous vehicle.

[0025] In some embodiments, the vehicle 10 includes a transmission that communicates with the crankshaft via a flywheel or a clutch or a fluid coupling. In some embodiments, the transmission includes a manual transmission. In some embodiments, the transmission includes an automatic transmission. In the case of an internal combustion engine or a hybrid vehicle, the vehicle 10 may include one or more pistons that operate in conjunction with the crankshaft to generate force that is transmitted to one or more shafts through the transmission, which causes the wheels 22 to rotate. When the vehicle 10 includes one or more electric motors, the vehicle battery and / or fuel cell provides energy to the electric motors to rotate the wheels 22.

[0026] The vehicle 10 may include an automatic vehicle propulsion system, such as cruise control, adaptive cruise control, automatic braking control, other automatic vehicle propulsion systems, or combinations thereof. The vehicle 10 may be an automatic or semi-automatic vehicle, or other suitable type of vehicle. The vehicle 10 may include additional features or fewer features than those generally shown and / or disclosed herein.

[0027] In some embodiments, the vehicle 10 may include an Ethernet component 24, a controller area network component (CAN) 26, a media-oriented system transport component (MOST) 28, a FlexRay component 30 (e.g., a wire-controlled brake system, etc.), and a local interconnect network component (LIN) 32. In some embodiments, the vehicle 10 is configured for domain control with over-the-air programming support. For example, as described, the vehicle 10 can receive updates for any suitable software components of the vehicle 10 via the Internet (e.g., or other suitable network). The vehicle 10 can update or change software components based on the update. The vehicle 10 may include additional or fewer features than those generally shown and / or disclosed herein.

[0028] In some embodiments, the vehicle 10 includes one or more radar sensors that provide position information corresponding to objects detected near the vehicle 10. For example, the one or more radar sensors may provide speed and position information of the detected objects relative to the vehicle 10. The position information and at least one image of a lane indicator corresponding to a lane of travel of the vehicle 10 captured by an image capture device may be used to determine a lane of travel (e.g., same, left, right, etc.) of the detected object behind or to the side of the vehicle 10.

[0029] For example, a rear-facing camera may be used for this purpose. However, if the vehicle 10 already uses a front-facing camera, this adds additional cost. Images captured from the front-facing camera can be used to determine the location of lane indicators behind and / or to the side of the vehicle 10, but images captured while the vehicle 10 is traversing forward cannot be used to infer the location of lane indicators behind and / or to the side of the vehicle 10. Any data from these images used to infer the location of the lane indicators may result in an inaccurate depiction of the location of the lane indicators (particularly on curved roads). Figure 2 Provided is the use of the position of lane indicators in front of the vehicle 10 on a curved road to infer the position of lane indicators behind and / or to the side of the vehicle 10. Figure 2 As generally shown in FIG. 1 , if the position of lane indicator 204 in front of vehicle 10 is used to infer the position of lane indicators behind vehicle 10, then the position of lane indicators behind and / or to the side of vehicle 10 is determined to be curved (as represented by lane indicator 202). Thus, based on the inferred position of lane indicator 202, vehicle 206 will be determined to be located in the same lane as vehicle 10 rather than in a lane adjacent to vehicle 10.

[0030] To prevent this, the vehicle 10 may include Figure 3 The long-range object detection system 300 is generally shown. The system 300 can be configured to assist an operator of the vehicle 10 in determining the lane of travel of an object detected near the vehicle 10 and selectively control the steering of the vehicle 10 in response to detecting an impending change in the lane of travel of the vehicle 10 to the assigned lane of the object. Figure 3 , system 300 may include a controller 302, one or more input devices 304, and memory 306. Controller 302 may include any suitable controller, such as a vehicle electronic control unit, a processor, or any other suitable controller, such as those described herein.

[0031] The controller 302 may be in communication with a memory 306. The memory 306 may include any suitable non-volatile or volatile memory. The memory 306 may include a memory array, a memory storage device, or any other suitable memory. The memory 306 may include instructions that, when executed by the controller 302, cause the controller 302 to at least control the steering of the vehicle 10. Additionally or alternatively, the instructions, when executed by the controller 302, may cause the controller 302 to perform various other functions of the vehicle 10. The controller 302 may also be configured to receive input from the input device 304.

[0032] The input device 304 may include any suitable input device or multiple suitable input devices. For example, the input device 304 may include an image capture device (e.g., a camera or other suitable image capture device), a radar sensor, a LIDAR sensor, a global positioning system (GPS) sensor, a sonar sensor, other suitable sensors, or a combination thereof. More specifically, in some embodiments, the input device 304 may include a view capture device (e.g., a front-facing camera) that captures an image of the external environment of the vehicle 10 within the field of view of the image capture device. The field of view may include a portion of the external environment of the vehicle 10 that is within the driving path of the vehicle 10. The controller 302 is configured to process the image data and / or other types of data collected by the input device 304. Additionally or alternatively, the controller 302 may use the convolutional neural network described herein to detect one or more lane indicators.

[0033] In some embodiments, the controller 302 is configured to receive data indicating the position of the lane indicator relative to the vehicle 10 (e.g., an image of the lane indicator), analyze the data to determine reference values ​​corresponding to the position of the lane indicator relative to the vehicle, and store the reference values ​​in the memory 306. The input from the input device 304 may indicate the position of the left lane indicator and the right lane indicator relative to the vehicle 10. In some embodiments, the controller 302 may analyze the input to determine the reference values, which represent the offset or distance of the left lane indicator and the right lane indicator from the vehicle 10 (e.g., from the center or edge of the vehicle 10). In some embodiments, the controller 302 may store the reference values ​​(e.g., 150 or more reference values) in an array in the memory 306.

[0034] In some embodiments, the controller 302 is further configured to associate each reference value with a value for a point located within a reference system associated with the direction of travel of the vehicle 10 (e.g., a heading angle of the vehicle 10). For example, as described above, the controller 302 may determine a reference value corresponding to the position of the lane indicator relative to the vehicle 10. The controller 302 may then use a coordinate transformation to associate each reference value with a value for a point located within the reference system associated with the direction of travel of the vehicle 10. In some embodiments, the reference points may be iteratively associated with the values ​​within the reference system.

[0035] To help further illustrate, we will now describe Figure 4 . Figure 4 1 illustrates how to associate a reference value with a value for locating a point in a reference system related to the direction of travel of the vehicle 10. For example, Figure 4 , as the vehicle 10 moves forward in a lane 402, the controller 302 may receive captured image data from a front-facing camera of an input device 304. The controller 302 may analyze the image data to determine reference values ​​corresponding to a position of a right lane indicator 406 relative to the vehicle 10 and a position of a left lane indicator 404 relative to the vehicle 10 at several time instances. According to this example, the controller 302 may represent the reference values ​​as a point 408 (representing the position of the left lane indicator 404) and a point 410 (representing the position of the right lane indicator 406) within a reference system or coordinate system 412, in which the center of the vehicle 10 is used as the origin. The points 408 and 410 may uniquely identify the positions of the right lane indicator 406 and the left lane indicator 404 within the coordinate system 412.

[0036] The controller 302 may then transform the points 408 and 410 to account for the direction of travel of the vehicle 10. For example, the points 408 and 410 within the reference frame 414 may be transformed to the reference frame 414 associated with the azimuth (α). The points 408 and 410 may be represented as points 408' and 410', respectively, in the reference frame 414. For example, in some embodiments, the relationship between the points 408 and 408' and the relationship between the points 410 and 410' may be represented by the following matrices:

[0037]

[0038] Where X = 0, Y = lane indicator offset from the vehicle, h = -(vehicle speed)*dt,

[0039] k=0,andα=-(yaw rate)*dt

[0040] Furthermore, in some embodiments, Figure 3The input device 304 generally shown in FIG. 3 may be configured to measure various characteristics of the vehicle 10 and / or the external environment of the vehicle 10. The input device 304 may communicate the measured characteristics to the controller 302. For example, the vehicle and steering signals may include measured signals received from one or more sensors, or may include estimated or calculated signals from one or more processors in the vehicle 10. The vehicle and steering signals may include yaw rate, vehicle speed, vehicle acceleration, steering angle, input torque applied to the steering wheel of the vehicle 10, etc. The controller 302 may use the vehicle and steering signals (e.g., yaw rate and vehicle speed) to associate point 408 and point 408' and point 410 and point 410'. In some embodiments, point 408' and point 410' may be stored in Figure 3 in the memory 306.

[0041] In some embodiments, as described, input device 304 may also include one or more radar sensors. Controller 302 may receive position information corresponding to objects detected in the vicinity of vehicle 10 from one or more sensors. For example, input device 304 (e.g., a rear corner radar of vehicle 10) may detect an object (e.g., Figure 2 vehicle 206 ) and provides input to the controller 302 to indicate location information corresponding to an object detected near the vehicle 10 .

[0042] In addition, the controller 302 may be configured to determine the lane of travel of the object (e.g., the right lane of the vehicle 10, the left lane of the vehicle 10, the same lane as the vehicle 10, etc.) based on the points 408' and 410' and the position information stored in the memory 306. In some embodiments, the object may include a stationary object in a lane adjacent to the vehicle 10, including a traffic cone / sign, a pedestrian, a wall, a road divider, or any other such object with which the vehicle 10 may collide.

[0043] To help further illustrate, we will now describe Figure 5 .like Figure 5 As generally shown in FIG. 5 , vehicle 206 is behind vehicle 10 in adjacent lane 502 . Figure 4 Point 408' represents the position of the right lane indicator relative to vehicle 10 in lane 402. Controller 302 can receive position information corresponding to vehicle 206 and analyze the position information to determine that the point in front of vehicle 206 is located at a Y distance from the side of vehicle 10 in the lateral direction, and the point in front of vehicle 206 is located at an X distance from the rear of vehicle 10.

[0044] The controller 302 may also be configured to determine the proximity of the vehicle 206 to the lane indicator represented by the point 408'. The controller 302 may compare the X distance to the X' value of the point 408' to approximate the position of the vehicle 206 behind the vehicle 10 relative to the lane indicator represented by the point 408'. For example, the X' value closest to X in the point 408' may indicate the minimum distance that the vehicle 206 is behind the vehicle 10. The controller 302 may also compare the Y distance to the Y' value of the point 408' to determine the distance 504 between the vehicle 206 and the lane indicator represented by the point 408'. Based on the distance 504, the controller 302 may determine the lane of travel of the vehicle 206. For example, if Y is greater than the distance 504, then the controller 302 may assign a sign to the vehicle 206 to indicate that the vehicle 206 is in an adjacent lane (e.g., in this case, the "right" lane is assigned to the vehicle 206). Conversely, where Y is less than distance 504 , controller 302 may assign a sign to vehicle 206 to indicate that vehicle 206 is in the same lane as vehicle 10 .

[0045] In some embodiments, the controller 302 may be configured to detect an impending change in the lane of travel of the vehicle 10 to the assigned lane of the object, such as Figure 3 . For example, the controller 302 may use the measured angular position of the steering mechanism to determine the steering direction of the vehicle 10. The steering direction of the vehicle 10 may include a direction (e.g., left or right) from the operator's perspective. The steering direction indicates the direction of the lane change.

[0046] In some embodiments, an operator of vehicle 10 may indicate a direction to change lanes by actuating a turn signal of the vehicle to a first position (e.g., indicating a first side of vehicle 10) or a second position (e.g., indicating an opposite side of vehicle 10). Controller 302 may be configured to receive input indicating a position of a turn signal. For example, controller 302 may receive input indicating a turn signal in a first position, which may indicate an impending change of a lane of travel of vehicle 10 to a right adjacent lane of travel. Conversely, input may indicate a turn signal in a second position, which may indicate an impending change of a lane of travel of vehicle 10 to a left adjacent lane of travel. These impending lane change detection techniques may be implemented in both autonomous vehicles and vehicles operated by an operator.

[0047] In some embodiments, the controller 302 may receive input from a driver monitoring system that measures various characteristics of the interior environment of the vehicle 10 (e.g., in the cabin). The driver monitoring system may include one or more image capture devices, such as a camera, a passive infrared sensor, or a wearable device that tracks the movement of the operator. The driver monitoring system captures image data of the operator. The controller 302 may use the image data to detect an impending change in the driving lane of the vehicle 10. For example, the controller 302 may receive image data of the location of the turn signal from the driver monitoring system. In some cases, the operator may not use the turn signal (e.g., because the vehicle 10 does not have a turn signal, because it is more troublesome to use the turn signal, etc.). In some embodiments, because the driver monitoring system can use a trained convolutional neural network to determine where the driver is looking. The controller 302 may be configured to determine the operator's intention to change the driving lane based on where the driver is looking. Additionally or alternatively, the driver monitoring system may track additional indications (e.g., the operator's head posture or movement, gaze vector, etc.), and the controller 302 may use additional indications to determine that the driver intends to change the lane of the vehicle 10.

[0048] In some embodiments, the controller 302 is further configured to generate a steering control value based on an impending change. For example, if the controller 302 determines that a lane change is imminent, the controller 302 may calculate a time to collision (TTC) between the vehicle 10 and the object and selectively control the steering of the vehicle 10 based on the calculation.

[0049] Figure 6 Another exemplary embodiment of a long-range object detection system 300 is generally shown (e.g., as Figure 3 ). Figure 6 As generally shown in FIG. 1 , the controller 302 is configured to communicate with the steering system 40 that generates assist torque. In some embodiments, the controller 302 is a “blind spot” assist system that detects distant objects in a blind spot of the vehicle 10 .

[0050] In some embodiments, controller 302 is part of a steering system (e.g., steering system 40) that performs one or more operations based on output signals from one or more sensors 312. Alternatively or additionally, controller 302 is a separate processing unit, such as an electronic circuit unit (ECU). In some embodiments, controller 302 includes one or more computer executable instructions that are read and executed by the processing unit.

[0051] The controller 302 may use one or more sensors 312 that the vehicle 10 is equipped with, such as cameras, radars, LIDARs, ultrasonic sensors, etc. In addition, other sensors, such as GPS, may be used. In addition, vehicle and steering signals 314 also provide input to the controller 302. The vehicle and steering signals 314 may include measurement signals received from one or more sensors 312 (e.g., those in the steering system 40), or may include estimated or calculated signals from one or more ECUs in the vehicle 10. The vehicle and steering signals 314 may include yaw rate, vehicle speed, vehicle acceleration, steering angle, input torque applied to the steering wheel, etc. The vehicle and steering signals 314 and the one or more sensors 312 are Figure 3 An exemplary embodiment of an input device 304 is provided.

[0052] like Figure 6 As generally shown in FIG. 3 , the controller 302 may include a sensor fusion system 316 and a situation analysis system 318. In some embodiments, a logical combination of two or more sensors 312 may be used by the sensor fusion system 316. The sensor fusion system 316 is responsible for receiving input signals from the sensors 312 and other vehicle / turn signals 314 and processing the input signals using one or more sensor fusion techniques. The sensor fusion system 316 facilitates combining sensory data or data derived from different sources, sensors 312, and vehicle / turn signals 314, so that the resulting information has less uncertainty than using the sources alone.

[0053] The sensor fusion system 316 uses the input signals to monitor the surrounding environment within a predetermined proximity of the vehicle 10 (e.g., a predetermined distance from the vehicle 10). The sensor fusion system 316 detects one or more long-range objects in the surrounding environment and analyzes whether a potential collision of the vehicle 10 with the detected long-range objects is likely. The detected long-range objects may include other vehicles (long-range vehicles or target vehicles), traffic cones / signs, lane markings, pedestrians, walls, road dividers, or any other such objects with which the vehicle 10 may collide.

[0054] The sensor fusion system 316 calculates the TTC with a distant object that is detected using input signals and one or more known techniques. For example, the sensor fusion system 316 calculates the distance and speed of a distant vehicle that is close to the vehicle 10, in front of the vehicle 10, or in the same lane as the vehicle 10. The sensor fusion system 316 can also calculate the distance and speed of an incoming vehicle in an adjacent lane of the vehicle 10. For example, the distant object detection system 300 provides blind spot assistance via the steering system 40. In some embodiments, the sensor fusion system 316 sends a front TTC signal to a situation analysis system 318, which is used to determine what steering action can be taken (e.g., buzzing, reducing assistance, torque overlay, position control). The front TTC signal indicates the TTC with a target vehicle located in front of the vehicle 10.

[0055] Figure 7 An example scenario in which the long-distance object detection system 300 provides steering assistance according to one or more embodiments is described. It should be noted that the described scenario is only an example, and the technical solution of this article can also be implemented in other scenarios. Figure 7 The scenario depicts the controller 302 using one or more input signals from the front camera and the data from the two rear corner radar sensors. As shown, the first area 710 represents the field of view (FOV) of the camera, the second area 720 represents the field of view of the radar sensor, and the third area 722 represents the overlap area between the two (or more) radar sensors.

[0056] refer to Figure 6 For example, in the second area 720 of the vehicle 10, the radar signal is used to keep track of the target 725 that is close to the vehicle 10. The turn signal 314 and the lane indicator of the lane 705 are also used to predict the lane change state of the vehicle 10. The lane change state indicates whether the vehicle 10 intends to perform a lane change maneuver (left lane change, right lane change, etc.) or continue to travel in the lane 705.

[0057] A situation analysis system 318, which uses the lane change status, the front collision time, and the side collision time to determine whether to trigger one or more levels of steering intervention. For example, if there is an impending front collision determined based on the front collision time, the side collision prevention function is switched to OFF using a flag provided to the steering system 40. Alternatively or additionally, based on the side collision time, a steering intervention is triggered to prevent a side collision. The steering intervention may include scaling an assist torque generated based on the driver input torque, a scaling factor for scaling based on the analysis of the situation analysis system 318. Alternatively or additionally, the steering intervention may include generating an overlay torque to counteract or further assist the input torque from the driver. For example, the steering system 40 determines one or more levels of steering intervention.

[0058] As described, TTC is calculated based on the object proximity and relative speed. Additionally, in some embodiments, a steering action for controlling the steering of the vehicle 10 is taken based on the TTC and the intention to change lanes. For example, depending on the proximity of the object to the vehicle 10, a more invasive steering action may need to be taken. Figure 8 , an object detected in region A may prompt a steering intervention, which includes generating an overlay torque to resist or further assist the input torque from the driver. In contrast, an object detected in region C may not trigger a steering intervention, but rather provide a warning to the operator of the vehicle 10.

[0059] In addition, to prevent false alarms, the radar sensor of the vehicle 10 can track the number of times the signal is returned from an area within a predetermined distance of the vehicle. For example, the radar sensor can count the number of times the signal bounces back from an area within a predetermined distance. The controller 302 can then generate a steering control value based on the number of times the signal returns and the predetermined distance. For example, if the number of times the signal is returned is lower than a threshold (e.g., the signal is returned twenty-five times), the controller 302 can generate a steering control value indicating that the steering of the vehicle 10 is not interfered with, because the detected object may no longer be within the predetermined distance. On the contrary, if the number of times the signal is returned is greater than the threshold, the controller 302 can generate a steering control value indicating intervention in the steering of the vehicle 10, and depending on which area within the predetermined distance the object is detected, the control of the steering of the vehicle 10 is more or less intrusive.

[0060] Fig. 9 is a flow chart generally illustrating a method 900 for providing long-range object assistance according to the principles of the present disclosure. At 902, the method 900 receives an image of a lane indicator corresponding to a lane of travel of a vehicle from an image capture device. For example, Figure 3 and Figure 6 The controller 302 receives an image of a lane indicator corresponding to a driving lane of the vehicle from an image capture device.

[0061] At 904, method 900 determines a reference value corresponding to the position of the lane indicator relative to the vehicle based on the image. For example, Figure 3 and Figure 6 The controller 302 determines a reference value corresponding to the position of the lane indicator relative to the vehicle based on the image.

[0062] At 906, method 900 associates each reference value with a value for locating a point within a reference frame associated with the direction of travel of the vehicle. Figure 3 and Figure 6 The controller 302 associates each reference value with a value for locating a point within a reference system related to the direction of travel of the vehicle.

[0063] At 908, method 900 receives position information of objects in the environment outside the vehicle from a radar sensor. For example, Figure 3 and Figure 6 The controller 302 receives position information of objects in the environment outside the vehicle from the radar sensor.

[0064] At 910, method 900 determines an assigned lane for the object based on the value and the location information. For example, Figure 3 and Figure 6 The controller 302 determines an assigned lane for the object based on the value and position information.

[0065] At 912, method 900 detects an impending change in the vehicle's lane of travel to the object's lane of travel. For example, Figure 3 and Figure 6 The controller 302 detects an impending change of the vehicle's lane of travel to the subject's lane of travel.

[0066] At 914, method 900 generates a steering control value based on the upcoming change. For example, Figure 3 and Figure 6 The controller 302 generates a steering control value based on the impending change.

[0067] At 916, method 900 selectively controls the steering of the vehicle based on the steering control value. For example, Figure 3 and Figure 6 The controller 302 selectively controls the steering of the vehicle based on the steering control value.

[0068] In some embodiments, a long-range object detection system for a vehicle includes: a processor and a memory including instructions that, when executed by the processor, cause the processor to: receive an image of a lane indicator corresponding to a lane of travel of the vehicle from an image capture device; determine, based on the image, reference values ​​corresponding to a position of the lane indicator relative to the vehicle; associate each reference value with a value for locating a point in a reference system related to a direction of travel of the vehicle; receive position information of an object in an environment external to the vehicle from a radar sensor; determine an assigned lane of the object based on the values ​​and the position information; detect an impending change from the lane of travel of the vehicle to the assigned lane of the object; generate a steering control value based on the impending change; and selectively control the steering of the vehicle based on the steering control value.

[0069] In some embodiments, the instructions further cause the processor to: associate each reference value with the value using a coordinate transformation.

[0070] In some embodiments, the instructions further cause the processor to: determine, based on the image, a reference value corresponding to a position of another lane indicator relative to the vehicle.

[0071] In some embodiments, the instructions further cause the processor to: determine a proximity of the object to the lane indicator based on the value and the position information.

[0072] In some embodiments, the object is located in a blind spot of the vehicle.

[0073] In some embodiments, the image represents a portion of the vehicle's external environment that is within the field of view of the image capture device, and the position information indicates that the object is located in a blind spot of the vehicle.

[0074] In some embodiments, the image is a front view of the vehicle.

[0075] In some embodiments, a method for assisting vehicle steering in response to detecting an object near the vehicle includes: receiving an image of a lane indicator corresponding to a lane of travel of the vehicle from an image capture device; based on the image, determining reference values ​​corresponding to the position of the lane indicator relative to the vehicle; associating each reference value with a value for locating a point in a reference system related to a direction of travel of the vehicle; receiving position information of an object in an environment external to the vehicle from a radar sensor; determining an assigned lane of the object based on the value and the position information; detecting an impending change from the vehicle's lane of travel to the object's assigned lane; generating a steering control value based on the impending change; and selectively controlling the steering of the vehicle based on the steering control value.

[0076] In some embodiments, associating each reference value with a value is performed using a coordinate transformation.

[0077] In some embodiments, the method further comprises determining, based on the image, a reference value corresponding to a position of another lane indicator relative to the vehicle.

[0078] In some embodiments, the method further includes determining a proximity of the object to the lane indicator based on the value and the position information.

[0079] In some embodiments, the object is located in a blind spot of the vehicle.

[0080] In some embodiments, the image represents a portion of the vehicle's external environment that is within the field of view of the image capture device, and the position information indicates that the object is in a blind spot of the vehicle.

[0081] In some embodiments, the image is a front view of the vehicle.

[0082] In some embodiments, a long-range object detection system for a vehicle includes: a radar sensor configured to detect objects in an environment external to the vehicle; an image capture device configured to capture an image representing an environment within a field of view of the image capture device; a processor; and a memory including instructions that, when executed by the processor, cause the processor to: receive an image from the image capture device, wherein the image is a lane indicator corresponding to a lane of travel of the vehicle; based on the image, determine reference values ​​corresponding to a position of the lane indicator relative to the vehicle; associate each reference value with a value for locating a point in a reference system related to a direction of travel of the vehicle; receive position information of the object from the radar sensor; determine an assigned lane of the object based on the values ​​and the position information; detect an impending change from the vehicle's lane of travel to the object's assigned lane; generate a steering control value based on the impending change; and selectively control steering of the vehicle based on the steering control value.

[0083] In some embodiments, the instructions further cause the processor to: associate each reference value with the value using a coordinate transformation.

[0084] In some embodiments, the instructions further cause the processor to: determine, based on the image, a reference value corresponding to a position of another lane indicator relative to the vehicle.

[0085] In some embodiments, the instructions further cause the processor to: determine a proximity of the object to the lane indicator based on the value and the position information.

[0086] In some embodiments, the object is located in a blind spot of the vehicle.

[0087] In some embodiments, the radar sensor is further configured to track the number of times the signal returns from an area within a predetermined distance of the vehicle; and the instructions further cause the processor to: generate a steering control value based on the number of returns of the signal and the predetermined distance.

[0088] The above discussion is intended to illustrate the principles and various embodiments of the present invention. Once the above disclosure is fully understood, many changes and modifications will become apparent to those skilled in the art. It is intended that the following claims be interpreted to include all such changes and modifications.

[0089] The word "example" is used herein to mean used as an example, instance or illustration. Any aspect or design described herein as an "example" is not necessarily to be interpreted as being more preferred or advantageous than other aspects or designs. On the contrary, the use of the word "example" is intended to present the concept in a specific way. As used in this application, the term "or" is intended to represent an inclusive "or" rather than an exclusive "or". That is, unless otherwise specified or clearly seen from the context, "X includes A or B" is intended to represent any natural inclusive arrangement. That is, if X includes A; X includes B; or X includes both A and B, then "X includes A or B" is satisfied in any of the foregoing cases. In addition, the article "a / an" used in this application and the appended claims should generally be interpreted as meaning "one or more", unless otherwise specified or clearly pointed to the singular form from the context. In addition, unless so described, the use of the term "embodiment" or "one embodiment" throughout the text is not intended to represent the same embodiment or embodiment.

[0090] The implementation of the systems, algorithms, methods, instructions, etc. described herein may be implemented in hardware, software, or any combination thereof. The hardware may include, for example, a computer, an intellectual property (IP) core, an application specific integrated circuit (ASIC), a programmable logic array, an optical processor, a programmable logic controller, microcode, a microcontroller, a server, a microprocessor, a digital signal processor, or any other suitable circuit. In the claims, the term "processor" shall be understood to include any of the foregoing hardware, alone or in combination. The terms "signal" and "data" are used interchangeably.

[0091] As used herein, the term system may include packaged functional hardware units designed for use with other components, a set of instructions executable by a controller (e.g., a processor executing software or firmware), processing circuits configured to perform specific functions, and self-contained hardware or software components that interface with larger systems. For example, a system may include an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), an electrical circuit, a digital logic circuit, an analog circuit, a combination of discrete circuits, gate circuits, and other types of hardware, or a combination thereof. In other embodiments, a system may include a memory that stores instructions executable by a controller to implement features of the system.

[0092] Furthermore, in one aspect, for example, the systems described herein may be implemented using a general purpose computer or general purpose processor with a computer program that, when executed, performs any corresponding method, algorithm, and / or instruction described herein. Additionally or alternatively, for example, a special purpose computer / processor may be utilized that may include other hardware for performing any method, algorithm, or instruction described herein.

[0093] In addition, all or part of the implementation of the present disclosure may take the form of a computer program product accessible from, for example, a computer usable or computer readable medium. A computer usable or computer readable medium may be, for example, any device that can tangibly contain, store, communicate, or transport a program for use by or in conjunction with any processor. The medium may be, for example, an electronic, magnetic, optical, electromagnetic device, or a semiconductor device. Other suitable media may also be used.

[0094] The above embodiments, implementations and aspects have been described to allow easy understanding of the present invention and do not limit the present disclosure. On the contrary, the present invention is intended to cover various modifications and equivalent arrangements included within the scope of the appended claims, which scope should be given the broadest interpretation to cover all such modifications and equivalent structures permitted by law.

Claims

1. A long-range object detection system for a vehicle, comprising: processor; as well as a memory comprising instructions that, when executed by the processor, cause the processor to: receiving, from an image capture device, an image of a lane indicator in front of the vehicle, the lane indicator corresponding to a driving lane of the vehicle; determining, based on the image, a reference value in a first reference frame corresponding to a position of the lane indicator relative to the vehicle, wherein the position of the lane indicator is behind or to the side of the vehicle; associating each reference value with a value in a second reference system related to a direction of travel of the vehicle, wherein the value in the second reference system related to the direction of travel of the vehicle corresponds to a position of the lane indicator to the side or rear of the vehicle, the position describing the direction of travel of the vehicle; receiving position information of objects in the environment outside the vehicle from a radar sensor; determining an assigned lane for the object based on the value in the second reference system associated with the vehicle's direction of travel and the position information; detecting an impending change of a lane of travel of the vehicle to an assigned lane of the object; generating a steering control value based on the impending change; and Steering of the vehicle is selectively controlled based on the steering control value.

2. The long-distance object detection system according to claim 1, wherein: The instructions further cause the processor to: associate each of the reference values ​​with a value in the second reference system associated with the direction of vehicle travel using a coordinate transformation.

3. The long-distance object detection system according to claim 1, wherein: The instructions also cause the processor to determine, based on the image, a reference value corresponding to a position of another lane indicator relative to the vehicle.

4. The long-distance object detection system according to claim 1, wherein: The instructions further cause the processor to determine a proximity of the object to the lane indicator based on the value in the second reference system associated with the vehicle's direction of travel and the position information.

5. The long-distance object detection system according to claim 1, wherein: The object is located in a blind spot of the vehicle.

6. The long-distance object detection system according to claim 1, wherein: The image represents a portion of an environment external to the vehicle that is within a field of view of the image capture device, and the position information indicates that the object is located in a blind spot of the vehicle.

7. The long-distance object detection system according to claim 6, wherein: The image is an image of a front view of the vehicle.

8. A method for assisting steering of a vehicle in response to detecting an object in the vicinity of the vehicle, comprising: receiving, from an image capture device, an image of a lane indicator in front of the vehicle, the lane indicator corresponding to a driving lane of the vehicle; determining, based on the image, a reference value in a first reference frame corresponding to a position of the lane indicator relative to the vehicle, wherein the position of the lane indicator is behind or to the side of the vehicle; associating each reference value with a value in a second reference system related to a direction of travel of the vehicle, wherein the value in the second reference system related to the direction of travel of the vehicle corresponds to a position of the lane indicator to the side or rear of the vehicle, the position describing the direction of travel of the vehicle; receiving position information of objects in the environment outside the vehicle from a radar sensor; determining an assigned lane for the object based on the value in the second reference system associated with the vehicle's direction of travel and the position information; detecting an impending change of a lane of travel of the vehicle to an assigned lane of the object; generating a steering control value based on the impending change; and Steering of the vehicle is selectively controlled based on the steering control value.

9. The method according to claim 8, wherein: Associating each reference value with a value in the second reference system associated with the direction of travel of the vehicle is performed using a coordinate transformation.

10. The method according to claim 8, further comprising: A reference value corresponding to a position of another lane indicator relative to the vehicle is determined based on the image.

11. The method according to claim 8, further comprising: The proximity of the object to the lane indicator is determined based on the value in the second reference system related to the vehicle travel direction and the position information.

12. The method according to claim 8, wherein: The object is located in a blind spot of the vehicle.

13. The method according to claim 8, wherein: The image represents a portion of an environment external to the vehicle that is within a field of view of the image capture device, and the position information indicates that the object is in a blind spot of the vehicle.

14. The method according to claim 13, wherein: The image is an image of a front view of the vehicle.

15. A long-range object detection system for a vehicle, comprising: a radar sensor configured to detect objects in an environment external to the vehicle; an image capture device configured to capture an image representative of an environment within a field of view of the image capture device; Processor; and a memory comprising instructions that, when executed by the processor, cause the processor to: receiving the image from the image capture device, wherein the image is an image of a lane indicator in front of the vehicle, the lane indicator corresponding to a driving lane of the vehicle; determining, based on the image, a reference value in a first reference frame corresponding to a position of the lane indicator relative to the vehicle, wherein the position of the lane indicator is behind or to the side of the vehicle; associating each reference value with a value in a second reference system related to a direction of travel of the vehicle, wherein the value in the second reference system related to the direction of travel of the vehicle corresponds to a position of the lane indicator to the side or rear of the vehicle, the position describing the direction of travel of the vehicle; receiving position information of the object from the radar sensor; determining an assigned lane for the object based on the value in the second reference system associated with the vehicle's direction of travel and the position information; detecting an impending change of a lane of travel of the vehicle to an assigned lane of the object; generating a steering control value based on the impending change; and Steering of the vehicle is selectively controlled based on the steering control value.

16. The long-distance object detection system according to claim 15, wherein: The instructions further cause the processor to: associate each of the reference values ​​with a value in the second reference system associated with the direction of vehicle travel using a coordinate transformation.

17. The long-distance object detection system according to claim 15, wherein: The instructions also cause the processor to determine, based on the image, a reference value corresponding to a position of another lane indicator relative to the vehicle.

18. The long-distance object detection system according to claim 15, wherein: The instructions further cause the processor to determine a proximity of the object to the lane indicator based on the value in the second reference system associated with the vehicle's direction of travel and the position information.

19. The long-distance object detection system according to claim 15, wherein: The object is located in a blind spot of the vehicle.

20. The long-distance object detection system of claim 15, wherein: The radar sensor is further configured to track the number of times a signal returns from an area within a predetermined distance of the vehicle; and The instructions further cause the processor to generate the steering control value based on the number of signal returns and the predetermined distance.

Images