Driver assistance device and driver assistance method

Abstract

Driver assistance device and driver assistance method. A device for assisting the driving of a vehicle includes: a camera mounted on the vehicle and having a field of view outside the vehicle, the camera configured to obtain image data; and a controller configured to process the image data, identify at least one object obstructing the travel of the vehicle based on the image data, predict a collision with the at least one object, identify whether the vehicle is moving forward or backward based on the image data, and control a braking device of the vehicle to brake the vehicle according to whether the vehicle is moving forward or backward.

Description

Technical Field

[0001] The present disclosure relates to a driver assistance device, and more particularly, to a driver assistance device capable of avoiding a collision with an object. Background Art

[0002] Generally, a vehicle refers to a mobile device or transportation device designed to travel on roads or railways using a power source such as fossil fuels or electricity. A vehicle can be moved to various locations primarily using one or more wheels mounted on its body. Such vehicles may include three- or four-wheeled vehicles, two-wheeled vehicles such as motorcycles, construction machinery, bicycles, and trains traveling on railroads arranged on tracks.

[0003] Vehicles are the most common means of transportation in modern society, and the number of people using them is growing. Advances in vehicle technology have made long-distance travel easier and made life more accessible. However, in areas with high population density, road traffic conditions have deteriorated and traffic jams are common.

[0004] In recent years, there has been active research into vehicles equipped with advanced driver assistance systems (ADAS), which proactively provide information about the vehicle status, driver status, and surrounding environment to reduce the burden on the driver while enhancing driver convenience.

[0005] Examples of advanced driver assistance systems installed on vehicles include forward collision avoidance (FCA), autonomous emergency braking (AEB), and driver attention warning (DAW). Such systems are systems for determining the risk of collision with an object in a vehicle's driving situation and providing collision avoidance and warning through emergency braking in the event of a collision.

[0006] Driver assistance devices rely on information received from the engine, transmission, brakes, and steering devices included in the vehicle regarding the vehicle's travel. However, when erroneous information is received from the engine, transmission, brakes, and steering devices, the driver assistance devices may malfunction. Summary of the Invention

[0007] For the above reasons, an aspect of the present disclosure is to provide a driver assistance apparatus and a driver assistance method capable of recognizing forward movement or backward movement of a vehicle based on image data.

[0008] Therefore, one aspect of the present disclosure is to provide a device for assisting the driving of a vehicle, the device comprising: a camera mounted on the vehicle and having a field of view outside the vehicle, the camera being configured to obtain image data; and a controller being configured to process the image data, identify at least one object that obstructs the travel of the vehicle based on the image data, predict a collision with the at least one object, identify whether the vehicle is moving forward or backward based on the image data, and control a braking device of the vehicle to brake the vehicle according to whether the vehicle is moving forward or backward.

[0009] The controller may be configured to control the braking device of the host vehicle to brake the host vehicle in response to forward movement of the host vehicle.

[0010] The controller may be configured to control at least one of a display, audio, and a steering wheel of the host vehicle to warn the host vehicle of a collision in response to the rearward movement of the host vehicle.

[0011] The controller may be configured to identify a stationary object within an image frame through the image data, and identify whether the host vehicle is moving forward or backward based on movement of the stationary object over time.

[0012] The controller may be configured to identify at least one of a lane marking and a sign within an image frame through the image data, and identify whether the host vehicle is moving forward or backward based on movement of at least one of the lane marking and the sign over time.

[0013] The controller may be configured to recognize that the host vehicle is moving forward in response to the stationary object moving in a direction from the center toward the edge of the image frame.

[0014] The controller may be configured to recognize that the host vehicle is moving backward in response to the stationary object moving in a direction from an edge toward the center of the image frame.

[0015] Another aspect of the present disclosure is to provide a method for assisting driving of a vehicle, the method comprising the following steps: processing image data obtained by a camera installed on the vehicle and having a field of view outside the vehicle and a field of view in front of the vehicle; identifying at least one object that hinders the travel of the vehicle based on the image data; predicting a collision with the at least one object; identifying whether the vehicle is moving forward or backward based on the image data; and controlling a braking device of the vehicle to brake the vehicle according to whether the vehicle is moving forward or backward.

[0016] Controlling a brake device of the host vehicle to brake the host vehicle may include controlling the brake device of the host vehicle to brake the host vehicle in response to forward movement of the host vehicle.

[0017] The driver assistance method may further include controlling at least one of a display, audio, and a steering wheel of the host vehicle to warn the host vehicle of a collision in response to the rearward movement of the host vehicle.

[0018] Identifying whether the host vehicle is moving forward or backward may include: identifying a stationary object within an image frame based on the image data; and identifying whether the host vehicle is moving forward or backward based on movement of the stationary object over time.

[0019] Identifying the stationary object within the image frame using the image data may include identifying at least one of a lane marking and a sign within the image frame using the image data.

[0020] Identifying whether the host vehicle is moving forward or backward based on the movement of the stationary object over time may include identifying that the host vehicle is moving forward in response to the stationary object moving in a direction from a center toward an edge of the image frame.

[0021] Identifying whether the host vehicle is moving forward or backward based on the movement of the stationary object over time may include identifying that the host vehicle is moving backward in response to the stationary object moving in a direction from an edge toward the center of the image frame.

[0022] Predicting a collision with the at least one object may include: obtaining detection data by a sensor in a group including a radar sensor and a lidar sensor, which is installed in the vehicle and has a detection field of view of at least one of the front and side of the vehicle; and predicting a collision with the at least one object based on the detection data.

[0023] Another aspect of the present disclosure is to provide a device for assisting driving of a vehicle, the device including: an image sensor mounted on the vehicle and having a field of view outside the vehicle, the image sensor being configured to obtain image data; a radar sensor mounted on the vehicle and having a detection field of view of at least one of the front and side of the vehicle, and being configured to obtain detection data; and a controller configured to process the image data and the detection data, identify at least one object that obstructs the travel of the vehicle based on the image data, predict a collision with the at least one object based on the detection data, identify whether the vehicle is moving forward or backward based on the image data, and control a braking device of the vehicle to brake the vehicle according to whether the vehicle is moving forward or backward. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] These and / or other aspects of the present disclosure will become more apparent and more readily understood from the following description of embodiments in conjunction with the accompanying drawings, in which:

[0025] Figure 1 is a block diagram illustrating a configuration of a vehicle according to an embodiment;

[0026] Figure 2 is a block diagram illustrating a configuration of a driver assistance device according to an embodiment;

[0027] Figure 3 is a diagram illustrating fields of view of a camera and a radar included in a driver assistance device according to an embodiment;

[0028] Figure 4 is a diagram illustrating a method of determining driving of a driver assistance device according to an exemplary embodiment.

[0029] Figure 5 is a diagram illustrating an image captured by a camera included in a driver assistance apparatus according to an embodiment.

[0030] Figure 6 is a diagram illustrating an emergency braking method of a driver assistance apparatus according to an embodiment.

[0031] Figure 7 is a diagram illustrating that a vehicle according to an embodiment travels forward.

[0032] Figure 8 It is an example of Figure 7 Schematic diagram of an image captured by a camera while the vehicle is traveling.

[0033] Figure 9 2 is a diagram illustrating that the vehicle travels backward according to the embodiment.

[0034] Figure 10 It is an example of Figure 9 Schematic diagram of an image captured by a camera while the vehicle is traveling. DETAILED DESCRIPTION

[0035] The following detailed description is provided to help the reader gain a comprehensive understanding of the methods, apparatuses, and / or systems described herein. Therefore, various variations, modifications, and equivalents of the methods, apparatuses, and / or systems described herein will be suggested to those skilled in the art. The described progression of processing operations is an example; however, the order of operations is not limited to that set forth herein and may be varied as known in the art, except for operations that must occur in a specific order. Additionally, descriptions of known functions and structures may be omitted for clarity and brevity.

[0036] Additionally, exemplary embodiments will now be described more fully hereinafter with reference to the accompanying drawings. However, exemplary embodiments may be implemented in many different forms and should not be construed as limited to the embodiments set forth herein. These embodiments are provided so that this disclosure will be thorough and complete and will fully convey the exemplary embodiments to those skilled in the art. Like reference numerals represent like elements throughout.

[0037] It should be understood that although the terms first, second, etc. may be used herein to describe various elements, these elements should not be limited by these terms. These terms are simply used to distinguish one element from another. As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items.

[0038] It should be understood that when an element is referred to as being “connected” or “coupled” to another element, it can be directly connected or coupled to the other element or intervening elements may exist. In contrast, when an element is referred to as being “directly connected” or “directly coupled” to another element, there are no intervening elements.

[0039] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting.As used herein, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.

[0040] Reference will now be made in detail to the exemplary embodiments of the present disclosure, examples of which are illustrated in the accompanying drawings, wherein like reference numerals refer to like elements throughout.

[0041] The expression "at least one of a, b, and c" should be understood to include only a, only b, only c, both a and b, both a and c, both b and c, or all of a, b, and c.

[0042] Hereinafter, principles and embodiments of the present disclosure will be described with reference to the accompanying drawings.

[0043] Figure 1 is a block diagram illustrating a configuration of a vehicle according to an embodiment.

[0044] like Figure 1 As shown in FIG, a vehicle 1 may include a driving system 10 , a braking system 30 , and a steering system 40 .

[0045] The driving system 10 moves the vehicle 1 and includes an engine management system (EMS) 11 , an engine 12 , a transmission control unit (TCU) 21 , and a transmission 22 .

[0046] The engine 12 may include cylinders and pistons, and generates power required for the vehicle 1 to travel. The transmission 22 may include a plurality of gears, and transmits the power generated by the engine 12 to wheels.

[0047] The EMS 11 may control the engine 12 in response to the driver's acceleration intention through the accelerator pedal or a request of the driver assistance device 100. For example, the EMS 11 may control the torque of the engine 12.

[0048] The TCU 21 may control the transmission 22 in response to a shift command from a driver via a shift lever and / or a driving speed of the vehicle 1. For example, the TCU 21 may adjust a gear ratio between the engine 12 and the wheels.

[0049] The brake system 30 stops the vehicle 1 and may include an electronic brake control module (EBCM) 31 and a brake device 32 .

[0050] The brake device 32 can decelerate the vehicle 1 or stop the vehicle 1 through friction with the wheels.

[0051] The EBCM 31 can control the braking device 32 in response to the driver's braking intention via the brake pedal and / or wheel slip. For example, the EBCM 31 can control the braking device 32 to temporarily release the brakes on the wheels in response to wheel slip sensed when braking the vehicle 1 (Anti-lock Braking System: ABS). The EBCM 31 can control the braking device 32 to selectively release the brakes on the wheels in response to oversteer and / or understeer sensed when turning the vehicle 1 (Electronic Stability Control: ESC). In addition, the EBCM 31 can control the braking device 32 to temporarily brake the wheels in response to wheel slip sensed when driving the vehicle 1 (Traction Control System: TCS).

[0052] In addition, the EBCM 31 may control the brake device 32 in response to a request from the driver assistance device 100. For example, the EBCM 31 may receive a deceleration request including a deceleration rate from the driver assistance device 100 and control the brake device 32 so that the vehicle 1 decelerates according to the requested deceleration rate.

[0053] The steering system 40 may include an electronic power steering control module (EPS) 41 and a steering device 42 .

[0054] The steering device 42 can change the driving direction of the vehicle 1 .

[0055] The EPS 41 can assist the operation of the steering device 42 in response to the driver's steering intention through the steering wheel, so that the driver can easily operate the steering wheel. For example, the EPS 41 can assist the operation of the steering device 42 so that the steering force is reduced during low-speed driving or parking and the steering force is increased during high-speed driving.

[0056] In addition, EPS 41 may control steering device 42 in response to a request from driver assistance device 100. For example, electronic steering control module 41 may receive a steering request including a steering torque from driver assistance device 100 and control steering device 42 to steer vehicle 1 according to the requested steering torque.

[0057] The vehicle 1 also includes a body control module (BCM) 51 and a driver assistance device 100 for driver safety and convenience.

[0058] The BCM 51 may control the operation of mechanical parts that provide convenience to the driver or ensure the driver's safety. For example, the BCM 51 may control headlights, wipers, an instrument panel, a multi-function switch, a direction indicator light, and the like.

[0059] The driver assistance device 100 can assist the driver in manipulating (driving, braking, and steering) the vehicle 1. For example, the driver assistance device 100 can sense the surrounding environment of the vehicle 1 (e.g., other vehicles, pedestrians, cyclists, lanes, road signs, etc.) and control the driving and / or braking and / or steering of the vehicle 1 in response to the sensed environment.

[0060] The driver assistance system 100 can provide various functions for the driver. For example, the DAS 100 can include lane departure warning (LDW), lane keeping assist (LKA), high beam assist (HBA), automatic emergency braking (AEB), traffic sign recognition (TSR), smart cruise control (SCC), blind spot detection (BSD), etc.

[0061] The driver assistance device 100 includes a camera module 101 for acquiring image data of the surroundings of the vehicle 1 and a radar module 102 for acquiring object data of the surroundings of the vehicle 1. The camera module 101 may include a camera 101a and an electronic control unit (ECU) 101b, and may capture at least one of the front and side views of the vehicle 1 and identify other vehicles, pedestrians, cyclists, lanes, road signs, etc. The radar module 102 may include a radar 102a and an ECU 102b, and may acquire the relative position, relative speed, etc. of objects (e.g., other vehicles, pedestrians, cyclists, etc.) surrounding the vehicle 1.

[0062] The driver assistance device 100 is not limited to Figure 1 and may also include a LiDAR that scans and detects objects around the vehicle 1 .

[0063] The aforementioned electronic components can communicate with each other via the vehicle communication network NT. For example, mechanical components can transmit data via Ethernet, Media Oriented Systems Transport (MOST), Flexray, Controller Area Network (CAN), Local Interconnect Network (LIN), etc. For example, the driver assistance device 100 can transmit driving signals, braking signals, and steering signals to the EMS 11, TCU 21, EBCM 31, and EPS 41, respectively.

[0064] Figure 2 is a block diagram illustrating a configuration of a driver assistance apparatus according to an embodiment. Figure 3 is a diagram illustrating fields of view of a camera and a radar included in a driver assistance apparatus according to an embodiment.

[0065] like Figure 2 As shown in FIG, a vehicle 1 may include a driving system 10 , a braking system 30 , a steering system 40 , and a driver assistance device 100 .

[0066] The driving system 10, the braking system 30 and the steering system 40 can be connected with Figure 1 The driving system 10, braking system 30 and steering system 40 shown in FIG. 1 are identical.

[0067] The driver assistance device 100 may include a front camera 110 , a front radar 120 , and a controller 140 .

[0068] The front camera 110 may have a field of view 110a facing the front of the vehicle 1, such as Figure 3 The front camera 110 may be installed on the front windshield of the vehicle 1 , for example.

[0069] The front camera 110 can capture the front of the vehicle 1 and obtain image data in front of the vehicle 1. The image data in front of the vehicle 1 may include position information of other vehicles, pedestrians, cyclists, or lanes in front of the vehicle 1.

[0070] The front camera 110 may include multiple lenses and an image sensor. The image sensor may include multiple photodiodes for converting light into electrical signals, and the multiple photodiodes may be arranged in a two-dimensional matrix.

[0071] The front camera 110 may be electrically connected to the controller 140. For example, the front camera 110 may be connected to the controller 140 via the vehicle communication network NT, may be connected to the controller 140 via hard wiring, or may be connected to the controller 140 via a printed circuit board (PCB).

[0072] The front camera 110 may transmit image data in front of the vehicle 1 to the controller 140 .

[0073] The front radar 120 may have a sensing field 120a facing the front of the vehicle 1, such as Figure 3 The front radar 120 may be mounted on, for example, a grille or a bumper of the vehicle 1 .

[0074] The front radar 120 may include a transmitting antenna (or a transmitting antenna array) that radiates transmitted radio waves toward the front of the vehicle 1 and a receiving antenna (or a receiving antenna array) that receives reflected radio waves reflected from an object. The front radar 120 can obtain front radar data from the transmitted radio waves sent by the transmitting antenna and the reflected radio waves received by the receiving antenna. The front radar data may include distance information and speed information about other vehicles, pedestrians, or cyclists in front of the vehicle 1. The front radar 120 can calculate the relative distance to the object based on the phase difference (or time difference) between the transmitted radio wave and the reflected radio wave, and calculate the relative speed of the object based on the frequency difference between the transmitted radio wave and the reflected radio wave.

[0075] The front radar 120 may be connected to the controller 140 via the vehicle communication network NT, hard wiring, or a printed circuit board. The front radar 120 may transmit front radar data to the controller 140.

[0076] The controller 140 may include a camera module ( Figure 1 ECU (101) Figure 1 101b) and / or radar module ( Figure 1 ECU (102) Figure 1 102b) and / or integrated ECU.

[0077] The controller 140 may be electrically connected to the front camera 110 and the front radar 120. In addition, the controller 140 may be connected to the driving system 10, the braking system 30, and the steering system 40 through the vehicle communication network NT.

[0078] The controller 140 includes a processor 141 and a memory 142 .

[0079] The processor 141 can process the front image data from the front camera 110 and the detection data from the front radar 120, and generate driving signals, braking signals, and steering signals for controlling the driving system 10, the braking system 30, and the steering system 40. For example, the processor 141 may include: an image signal processor for processing the front image data from the front camera 110; and / or a digital signal processor for processing radar data from the radars 120 and 130; and / or a microcontroller unit (MCU) for generating the driving signals, braking signals, and steering signals.

[0080] The processor 141 may sense objects in front of the vehicle 1 (eg, other vehicles, pedestrians, cyclists, etc.) based on the front image data of the front camera 110 and the detection data of the front radar 120 .

[0081] The processor 141 can obtain the position (direction) and classification (e.g., whether the object is another vehicle, a pedestrian, or a cyclist) of an object in front of the vehicle 1 based on the image data from the front camera 110. The processor 141 can also obtain the position (distance and direction) and relative speed of an object in front of the vehicle 1 based on the detection data from the front radar 120. In addition, the processor 141 can match the objects detected by the image data with the objects detected by the detection data, and obtain the classification, relative position, and relative speed of the object in front of the vehicle 1 based on the matching results.

[0082] The processor 141 can generate a driving signal, a braking signal, and a steering signal based on the relative position and relative speed of the object ahead. For example, the processor 141 can send a driving signal to the driving system 10 so that the distance to the vehicle ahead (or the time until reaching the position of the vehicle ahead) becomes the distance set by the driver. In addition, the processor 141 calculates the time to collision (TTC) or distance to collision (DTC) between the vehicle 1 and the object ahead based on the position (distance) and relative speed of the object ahead, and warns the driver of the collision based on the comparison result of the TTC with the reference value or sends a braking signal to the braking system 30. In addition, when it is determined that a collision with the object ahead is based on the time to collision or the distance to collision, the processor 141 can send a steering signal to the steering system 40 to avoid a collision with the object ahead.

[0083] The processor 141 may select a target with a collision risk among the front objects based on the image data of the front camera 110. For example, the processor 141 may select other vehicles traveling in the same lane as the vehicle 1 as the target.

[0084] The processor 141 can identify the movement of an object other than the target (hereinafter referred to as "another object") based on the image data of the front camera 110, and can identify whether the vehicle 1 is moving forward or backward based on the movement of the other object. For example, the processor 141 can identify whether the vehicle 1 is moving forward or backward based on the movement of a fixed object (e.g., a lane or a sign).

[0085] The processor 141 may output a braking signal and / or a steering signal for avoiding collision with a target based on the vehicle 1 moving forward. In addition, the processor 141 may output a driving signal for avoiding collision with a target based on the vehicle 1 moving backward.

[0086] The memory 142 stores programs and / or data for the processor 141 to process image data, programs and / or data for the processor 141 to process radar data, and programs and / or data for the processor 141 to generate driving signals and / or braking signals and / or turning signals.

[0087] The memory 142 may temporarily store image data received from the front camera 110 and / or radar data received from the radars 120 and 130 , and may temporarily store results of the processor 141 processing the image data and / or detection data.

[0088] The memory 142 may include a volatile memory such as S-RAM, D-RAM, etc. and a nonvolatile memory such as flash memory, read-only memory (ROM), erasable programmable read-only memory (EPROM), etc.

[0089] In this manner, the controller 140 may send a braking signal to the braking system 30 based on whether a collision with a forward object is predicted to occur. Additionally, the controller 140 may send a steering signal to the steering system 40 to avoid a collision with a forward object.

[0090] In addition, when a collision with a target is predicted, the controller 140 can identify whether the vehicle 1 is moving straight forward or backward based on the movement of another object. When the vehicle 1 is moving straight, the controller 140 can output a braking signal and / or a steering signal to avoid a collision with the target, and when the vehicle 1 is moving backward, the controller 140 may not output a braking signal and / or a steering signal.

[0091] The driver assistance device 100 is not limited to Figure 2 and may also include a LiDAR that scans and detects objects around the vehicle 1 .

[0092] Figure 4 is a diagram illustrating a method of determining driving of a driver assistance device according to an exemplary embodiment. Figure 5 is a diagram illustrating an image captured by a camera included in a driver assistance apparatus according to an embodiment.

[0093] Will refer to Figure 4 and Figure 5 A method 1000 for determining the operation of a driver assistance device will be described.

[0094] The driver assistance device 100 recognizes and classifies objects around the vehicle 1 ( 1010 ).

[0095] When the vehicle 1 is traveling or stopped, the front camera 110 of the driver assistance device 100 can acquire image data in front of and / or around the vehicle 1. The controller 140 of the driver assistance device 100 can acquire image data from the front camera 110. The controller 140 can identify and classify objects located in front of and / or around the vehicle 1 based on the image data. For example, the controller 140 can identify objects located in front of and / or around the vehicle 1, including lane markings, signs, other vehicles, pedestrians, cyclists, road boundaries, animals, and traffic lights. In addition, the controller 140 can classify the identified objects into lane markings, signs, other vehicles, pedestrians, bicycles, road boundaries, animals, traffic lights, etc.

[0096] When the vehicle 1 is traveling or stopped, the front radar 120 of the driver assistance device 100 may acquire detection data in front of and / or around the vehicle 1. The controller 140 may acquire the detection data from the front radar 120. The controller 140 may identify objects in front of and / or around the vehicle 1 based on the detection data.

[0097] The controller 140 may identify objects located in front of and / or around the vehicle 1 based only on image data from the front camera 110 , or may identify objects located in front of and / or around the vehicle 1 based only on detection data from the front radar 120 .

[0098] In addition, the controller 140 can identify objects located in front of and / or around the vehicle 1 based on the image data of the front camera 110 and the detection data of the front radar 120. For example, the controller 140 can identify common objects between the objects identified based on the image data of the front camera 110 and the objects identified based on the detection data of the front radar 120.

[0099] The driver assistance device 100 may recognize relative positions and relative speeds of surrounding objects of the vehicle 1 ( 1020 ).

[0100] The controller 140 can identify the relative position of an object located in front of and / or around the vehicle 1 based on the image data, and can identify the relative speed of an object located in front of and / or around the vehicle 1 based on a plurality of continuous image data. For example, the controller 140 can identify the relative position of an object based on the object position in the image (the coordinates of the pixels occupied by the object in the image frame) and the object size (the number of pixels occupied by the object in the image frame) through the image data. In addition, the controller 140 can identify the lateral relative speed of an object based on changes in the object position in the image (the coordinates of the pixels occupied by the object in the image frame) through the plurality of continuous image data, and can identify the longitudinal relative speed of an object based on changes in the object size (the number of pixels occupied by the object in the image frame).

[0101] In addition, the controller 140 can identify the relative position and relative speed of objects located in front of and / or around the vehicle 1 based on the detection data. For example, the controller 140 can identify the relative position of objects located in front of and / or around the vehicle 1 based on the time until the radio waves reflected from the objects are received and the angle at which the radio waves are received. In addition, the controller 140 can identify the relative speed of objects located in front of and / or around the vehicle 1 based on the frequency change (Doppler effect) of the radio waves reflected from the objects.

[0102] In addition, based on the image data from the front camera 110 and the detection data from the front radar 120, the controller 140 can identify the relative position and relative velocity of objects located in front of and / or around the vehicle 1. For example, the controller 140 can determine the lateral relative position and lateral relative velocity of the object based on the image data from the front camera 110, and can determine the longitudinal relative position and longitudinal relative velocity of the object based on the detection data from the front radar 120. Here, the lateral direction refers to a direction perpendicular to the direction of travel of the vehicle 1, and the longitudinal direction may refer to a direction parallel to the direction of travel of the vehicle 1.

[0103] The driver assistance apparatus 100 recognizes whether the vehicle 1 is moving forward or backward based on the movement of the stationary object ( 1030 ).

[0104] Typically, the driver assistance device 100 can receive driving state information including forward and reverse movement from the driving system 20. For example, the driver assistance device 100 can receive information about the position of a shift lever or information about the operation of the transmission 22 (hereinafter referred to as "shift information") from the transmission control unit 21. However, due to a malfunction of the transmission control unit 21, the driver assistance device 100 may receive incorrect shift information from the transmission control unit 21.

[0105] To prevent the driver assistance device 100 from performing an erroneous operation due to receiving incorrect shift information from the transmission control unit 21 , the controller 140 may determine a shift state (eg, forward or backward movement of the vehicle) based on image data acquired by the front camera 110 .

[0106] To determine the shift state of the vehicle 1 (e.g., forward or backward movement of the vehicle), the controller 140 may identify objects with fixed positions, such as lane markings or signs on the road. For example, the controller 140 may detect lane markings having a dotted line shape from the image data using an edge detection algorithm, etc., and may detect symbols from the image data using machine learning techniques.

[0107] The controller 140 may identify whether the vehicle 1 is moving forward or backward based on the movement of the stationary object within the image frame.

[0108] For example, the controller 140 can receive information such as Figure 5 The image data of the image frame 200 shown in . The image frame 200 may include a preceding vehicle image 2a, a lane marking image 3a, and a sign image 4a.

[0109] The controller 140 may identify the front vehicle image 2a, the lane marking image 3a, and the sign image 4a as objects in the image frame 200, and may identify relative positions and relative speeds of the front vehicle image 2a, the lane marking image 3a, and the sign image 4a, respectively.

[0110] Based on the image frame 200, the controller 140 can determine that the preceding vehicle is a moving object and that the lane marking and the display panel are stationary objects. In addition, the controller 140 can identify the positions (or pixel coordinates) of the lane marking image 3a and the sign image 4a in the frame. Figure 5 The positions (or pixel coordinates) of the lane marking image and the sign image are identified in the temporally next image frame of the illustrated image frame 200. Furthermore, the controller 140 may identify changes in the positions (or pixel coordinates) of the lane marking image 3a and the sign image 4a in the frame based on a comparison result between the image frame 200 and the next image frame.

[0111] The controller 140 may identify whether the vehicle 1 is moving forward or backward based on changes in positions (or pixel coordinates) within the frames of the lane marking image 3 a and the sign image 4 a .

[0112] For example, when the lane marking image 3a and the sign image 4a move in the first direction D1 from the center of the image frame 200 toward the edge of the image frame 200, the controller 140 can recognize that the vehicle 1 is moving forward. Figure 5When the sign image 4a in the right half of the image frame 200 in FIG. 1 further moves to the right, the controller 140 can determine that the sign is moving away from the vehicle 1 and can also recognize that the vehicle 1 is moving forward. Figure 5 When the lane marking image 3a at the lower left of the image frame 200 moves further to the lower left, the controller 140 can determine that the lane marking is moving away from the vehicle 1 and can also recognize that the vehicle 1 is moving forward.

[0113] In addition, when the size of the sign image 4 a (the number of pixels occupied by the sign image) increases, the controller 140 can recognize that the vehicle 1 is moving forward.

[0114] As another example, when the lane marking image 3a and the sign image 4a move in the second direction D2 from the edge of the image frame 200 toward the center of the image frame 200, the controller 140 may recognize that the vehicle 1 is moving backward. In other words, when the lane marking image 3a and the sign image 4a move in the second direction D2 from the edge of the image frame 200 toward the center of the image frame 200, the controller 140 may recognize that the vehicle 1 is moving backward. Figure 5 When the sign image 4a in the right half of the image frame 200 in FIG. 1 moves to the left, the controller 140 can determine that the sign is close to the vehicle 1 and can also recognize that the vehicle 1 is moving backward. Figure 5 When the lane marking image 3a at the lower left of the image frame 200 moves to the lower right, the controller 140 can determine that the lane marking is close to the vehicle 1 and can also recognize that the vehicle 1 is moving backward.

[0115] In addition, when the size of the sign image 4 a (the number of pixels occupied by the sign image) decreases, the controller 140 can recognize that the vehicle 1 is moving backward.

[0116] As described above, the driver assistance apparatus 100 can recognize whether the vehicle 1 is moving forward or backward based on the image data acquired by the front camera 110 .

[0117] Thus, the driver assistance apparatus 100 can recognize whether the vehicle 1 is moving forward or backward despite a malfunction of the transmission control unit 21 .

[0118] Figure 6 is a diagram illustrating an emergency braking method of a driver assistance apparatus according to an embodiment. Figure 7 is a diagram illustrating that a vehicle according to an embodiment travels forward. Figure 8 It is an example of Figure 7 Schematic diagram of an image captured by a camera while the vehicle is traveling. Figure 9 2 is a diagram illustrating that the vehicle travels backward according to the embodiment. Figure 10 It is an example of Figure 9 Schematic diagram of an image captured by a camera while the vehicle is traveling.

[0119] Together Figure 6 、 Figure 7 、 Figure 8 、 Figure 9 and Figure 10 Together, an emergency braking method 1100 of a driver assistance device will be described.

[0120] The driver assistance device 100 recognizes and classifies objects around the vehicle 1 ( 1110 ). The driver assistance device 100 recognizes relative positions and relative speeds of objects around the vehicle 1 ( 1120 ).

[0121] Operations 1110 and 1120 can be performed with Figure 4 Operations 1010 and 1020 shown in FIG are the same.

[0122] The driver assistance apparatus 100 selects a target among the objects ( 1130 ).

[0123] The controller 140 may select an object that has a collision risk with the vehicle 1 based on the image data acquired by the front camera 110 .

[0124] For example, the controller 140 can identify lane markings and other vehicles based on the image data of the front camera 110. The controller 140 can identify a preceding vehicle traveling in the same lane as the vehicle 1 among other vehicles based on the lane markings and the positions of the other vehicles. The controller 140 can identify the lane in which the vehicle 1 is traveling based on the positions of the lane markings, and can identify a preceding vehicle traveling in the same lane as the vehicle 1 based on the positions of the other vehicles. In addition, the controller 140 can select the preceding vehicle as a target.

[0125] As another example, the controller 140 may determine an object (eg, a preceding vehicle, a pedestrian, a bicycle, or an animal) that is expected to collide with the vehicle 1 based on the relative position and relative velocity of the object. The controller 140 may select the object that is expected to collide with the vehicle 1 as a target.

[0126] The driver assistance apparatus 100 determines whether the time until the collision with the target is less than a reference time ( 1140 ).

[0127] The controller 140 may predict the time to collision (TTC) between the vehicle 1 and the target based on the relative position and relative velocity of the target. For example, the controller 140 may calculate the remaining time to collision (TTC) between the vehicle 1 and the preceding vehicle based on the distance to the preceding vehicle and the relative velocity of the preceding vehicle. The controller 140 may also compare the remaining time to collision (TTC) with the target with a reference time and determine whether the remaining time to collision (TTC) with the target is less than the reference time.

[0128] However, operation 1140 may be replaced with an operation that depends on distance.

[0129] For example, the controller 140 may predict the distance to collision (DTC) between the vehicle 1 and the target based on the relative position and relative velocity of the target. The controller 140 may calculate the distance to collision (DTC) with the target based on the product of the vehicle 1's travel speed and the remaining time to collision (TTC) with the target. The controller 140 may also compare the remaining distance to collision (DTC) with the target with a reference distance and determine whether the remaining distance to collision (DTC) with the target is less than the reference distance.

[0130] When the collision time with the target is not less than the reference time ("No" in 1140), the driver assistance device 100 may re-identify the surrounding objects and re-identify the relative positions and relative speeds of the objects. In addition, the driver assistance device 100 may reselect the target and determine whether the collision time with the target is less than the reference time.

[0131] When the collision time with the target is less than the reference time (YES in 1140 ), the driver assistance apparatus 100 determines whether the vehicle 1 is moving forward or backward ( 1150 ).

[0132] As mentioned above Figure 4 As described above, the controller 140 can determine whether the vehicle 1 is moving forward or backward based on the image data of the front camera 110. For example, the controller 140 can determine whether the vehicle 1 is moving forward or backward based on the movement of the image representing the stationary object within the image frame captured by the front camera 110. Specifically, when the image of the stationary object in the image frame moves from the center toward the edge of the image frame, the controller 140 can determine that the vehicle 1 is moving forward. In addition, when the image of the stationary object in the image frame moves from the edge toward the center of the image frame, the controller 140 can determine that the vehicle 1 is moving backward.

[0133] When it is determined that the vehicle 1 is moving forward (“moving forward” in 1150 ), the driver assistance apparatus 100 performs emergency braking ( 1160 ).

[0134] When a collision between the vehicle 1 and a target is predicted and the vehicle 1 moves forward, it can be determined that the vehicle 1 moves toward the target (eg, a preceding vehicle).

[0135] For example, Figure 7 As shown in , a preceding vehicle 2 may be located in front of the vehicle 1. In addition, a lane marker 3 may be located on the left side of the vehicle 1, and a sign 4 may be located on the right front side of the vehicle 1.

[0136] In this case, the front camera 110 can capture Figure 8The image frame 210 is shown in FIG. The image frame 210 may include a preceding vehicle image 2 a representing the preceding vehicle 2 , a lane marking image 3 a representing the lane marking 3 , and a sign image 4 a representing the sign 4 .

[0137] When the front vehicle 2 stops and the vehicle 1 moves forward, the front vehicle image 2a increases in the image frame 210 captured by the front camera 110, and the remaining time to collision (TTC) between the vehicle 1 and the front vehicle 2 can be reduced. The controller 140 can predict the collision between the vehicle 1 and the front vehicle 2. In addition, as Figure 8 As shown in , lane marking image 3 a and / or sign image 4 a in image frame 210 may move from the center toward the edge of image frame 210 .

[0138] The controller 140 may determine that the vehicle 1 is traveling forward based on the movement of the lane marking image 3a and / or the sign image 4a. In addition, the controller 140 may determine that a collision is expected between the vehicle 1 and the preceding vehicle 2 because the vehicle 1 is moving toward the preceding vehicle 2.

[0139] Therefore, the controller 140 may brake the vehicle 1 in order to avoid a collision between the vehicle 1 and the preceding vehicle 2. For example, based on the remaining time to collision (TTC) between the vehicle 1 and the preceding vehicle 2, the controller 140 prepares for braking, then brakes the vehicle 1 with a first braking force, and then may send a braking signal to the braking system 30 to brake the vehicle 1 with a second braking force that is greater than the first braking force.

[0140] When determining that the vehicle 1 is moving backward (“moving backward” in 1150 ), the driver assistance apparatus 100 warns of a collision ( 1170 ).

[0141] When a collision between the vehicle 1 and an object is predicted and the vehicle 1 moves backward, it can be determined that the object (eg, a preceding vehicle) is moving toward the vehicle 1 .

[0142] For example, Figure 9 As shown in , the vehicle 1 can travel backward, and the preceding vehicle 2 can also travel backward in front of the vehicle 1. In addition, the lane marking 3 can be located on the left side of the vehicle 1, and the sign 4 can be located on the right front side of the vehicle (1).

[0143] In this case, the front camera 110 can capture Figure 10 The image frame 220 is shown in . The image frame 220 may include a preceding vehicle image 2 a representing the preceding vehicle 2 , a lane marking image 3 a representing the lane marking 3 , and a sign image 4 a representing the sign 4 .

[0144] When the vehicle 1 is traveling backward and the preceding vehicle 2 is traveling backward at a faster speed, the preceding vehicle image 2a is enlarged in the image frame 210 captured by the front camera 110, and the remaining time to collision (TTC) between the vehicle 1 and the preceding vehicle 2 may be reduced. The controller 140 may predict a collision between the vehicle 1 and the preceding vehicle 2. In addition, as Figure 10 As illustrated in FIG, the lane marking image 3 a and / or the sign image 4 a in the image frame 220 may move from the edge toward the center of the image frame 220 .

[0145] The controller 140 may determine that the vehicle 1 is traveling backward based on the movement of the lane marking image 3 a and / or the sign image 4 a. In addition, the controller 140 may determine that a collision is expected between the vehicle 1 and the preceding vehicle 2 because the preceding vehicle 2 is moving backward toward the vehicle 1 .

[0146] If it is determined that a collision between the vehicle 1 and the preceding vehicle 2 is expected and the vehicle 1 is therefore braked, a collision between the vehicle 1 and the preceding vehicle 2 may result.

[0147] Therefore, the controller 140 can provide a warning message to the driver to avoid a collision between the vehicle 1 and the preceding vehicle 2. For example, based on the remaining time to collision (TTC) between the vehicle 1 and the preceding vehicle 2, the controller 140 can display a graphic warning message on a display (e.g., instrument panel, AVN, etc.) of the vehicle 1, or can output an acoustic warning message through an audio device (e.g., AVN) of the vehicle 1 or a tactile warning message through the steering wheel of the vehicle 1.

[0148] As described above, the driver assistance device 100 can predict a collision with a target (e.g., a preceding vehicle) and can identify whether the vehicle 1 is moving forward or backward based on image data acquired by the front camera 110. In addition, the driver assistance device 100 can urgently brake the vehicle 1 when the vehicle 1 is moving forward and warn of a collision of the vehicle 1 when the vehicle 1 is moving backward.

[0149] Thus, due to the emergency braking of the vehicle 1 when the vehicle 1 moves backward, a collision with the preceding vehicle 2 can be prevented.

[0150] An aspect of the present disclosure is to provide a driver assistance apparatus and a driver assistance method capable of recognizing forward movement or backward movement of a vehicle based on image data.

[0151] The exemplary embodiments of the present disclosure have been described above. In the exemplary embodiments described above, some components may be implemented as "modules." Here, the term "module" refers to, but is not limited to, software and / or hardware components such as field programmable gate arrays (FPGAs) or application-specific integrated circuits (ASICs) that perform certain tasks. Modules may advantageously be configured to reside on an addressable storage medium and to execute on one or more processors.

[0152] Thus, for example, modules can include components such as software components, object-oriented software components, class components and task components, processes, functions, attributes, procedures, subroutines, program code segments, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, and variables. The operations provided in the components and modules can be combined into fewer components and modules, or further divided into additional components and modules. In addition, the components and modules can be implemented so that they execute one or more CPUs in the device.

[0153] That being said, and in addition to the exemplary embodiments described above, embodiments may be implemented by computer-readable code / instructions in / on a medium (e.g., a computer-readable medium) to control at least one processing element to implement any of the exemplary embodiments described above. The medium may correspond to any medium / media that allows storage and / or transmission of computer-readable code.

[0154] The computer readable code may be recorded on a medium or transmitted over the Internet. The medium may include a read-only memory (ROM), a random access memory (RAM), a compact disc read-only memory (CD-ROM), a magnetic tape, a floppy disk, and an optical recording medium. In addition, the medium may be a non-transitory computer readable medium. The medium may also be a distributed network so that the computer readable code is stored or transmitted and executed in a distributed manner. Furthermore, by way of example only, the processing element may include at least one processor or at least one computer processor, and the processing element may be distributed and / or included in a single device.

[0155] While exemplary embodiments have been described with respect to a limited number of embodiments, those skilled in the art, having benefit of this disclosure, will appreciate that other embodiments can be devised which do not depart from the scope disclosed herein. Therefore, the scope should be limited only by the appended claims.

[0156] CROSS-REFERENCE TO RELATED APPLICATIONS

[0157] This application is based on and claims the benefit of priority from Korean Patent Application No. 2019-0172319 filed in the Korean Intellectual Property Office on December 20, 2019, the disclosure of which is incorporated herein by reference.

Claims

1. A device for assisting driving of a vehicle, the device comprising: a camera mounted on the host vehicle and having a field of view outside the host vehicle, the camera being configured to obtain image data; as well as A controller configured to: processing the image data, identifying at least one object that obstructs travel of the vehicle based on the image data, the at least one object including a preceding vehicle; predicting a collision with the preceding vehicle, identifying whether the host vehicle is moving forward or backward based on the image data, and controlling a brake device of the host vehicle to brake the host vehicle according to whether the host vehicle is moving forward or backward, Wherein, the controller is configured as follows: When a collision with the preceding vehicle is predicted and the host vehicle is moving forward, a brake signal is output to the brake device to brake the host vehicle, and When a collision with the preceding vehicle is predicted and the host vehicle is moving backward, it is determined that the preceding vehicle is moving toward the host vehicle and then at least one of a display, audio, and steering wheel of the host vehicle is controlled to warn of the collision without outputting the brake signal.

2. The device according to claim 1, wherein The controller is configured to identify a stationary object within an image frame using the image data acquired by the front camera, and to identify whether the host vehicle is moving forward or backward based on movement of the stationary object over time.

3. The device according to claim 2, wherein The controller is configured to identify at least one of a lane marker and a sign within an image frame using the image data acquired by the front camera, and to identify whether the vehicle is moving forward or backward based on movement of at least one of the lane marker and the sign over time.

4. The device according to claim 2, wherein The controller is configured to recognize that the host vehicle is moving forward in response to the stationary object moving in a direction from a center toward an edge of the image frame.

5. The apparatus according to claim 2, wherein The controller is configured to recognize that the host vehicle is moving backward in response to the stationary object moving in a direction from an edge toward a center of the image frame.

6. A method for assisting driving of a vehicle, the method comprising the following steps: processing image data obtained by a camera mounted on the host vehicle and having a field of view external to the host vehicle; identifying at least one object obstructing travel of the host vehicle based on the image data, the at least one object including a preceding vehicle; predicting a collision with the preceding vehicle; identifying whether the vehicle is moving forward or backward based on the image data; as well as controlling a brake device of the host vehicle to brake the host vehicle according to whether the host vehicle is moving forward or backward, wherein, when a collision with the preceding vehicle is predicted and the host vehicle is moving forward, the step of controlling the braking device to brake the host vehicle is performed by outputting a braking signal to the braking device, and The method further comprises the following steps: When a collision with the preceding vehicle is predicted and the host vehicle is moving backward, it is determined that the preceding vehicle is moving toward the host vehicle and then at least one of a display, audio, and steering wheel of the host vehicle is controlled to warn of the collision without outputting the brake signal.

7. The method according to claim 6, wherein: The step of identifying whether the vehicle is moving forward or backward includes the following steps: identifying a stationary object within an image frame based on the image data; and Whether the host vehicle is moving forward or backward is identified based on the movement of the stationary object over time.

8. The method according to claim 7, wherein: The step of identifying a stationary object in an image frame using the image data comprises: At least one of a lane marking and a sign is identified within an image frame using the image data.

9. The method according to claim 7, wherein: The step of identifying whether the vehicle is moving forward or backward based on the movement of the stationary object over time includes: In response to the stationary object moving in a direction from the center toward the edge of the image frame, it is recognized that the host vehicle is moving forward.

10. The method according to claim 7, wherein: The step of identifying whether the vehicle is moving forward or backward based on the movement of the stationary object over time includes: It is recognized that the host vehicle is moving backward in response to the stationary object moving in a direction from an edge toward a center of the image frame.

11. A device for assisting driving of a vehicle, the device comprising: an image sensor mounted to the host vehicle and having a field of view outside the host vehicle, the image sensor being configured to obtain image data; a radar sensor mounted on the vehicle and having a detection field of view at least one of a front and a side of the vehicle, the radar sensor being configured to obtain detection data; as well as A controller configured to: processing the image data and the detection data, identifying at least one object that obstructs travel of the vehicle based on the image data, the at least one object including a preceding vehicle; predicting a collision with the preceding vehicle based on the detection data, identifying whether the host vehicle is moving forward or backward based on the image data, and controlling a brake device of the host vehicle to brake the host vehicle according to whether the host vehicle is moving forward or backward, Wherein, the controller is configured as follows: When a collision with the preceding vehicle is predicted and the host vehicle is moving forward, a brake signal is output to the brake device to brake the host vehicle, and When a collision with the preceding vehicle is predicted and the host vehicle is moving backward, it is determined that the preceding vehicle is moving toward the host vehicle and then at least one of a display, audio, and steering wheel of the host vehicle is controlled to warn of the collision without outputting the brake signal.

12. The apparatus according to claim 11, wherein The controller is configured to identify a stationary object within an image frame through the image data, and to identify whether the host vehicle is moving forward or backward based on movement of the stationary object over time.

13. The apparatus according to claim 12, wherein The controller is configured to identify at least one of a lane marker and a sign within an image frame through the image data, and to identify whether the host vehicle is moving forward or backward based on movement of at least one of the lane marker and the sign over time.

14. The apparatus according to claim 12, wherein The controller is configured to recognize that the host vehicle is moving forward in response to the stationary object moving in a direction from a center toward an edge of the image frame.

15. The apparatus according to claim 12, wherein The controller is configured to recognize that the host vehicle is moving backward in response to the stationary object moving in a direction from an edge toward a center of the image frame.

Images