Driver assistance system and control method thereof

Abstract

The present invention provides a driver assistance system and a control method thereof, comprising: a camera, which is arranged on the above-mentioned vehicle to have an external field of view of the vehicle and obtain image data; a radar, which is arranged on the vehicle to have an external sensing field of view of the vehicle and obtain radar data; and a control unit, which includes at least one processor, which processes the image data obtained by the camera and the radar data obtained by the radar, the control unit sends a steering signal to the vehicle based on at least one of the above-mentioned image data or radar data to achieve lane following assistance, senses the driver's operation of the steering device based on the steering angle of the steering device and the above-mentioned steering signal, and sends a warning signal to the vehicle in a manner of outputting a warning in stages according to the time when the operation is not sensed, and sends a warning signal to the vehicle in a manner of outputting a higher-level stage warning when a situation is determined to be unsuitable for assistance based on at least one of the image data or radar data.

Description

Technical Field

[0001] The disclosed invention relates to a driver assistance system. Background Art

[0002] In recent years, in order to reduce the burden on drivers and improve convenience, there has been active research on vehicles equipped with advanced driver assistance systems (ADAS) that can actively provide information about the vehicle status, driver status, and surrounding environment.

[0003] As an example of a front-end driver assistance system installed in a vehicle, there is a lane following assist system (Lane Following Assist; LFA). The lane following assist system provides convenience to the driver by controlling the driving path within the planned lane and tracking the target path through active control of the electronic power steering (Electronic Power Steering, EPS).

[0004] Specifically, a lane following assist system controls the vehicle to maintain its lane or provides warnings while the vehicle is traveling. However, if the lane following assist system is not suitable for operation and the driver is in a hands-off state, the possibility of an accident increases. Summary of the Invention

[0005] Problems to be solved by the invention

[0006] The present invention provides a driver assistance system and a control method thereof: during lane following assistance, warnings are output in stages according to the time when the driver's steering operation is not sensed, and when a situation is determined to be unsuitable for assistance, a higher-level warning is immediately output.

[0007] Means for solving problems

[0008] A driver assistance system according to one embodiment includes: a camera, which is provided on the vehicle to have an external field of view of the vehicle and obtain image data; a radar, which is provided on the vehicle to have an external sensing field of view of the vehicle and obtain radar data; and a control unit, which includes at least one processor, which processes the image data obtained by the camera and the radar data obtained by the radar, the control unit sending a steering signal to the vehicle based on at least one of the image data or the radar data to achieve lane following assistance, and sensing the driver's operation of the steering device based on the steering angle of the steering device and the steering signal, so as to send a warning signal to the vehicle in a manner that outputs a warning in stages according to the time when the operation is not sensed, and sending a warning signal to the vehicle in a manner that outputs a higher-level warning when a situation is determined to be unsuitable for assistance based on at least one of the image data or the radar data.

[0009] The control unit sends a first warning signal to the vehicle in such a manner that if no operation occurs within a first time, the vehicle is determined to be in a hands-off state and a lower-level warning is output; and sends a second warning signal to the vehicle in such a manner that if no operation occurs within a second time after the lower-level warning is output, an upper-level warning is output.

[0010] The lower-level warning is a visual warning, and the upper-level warning is an auditory warning or at least one of a visual and an auditory warning.

[0011] If the situation is determined to be unsuitable for assistance before the first time has passed or before the second time has passed after the output of the lower-level warning, the control unit transmits a warning signal to the vehicle to immediately output a higher-level warning.

[0012] The control unit determines that the situation is not suitable for the assistance when at least one of a situation that deviates from a control condition for implementing the lane following assistance or a collision risk situation occurs.

[0013] When the lane disappears while the lane is recognized and the lane following assist is being executed, the control unit determines that the situation is out of a control condition for implementing the lane following assist.

[0014] The control unit determines that the collision risk situation is the collision risk situation when there is a risk of collision with the surrounding objects based on the relative positions of the surrounding objects.

[0015] The control unit determines that the lane following assist is not suitable when the leading vehicle changes its path while the lane following assist is being performed while the leading vehicle is being followed.

[0016] The control unit determines that the situation is not suitable for assistance when a branch point where the lane interval widens is reached during lane recognition and lane following assistance is performed.

[0017] The control unit determines that the assist is not suitable when the vehicle reaches a merging point where the lane interval is narrowed while the vehicle recognizes a lane and performs the lane following assist.

[0018] A method for controlling a driver assistance system, the driver assistance system comprising: a camera, which is arranged on the vehicle to have an external field of view of the vehicle and obtain image data; and a radar, which is arranged on the vehicle to have an external sensing field of view of the vehicle and obtain radar data. The method for controlling the driver assistance system comprises: sending a steering signal to the vehicle based on at least one of the image data or the radar data to achieve lane following assistance; sensing the driver's operation of the steering device based on the steering angle of the steering device and the steering signal; sending a warning signal to the vehicle in a manner that outputs a warning in stages according to the time when the operation is not sensed; sending a warning signal to the vehicle in a manner that outputs a higher-level stage warning when a situation is determined to be unsuitable for assistance based on at least one of the image data or the radar data.

[0019] The method of sending a warning signal to the vehicle in such a manner that a warning is output in stages according to the time when the above-mentioned operation is not sensed includes: sending a first warning signal to the vehicle in such a manner that a lower-level stage warning is output when no operation occurs within a first time, and determining that the vehicle is in a hands-free state; and sending a second warning signal to the vehicle in such a manner that an upper-level stage warning is output when no operation occurs within a second time after outputting the lower-level stage warning.

[0020] The lower-level warning is a visual warning, and the upper-level warning is an auditory warning or at least one of a visual and an auditory warning.

[0021] Sending a warning signal to the above-mentioned vehicle to output the above-mentioned upper-level stage warning includes: sending a warning signal to the above-mentioned vehicle to immediately output the upper-level stage warning when it is determined that the situation is not suitable for assistance before the above-mentioned first time or before the second time after the above-mentioned lower-level stage warning is output.

[0022] The situation determined as unsuitable for the assistance includes: when at least one of a situation deviating from a control condition for implementing the lane following assistance or a collision risk situation occurs, the situation is determined as unsuitable for the assistance.

[0023] The situation determined as unsuitable for the assistance includes a situation where the lane disappears during lane recognition and lane following assistance is performed, and a situation determined as deviating from a control condition for implementing the lane following assistance.

[0024] The situation determined as unsuitable for assistance includes a situation determined as the collision risk situation when there is a risk of collision with the surrounding objects based on the relative position between the vehicle and the surrounding objects.

[0025] The situation determined as not suitable for the assistance includes a situation determined as not suitable for the assistance when the leading vehicle changes its path while the lane following assistance is being executed while following the leading vehicle.

[0026] The situation determined as unsuitable for the assistance includes a situation determined as unsuitable for the assistance when a branch point where the lane interval widens is reached during lane recognition and lane following assistance is performed.

[0027] The situation determined as unsuitable for assistance includes a situation determined as unsuitable for assistance when a merging point where the lane interval narrows is reached during lane recognition and lane following assistance is executed.

[0028] Effects of the Invention

[0029] According to one aspect of the disclosed invention, during the execution of lane following assistance, warnings are output in stages according to the time when the driver's operation of the steering device is not sensed. When the situation is determined to be unsuitable for assistance, a higher-level warning is immediately output, thereby reducing the possibility of an accident. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 The structure of a vehicle according to one embodiment is shown.

[0031] Figure 2 The structure of a driver assistance system according to one embodiment is shown.

[0032] Figure 3 Represents a camera and radar included in a driver assistance system of one embodiment.

[0033] Figure 4 This shows a case where the driver assistance system according to one embodiment recognizes a lane and performs lane following assistance.

[0034] Figure 5 FIG. 1 shows a case where a driver assistance system according to an embodiment tracks a preceding vehicle and performs lane following assistance.

[0035] Figure 6This shows a situation in which a driver assistance system according to an embodiment senses a hands-off state and issues warnings in stages.

[0036] Figure 7 This shows a situation where the driver assistance system of one embodiment senses a situation that is not suitable for assistance and issues a warning.

[0037] Figure 8 This figure shows a situation in which a driver assistance system according to an embodiment senses a hands-off state and issues warnings in stages.

[0038] Figure 9 The present invention shows a case where the driver assistance system according to one embodiment determines that the situation deviates from the control conditions for implementing lane following assist due to lane disappearance.

[0039] Figure 10 The following shows a case where a driver identification system according to one embodiment determines that a collision risk situation exists.

[0040] Figure 11 The following describes a case where the driver recognition system according to one embodiment determines that a situation is not suitable for assistance due to a change in the path of a preceding vehicle.

[0041] Figure 12 This shows a case where the driver identification system according to one embodiment determines that the situation is not suitable for assistance due to reaching a branch point.

[0042] Figure 13 The following describes a case where the driver identification system according to one embodiment determines that the driver has reached a merging point and the situation is not suitable for assistance.

[0043] Figure 14 This is a sequence diagram showing a method for outputting a warning in lane following assistance by a driver assistance system according to one embodiment.

[0044] (Explanation of Symbols)

[0045] 1: Vehicle 100: Driver Assistance System

[0046] 110: Camera 120: Radar

[0047] 130: Control Department DETAILED DESCRIPTION

[0048] Throughout this specification, identical symbols represent identical components. This specification does not describe all the components of the embodiments, and any general information related to the technical field of the disclosed invention or any overlapping information between the embodiments is omitted. Terms such as "unit, module, component, and block" used in this specification may refer to software or hardware. Depending on the embodiment, multiple "units, modules, components, and blocks" may be included as one component, or one "unit, module, component, and block" may include multiple components.

[0049] Throughout this specification, when a certain component is described as being “connected” to another component, this includes not only direct connections but also indirect connections. Indirect connections include connections via a wireless communication network.

[0050] Furthermore, when a certain part is described as “including” a certain constituent element, unless otherwise stated, this does not exclude other constituent elements but rather indicates that other constituent elements may be included.

[0051] Throughout the specification, when a component is described as being “on” another component, this includes not only a case where the component is in contact with the other component but also a case where the other component exists between the two components.

[0052] Terms such as first and second are used to distinguish one constituent element from other constituent elements, and the above terms do not limit the constituent elements.

[0053] Unless otherwise specified in the text, expressions in the singular include the plural.

[0054] In each stage, serial numbers are used for ease of explanation and do not indicate the order of the stages. Unless a specific order is explicitly stated in the text, the stages may be implemented in an order different from the stated order.

[0055] Hereinafter, the working principle and embodiments of the disclosed invention will be described with reference to the accompanying drawings.

[0056] Figure 1 The structure of a vehicle according to one embodiment is shown.

[0057] like Figure 1 As shown, vehicle 1 includes an engine 10, a transmission 20, a brake system 30, and a steering system 40. Engine 10, which includes cylinders and pistons, generates power for vehicle 1 to travel. Transmission 20, which includes multiple gears, transmits the power generated by engine 10 to the wheels. Braking system 30 decelerates or stops vehicle 1 through friction with the wheels. Steering system 40 can change the direction of travel of vehicle 1.

[0058] However, the structure of the vehicle 1 of the present invention is not limited to the above example. According to an embodiment, the vehicle 1 may include a motor (not shown) for transmitting power to the wheels in addition to the engine 10 and the transmission 20, or may include only the motor instead of the engine 10 and the transmission 20.

[0059] Vehicle 1 includes multiple electronic components. For example, vehicle 1 also includes an engine management system (EMS) 11, a transmission control unit (TCU) 21, an electronic brake control module (EBCM) 31, an electronic power steering (EPS) 41, a body control module (BCM) 51, a steering angle sensor 61 for sensing the steering wheel angle, and a driver assistance system (DAS) 100.

[0060] The engine management system 11 controls the engine 10 in response to the driver's acceleration intention expressed by the accelerator pedal or a request from the driver assistance system 100. For example, the engine management system 11 controls the torque of the engine 10.

[0061] The transmission control unit 21 controls the transmission 20 in response to a shift command from the driver via the shift lever and / or the driving speed of the vehicle 1. For example, the transmission control unit 21 adjusts the transmission ratio from the engine 10 to the wheels.

[0062] However, as described above, according to embodiments, the vehicle 1 may be a hybrid vehicle further including a motor or an electric vehicle including only a motor, and thus the engine management system 11 and the transmission control unit 21 may be omitted according to embodiments.

[0063] The electronic brake control module 31 controls the brake device 30 in response to the driver's braking intention expressed through the brake pedal and / or wheel slip. For example, the electronic brake control module 31 temporarily releases wheel brakes in response to wheel slip detected when braking the vehicle 1 (Anti-lock Braking Systems, ABS). The electronic brake control module 31 selectively releases wheel brakes in response to oversteering and / or understeering detected when steering the vehicle 1 (Electronic Stability Control, ESC). Furthermore, the electronic brake control module 31 temporarily applies wheel brakes in response to wheel slip detected when driving the vehicle 1 (Traction Control System, TCS).

[0064] The electronic steering system 41 assists the steering system 40 in response to the driver's steering intention expressed through the steering wheel, making it easier for the driver to operate the steering wheel. For example, the electronic steering system 41 assists the steering system 40 by reducing steering force during low-speed driving or parking and increasing steering force during high-speed driving.

[0065] The main body control module 51 controls the operation of electronic components that provide convenience to the driver or ensure the driver's safety. For example, the main body control module 51 controls the headlights, wipers, instrument panel, multi-function switch and direction indicator lights.

[0066] The steering angle sensor 61 can sense the steering angle of the steering device 40 and detect whether the driver has operated the steering wheel. To this end, the steering angle sensor 61 can be a torque sensor provided on one side of the steering device 40, which senses whether the driver has operated the steering wheel by sensing the torque applied to the steering device 40.

[0067] The driver assistance system 100 can assist the driver in operating (driving, braking, steering) the vehicle 1. For example, the driver assistance system 100 senses the environment surrounding the vehicle 1 (e.g., other vehicles, pedestrians, cyclists, lanes, road signs), etc., and controls the driving, braking, and / or steering of the vehicle 1 in response to the sensed environment.

[0068] The driver assistance system 100 provides various functions to the driver. For example, the driver assistance system 100 can provide Lane Departure Warning (LDW), Lane Following Assist (Lane Following Assist, LFA), Blind Spot Detection (BSD), Lane Change Assist (Lane Change Assist, LCA), High Beam Assist (HBA), Autonomous Emergency Braking (AEB), Traffic Sign Recognition (TSR), Smart Cruise Control (SCC), etc.

[0069] The driver assistance system 100 includes a camera module 101 that obtains image data around the vehicle 1 and a radar module 102 that obtains data on objects around the vehicle 1. The camera module 101, which includes a camera 101a and an Electronic Control Unit (ECU) 101b, captures images of the area in front of the vehicle 1 and identifies other vehicles, pedestrians, cyclists, lanes, road signs, and so on. The radar module 102, which includes a radar 102a and a controller 102b, obtains the relative position and relative speed of objects around the vehicle 1 (e.g., other vehicles, pedestrians, cyclists, etc.).

[0070] The driver assistance system 100 is not limited to Figure 1 In the illustrated case, a radar (lidar), an infrared sensor, etc. that scans the periphery of the vehicle 1 and senses an object may also be included.

[0071] The aforementioned electronic components can communicate with each other via the vehicle communication network NT. For example, the electronic components transmit and receive data via Ethernet, Media Oriented Systems Transport (MOST), Flexray, Controller Area Network (CAN), and Local Interconnect Network (LIN). For example, the driver assistance system 100 transmits drive control signals, brake signals, and steering signals to the engine management system 11, the electronic brake control module 31, and the electronic steering system 41 via the vehicle communication network NT. Furthermore, the driver assistance system 100 receives steering angle information from the steering system 40 from the steering angle sensor 61 via the vehicle communication network NT.

[0072] Figure 2 FIG. 1 shows the structure of a driver assistance system 100 according to one embodiment. Figure 3 1 shows a camera and a radar included in the driver assistance system 100 in one embodiment.

[0073] like Figure 2 As shown, the vehicle 1 includes a steering angle sensor 61 , a steering system 42 , a warning system 72 , and a driver assistance system 100 .

[0074] The steering angle sensor 61 senses the steering angle information of the steering device 40 and provides it to the driver assistance system 100. The steering system 42 includes an electronic steering device (41, see Figure 1 ) and the steering device (40, refer to Figure 1 ).

[0075] The warning system 72 provides a warning to the driver based on a warning signal received from the driver assistance system 100 via the vehicle network NT.

[0076] To this end, the warning system 72 includes a processor, a speaker that provides a warning sound to the driver based on the warning signal, and a display that displays a warning message to the driver based on the warning signal. The speaker and display are installed inside the vehicle 1 to provide a warning to the driver.

[0077] In addition, the warning system 72 includes warning lights provided on the left and right rearview mirrors, respectively. Based on the warning signal, the warning lights on the rearview mirrors are illuminated at positions where danger is predicted to occur, providing directional warnings to the driver.

[0078] The driver assistance system 100 includes a camera 110 and a radar 120 .

[0079] like Figure 3 As shown, the camera 110 has fields of view 111 a , 112 a , 113 a , and 114 a facing outside the vehicle 1 .

[0080] To this end, the camera 110 includes a right side camera 111 for obtaining image data about the right side, a left side camera 112 for obtaining image data about the left side, a front camera 113 for obtaining image data about the front, and a rear camera 114 for obtaining image data about the rear.

[0081] The right side camera 111 and the left side camera 112 are respectively disposed on the right side rearview mirror and the left side rearview mirror, for example.

[0082] The front camera 113 is installed on, for example, the front windshield of the vehicle 1 , and the rear camera 114 is installed on, for example, the trunk of the vehicle 1 .

[0083] However, the installation position and number of each camera 110 are not limited to the above example, and can be set in various ways according to embodiments.

[0084] The camera 110 captures images of the exterior of the vehicle 1 and obtains image data of the exterior of the vehicle 1. The image data of the exterior of the vehicle 1 includes positions of other vehicles, pedestrians, cyclists, and lanes located outside the vehicle 1.

[0085] The camera 110 includes a plurality of lenses and an image sensor. The image sensor includes a plurality of photodiodes that convert light into electrical signals. The plurality of photodiodes are arranged in a two-dimensional matrix.

[0086] The camera 110 is electrically connected to the control unit 130. For example, the camera 110 is connected to the control unit 130 via the vehicle communication network NT, a hard wire, or a printed circuit board (PCB).

[0087] The camera 110 transmits image data of the exterior of the vehicle 1 to the control unit 130 .

[0088] like Figure 3 As shown, the radar 120 includes corner radars 121 , 122 , 123 , and 124 installed at corners of the vehicle 1 , a front radar 125 having a sensing field toward the front of the vehicle 1 , and a rear radar 126 having a sensing field toward the rear of the vehicle 1 .

[0089] Multiple corner radars 121, 122, 123, and 124 include a first corner radar 121 located at the right front of the vehicle 1, a second corner radar 122 located at the left front of the vehicle 1, a third corner radar 123 located at the right rear of the vehicle 1, and a fourth corner radar 124 located at the left rear of the vehicle 1.

[0090] like Figure 3 As shown, the first corner radar 121 has a sensing field of view 121a facing the right front of the vehicle 1. The second corner radar 122 has a sensing field of view 122a facing the left front of the vehicle 1 and is installed, for example, on the left side of the front bumper of the vehicle 1. The third corner radar 123 has a sensing field of view 123a facing the right rear of the vehicle 1 and is installed, for example, on the right side of the rear bumper of the vehicle 1. The fourth corner radar 124 has a sensing field of view 124a facing the left rear of the vehicle 1 and is installed, for example, on the left side of the rear bumper of the vehicle 1.

[0091] The first corner radar 121, the second corner radar 122, the third corner radar 123, and the fourth corner radar 124 each include a transmitting antenna and a receiving antenna. The first corner radar 121, the second corner radar 122, the third corner radar 123, and the fourth corner radar 124 each obtain first corner radar data, second corner radar data, third corner radar data, and fourth corner radar data. The first corner radar data includes distance information and speed of other vehicles, pedestrians, or cyclists (hereinafter referred to as "objects") located in front of the right side of the vehicle 1. The second corner radar data includes distance information and speed of objects located in front of the left side of the vehicle 1. The third radar data and the fourth corner radar data include distance information and relative speed of objects located behind the right side of the vehicle 1 and behind the left side of the vehicle 1.

[0092] First corner radar 121, second corner radar 122, third corner radar 123, and fourth corner radar 124 are each connected to control unit 130 via, for example, a vehicle communication network NT, a hard line, or a printed circuit board. First corner radar 121, second corner radar 122, third corner radar 123, and fourth corner radar 124 transmit first, second, third, and fourth corner radar data, respectively, to control unit 130.

[0093] The front radar 125 is provided, for example, on the right side of the front bumper of the vehicle 1 .

[0094] like Figure 3 As shown, the front radar 125 includes a sensing field 125a directed toward the front of the vehicle 1. The front radar 125 is provided on a grille or a bumper of the vehicle 1, for example.

[0095] The forward radar 125 includes a transmitting antenna (or transmitting antenna array) that radiates transmitted radio waves toward the front of the vehicle 1, and a receiving antenna (or receiving antenna array) that receives reflected radio waves from objects. The forward radar 125 obtains forward radar data from the transmitted radio waves transmitted by the transmitting antenna and the reflected radio waves received by the receiving antenna. The forward radar data includes distance information and speed information about other vehicles, pedestrians, or cyclists located in front of the vehicle 1. The forward radar 125 calculates the state distance to the object based on the phase difference (or time difference) between the transmitted radio wave and the reflected radio wave, and calculates the relative speed of the object based on the frequency difference between the transmitted radio wave and the reflected radio wave.

[0096] The front radar 125 is connected to the control unit 130 via, for example, a vehicle communication network NT, a hard line, or a printed circuit board. The front radar 125 transmits front radar data to the control unit 130.

[0097] The rear radar 126 obtains rear radar data about the rear of the vehicle 1 and is provided on the bumper of the vehicle 1 .

[0098] The control unit 130 includes a camera module (101, see Figure 1 ), controller (101b, Figure 1 Reference) and / or radar module (102, reference Figure 1 ) of the controller (102b, refer to Figure 1 ) and / or a separate master controller.

[0099] The control unit 130 includes a processor 131 and a memory 132 .

[0100] The processor 131 processes the image data from the camera 110, the radar data from the radar 120, and the steering angle information from the steering angle sensor 61 to generate a steering signal and a warning signal for controlling the steering system 42 and the warning system 72. For example, the processor 131 includes an image processor for processing the image data from the camera 110 and / or a digital-to-analog processor for processing the radar data from the radar 120 and / or a microcontroller unit (MCU) for generating the steering signal and the warning signal.

[0101] The processor 131 sends a turn signal to the vehicle 1 based on at least one of the image data from the camera 110 or the radar data from the radar 120 to implement lane following assistance.

[0102] Specifically, the processor 131 identifies a lane or a preceding vehicle based on at least one of the image data or the radar data, determines a target path based on the identified lane or preceding vehicle, and sends a steering signal to the steering system 42 to track the target path.

[0103] For example, when a lane is recognized, the processor 131 determines a target path for traveling in the center of the lane. In addition, when traveling in a lane-disappearing section such as an intersection during lane control, the processor 131 determines a target path for tracking the vehicle ahead.

[0104] The processor 131 sends a steering signal to the steering system 42 so that the vehicle 1 travels on the target path. The electronic steering device 41 of the steering system 42 controls the steering device 40 based on the steering signal received from the driver assistance system 100 so that the vehicle 1 travels on the target path.

[0105] The processor 131 determines the steering angle of the steering device 40 based on the output of the steering angle sensor 61. At this time, the steering angle of the steering device 40 corresponds to the steering angle of the steering wheel.

[0106] The processor 131 senses the driver's operation of the steering device 40 based on the steering angle and the steering signal of the steering device 40 .

[0107] Specifically, the processor 131 determines the steering angle and steering angle change speed of the steering device 40 based on the output of the steering angle sensor 61. When the steering angle and the steering angle change speed are maintained within a preset range for a preset time, it is judged that there is no hands-off state of the driver's operation of the steering device 40.

[0108] In this case, processor 131 adjusts the setting ranges corresponding to the steering angle and steering angle change rate based on the turn signal used to implement lane following assist. In other words, processor 131 sets the range for determining the driver's operation based on the steering angle and steering angle change rate achieved in response to the turn signal.

[0109] The processor 131 transmits a warning signal to the vehicle 1 in such a manner that a warning regarding the hands-off is output in stages according to a time period during which the driver's operation of the steering device 40 is not sensed.

[0110] For example, the processor 131 generates a first warning signal so that a lower-level (primary) warning is output if a first time period has passed after the hands-off state is determined and no driver operation is sensed. Furthermore, the processor 131 generates a second warning signal so that a higher-level (secondary) warning is output if a second time period has passed after the lower-level warning is output and no driver operation is sensed.

[0111] In this case, the lower-level warning corresponds to a visual warning, and the higher-level warning corresponds to an auditory warning or at least one of a visual warning and an auditory warning. Specifically, processor 131 generates a first warning signal targeted at the display to cause the display of warning system 72 to output a warning, and generates a second warning signal targeted at the speaker to cause the speaker of warning system 72 to output a warning. According to an embodiment, the second warning signal includes a signal targeted at the display.

[0112] At this time, if a third time has passed since the output of the higher-level warning, processor 131 transmits a third warning signal to vehicle 1 to warn that the lane following assist system can be disabled. Furthermore, if a fourth time has passed since the warning was issued in response to the third warning signal, processor 131 disables the lane following assist function.

[0113] In addition, the processor 131 sends a second warning signal to the vehicle 1 in a manner that outputs a warning about the hands-off state in stages according to the time when the driver's operation of the steering device 40 is not sensed, and immediately outputs a higher-level (secondary) warning when it is determined that the situation is not suitable for assistance based on at least one data of the image data or the radar data.

[0114] That is, the processor 131 sends a warning signal to the vehicle 1 in such a manner that a higher-level (secondary) warning is immediately output when a condition is determined to be unsuitable for assistance before a first time has passed after the hands-off state is determined or before a second time has passed after the lower-level (primary) warning is output.

[0115] In other words, in the event of at least one of a situation that deviates from the control conditions for achieving lane following assist or a collision hazard situation, the processor 131 determines that it is not appropriate to perform lane following assist, determines the time point as a situation that is not appropriate for assistance, and sends a warning signal to immediately output a higher-level (secondary) warning.

[0116] If the lane disappears during lane recognition and lane following assist, the processor 131 determines that the situation is out of the control condition for implementing the lane following assist.

[0117] Furthermore, if the relative position of the vehicle with respect to a surrounding object (e.g., another vehicle, pedestrian, cyclist, etc.) is determined based on at least one of the image data or the radar data, and if a risk of direct collision with the surrounding object exists based on the relative position, the processor 131 determines a collision risk condition. Specifically, the processor 131 calculates the expected time to collision (TTC) or distance to collision (DTC) with the surrounding object based on at least one of the image data or the radar data, and determines whether a collision risk exists by comparing the calculated time to collision (TTC) or distance to collision (DTC) with the surrounding object.

[0118] Furthermore, if the preceding vehicle changes its path while lane following assist is being performed while tracking the preceding vehicle, processor 131 determines that the situation is not suitable for assistance. Specifically, if the preceding vehicle rotates by an angle greater than a preset angle, processor 131 determines that the preceding vehicle has changed its path and that the preceding vehicle's path change has deviated from the control conditions for lane following assist or that there is a risk of collision with the preceding vehicle, thus determining the situation as not suitable for assistance.

[0119] In addition, when a lane is recognized and lane following assist is performed and a branch point is reached where the lane spacing becomes narrower, the processor 131 determines that the situation deviates from the control conditions for implementing lane following assist or is a collision risk situation, thereby determining that the situation is not suitable for assistance.

[0120] In addition, when the lane is recognized and lane following assist is performed and the lane distance narrows, the processor 131 determines that the situation deviates from the control conditions for implementing lane following assist or is a collision risk situation, thereby determining that the situation is not suitable for assistance.

[0121] The memory 132 stores programs and / or data for the processor 131 to process image data, programs and / or data for processing radar data, and programs and / or data for the processor 131 to generate turn signals and / or warning signals.

[0122] The memory 132 temporarily stores the image data received from the camera 110 and / or the radar data received from the radar 120 , and temporarily stores the processing results of the image data and / or radar data by the processor 131 .

[0123] The memory 132 includes not only volatile memories such as S random access memory (S-RAM) and D random access memory (D-RAM), but also non-volatile memories such as flash memory, read-only memory (ROM), and erasable programmable read-only memory (EPROM).

[0124] The driver assistance system 100 is not limited to Figure 2 The illustrated embodiment may also include a radar (lidar), an infrared sensor, etc. that scans the periphery of the vehicle 1 and senses external objects such as lanes and vehicles ahead.

[0125] The above describes the configuration of the driver assistance system 100. Next, a detailed description will be given of the case where the driver assistance system 100 performs lane following assistance.

[0126] Figure 4 FIG. 1 shows a case where the driver assistance system 100 of one embodiment recognizes a lane and performs lane following assistance. Figure 5 FIG. 1 shows a case where the driver assistance system 100 according to one embodiment tracks a preceding vehicle and performs lane following assistance.

[0127] Reference Figure 4 and Figure 5 The processor 131 sends a turn signal to the vehicle 1 based on at least one of the image data from the camera 110 or the radar data from the radar 120 to perform lane following assistance.

[0128] Specifically, the processor 131 identifies the lane 400 or the front vehicle 2 based on at least one of the image data or the radar data, determines the target path 450 based on the identified lane 400 or the front vehicle 2, and sends a steering signal to the steering system 42 to track the target path 450.

[0129] For example, when lane 400 is identified, processor 131 determines target path 450 so as to travel in the center of the lane. Specifically, processor 131 determines target path 450 as the center of a lane formed by first lane 400a on the left side of vehicle 1 and second lane 400b on the right side of vehicle 1.

[0130] Furthermore, when driving in a lane-disappearing section such as an intersection during lane control, processor 131 determines target path 450 to track leading vehicle 2. Specifically, processor 131 determines the lateral distance x, longitudinal distance y, and azimuth angle θ between the vehicle and the leading vehicle based on at least one of image data or radar data, thereby determining the relative position of the vehicle and the leading vehicle, and determines the path for tracking the leading vehicle as target path 450.

[0131] The processor 131 sends a steering signal to the steering system 42 so that the vehicle 1 travels on the target path 450. The electronic steering device 41 of the steering system 42 controls the steering device 40 based on the steering signal received from the driver assistance system 100 so that the vehicle 1 travels on the target path.

[0132] The above description describes the case where the driver assistance system 100 performs lane following assistance. Next, the case where the driver assistance system 100 senses the driver's operation of the steering device 40, determines the hands-off state, and issues a warning will be described in detail.

[0133] Figure 6 The driver assistance system of one embodiment detects a hands-off state and issues warnings in stages. Figure 7 This shows a situation where a driver assistance system according to an embodiment senses a situation that is not suitable for assistance and issues a warning. Figure 8 This figure shows a situation in which a driver assistance system according to an embodiment senses a hands-off state and issues warnings in stages.

[0134] Reference Figure 6 The processor 131 determines the steering angle of the steering device 40 based on the output of the steering angle sensor 61. At this time, the steering angle of the steering device 40 corresponds to the steering angle of the steering wheel.

[0135] Specifically, the processor 131 determines the steering angle and steering angle change speed of the steering device 40 based on the output of the steering angle sensor 61. When the steering angle and the steering angle change speed are maintained within a preset range for a preset time, it is judged that the driver has not released the steering device 40. The hands-off state.

[0136] In this case, processor 131 adjusts the setting ranges for the steering angle and steering angle change rate based on the turn signal used to implement lane following assist. In other words, processor 131 sets the ranges for determining the driver's operation based on the steering angle and steering angle change rate reflected by the turn signal.

[0137] like Figure 6 As shown, the processor 131 sends a warning signal to the vehicle 1 according to the time when the driver's operation on the steering device 40 is not sensed, so as to output a warning about the hands-off state in stages.

[0138] For example, the processor 131 generates a first warning signal so that a lower-level (primary) warning is output if a first time period has passed after the hands-off state is determined and no driver operation is sensed. Furthermore, the processor 131 generates a second warning signal so that a higher-level (secondary) warning is output if a second time period has passed after the lower-level warning is output and no driver operation is sensed.

[0139] In this case, the lower-level warning corresponds to a visual warning, and the upper-level warning corresponds to an auditory warning or at least one of a visual warning and an auditory warning. For example, the visual warning may be a message on a display warning to hold the steering wheel ("Please hold the steering wheel"), and the auditory warning may be a message on a speaker warning to hold the steering wheel ("Please hold the steering wheel").

[0140] That is, the processor 131 generates a first warning signal targeted at the display to output a warning to the display of the warning system 72, and generates a second warning signal targeted at the speaker to output a warning to the speaker of the warning system 72. According to an embodiment, the second warning signal includes a signal targeted at the display.

[0141] At this time, if a third time has passed after the output of the higher-level warning, processor 131 transmits a third warning signal to vehicle 1 to warn that the lane following assist system can be disabled. Furthermore, if a fourth time has passed after the warning issued by the third warning signal, processor 131 disables the lane following assist function and disables the lane following assist system.

[0142] like Figure 7 and Figure 8 As shown, the processor 131 sends a second warning signal to the vehicle 1 in a manner that periodically outputs a warning about the hands-off state according to the time when the driver's operation of the steering device 40 is not sensed, and immediately outputs a higher-level (secondary) warning when it is determined that the situation is not suitable for assistance based on at least one data in the image data or radar data.

[0143] That is, Figure 7 As shown, the processor 131 sends a warning signal to the vehicle 1 in such a manner that, when the situation is determined to be unsuitable for assistance before a first time has passed after the hands-off state is determined, the upper-level (secondary) warning is immediately output.

[0144] In addition, if Figure 8 As shown, the processor 131 transmits a warning signal to the vehicle 1 so as to immediately output a high-level (secondary) warning when the situation is determined to be unsuitable for assistance before the second time elapses after the output of the low-level (primary) warning.

[0145] In other words, in the event of at least one of a situation that deviates from the control conditions for achieving lane following assist or a collision hazard situation, the processor 131 determines that it is not appropriate to perform lane following assist, determines the time point as a situation that is not appropriate for assistance, and sends a warning signal to immediately output a higher-level (secondary) warning.

[0146] The above describes a case where the driver assistance system 100 senses the driver's operation of the steering device 40, determines a hands-off state, and issues a warning. Next, a case where the driver assistance system 100 determines a situation inappropriate for assistance will be described in detail.

[0147] Figure 9 This shows a case where the driver assistance system 100 of one embodiment determines that the situation is out of the control condition for implementing lane following assist due to lane disappearance. Figure 10 This shows a case where the driver identification system 100 of one embodiment determines that a collision risk situation exists. Figure 11 This shows a case where the driver recognition system 100 of one embodiment determines that the path of the leading vehicle 2 has changed and the situation is not suitable for assistance. Figure 12 This shows a case where the driver identification system 100 of one embodiment determines that the driver is not suitable for assistance due to reaching a branch point. Figure 13 The driver identification system 100 according to one embodiment shows a case where the driver reaches a merging point and determines that the situation is not suitable for assistance.

[0148] Reference Figure 9 When lane (400a, 400b; 400) disappears while lane following assist is being executed after lane (400a, 400b) is recognized, processor 131 determines that the situation is a departure from the control conditions for lane following assist. Specifically, when vehicle 1 enters a lane disappearance zone while lane (400a, 400b; 400) is recognized and lane following assist is being executed, processor 131 determines that the situation is a departure from the control conditions for lane following assist, and transmits a warning signal when no driver operation of steering device 40 is sensed, thereby immediately outputting a higher-level (secondary) warning.

[0149] Furthermore, processor 131 determines the relative position of the vehicle with respect to surrounding objects (e.g., other vehicles, pedestrians, cyclists, etc.) based on at least one of the image data or radar data. If the vehicle determines that there is a risk of collision with the surrounding objects based on the relative position, the vehicle determines that a collision risk situation exists. Specifically, processor 131 calculates the expected time to collision (TTC) or distance to collision (DTC) with the surrounding objects based on at least one of the image data or radar data, and determines whether there is a risk of collision by comparing the calculated time to collision (TTC) or distance to collision (DTC) with the surrounding objects.

[0150] For example, Figure 10 As shown, when a collision with another vehicle 3 is expected due to a path change of the front vehicle 2 while performing lane following assist while tracking the front vehicle 2, the processor 131 determines that it is a collision risk situation.

[0151] In addition, if Figure 11 As shown, when the preceding vehicle 2 changes its path while performing lane following assist while tracking the preceding vehicle 2, the processor 131 determines that the situation is not suitable for assist. Specifically, if the preceding vehicle 2 rotates by an angle greater than a preset angle, the processor 131 determines that the preceding vehicle 2 has changed its path and that the situation is not suitable for assist because the preceding vehicle 2's path change has deviated from the control conditions for implementing lane following assist or there is a risk of collision with the preceding vehicle 2.

[0152] In addition, if Figure 12 As shown, when a vehicle reaches a branch point where the lane spacing l widens while lane recognition is being performed and lane following assistance is being executed, processor 131 determines that the vehicle is not in a condition suitable for lane following assistance due to a violation of the control conditions for lane following assistance or a collision risk. Specifically, when the vehicle reaches a branch point, processor 131 determines that lane following assistance is difficult to perform or that the risk of collision with another vehicle is increasing, thereby determining that the vehicle is in a condition suitable for assistance.

[0153] Specifically, when the lane width (lane spacing) I reaches a preset first value, processor 131 determines that the vehicle has reached a branch point, i.e., a critical position within the branch interval. In this case, processor 131 determines that the vehicle has deviated from the control conditions for lane following assist or is in a collision-prone state, making the vehicle unsuitable for assist.

[0154] In addition, if Figure 13As shown, when a vehicle reaches a merging point where the lane spacing l narrows while lane recognition is being performed and lane following assist is in progress, processor 131 determines that the vehicle is not in a condition suitable for lane following assist, or that there is a risk of collision, and therefore the vehicle is not suitable for assist. Specifically, when the vehicle reaches the merging point, processor 131 determines that lane following assist is difficult to perform or that the risk of collision with another vehicle is increasing, and therefore determines the vehicle is not suitable for assist.

[0155] Specifically, when lane width (lane spacing) I reaches a preset second value, processor 131 determines that the vehicle has reached a merging point, i.e., a critical position within the merging interval. In this case, processor 131 determines that the vehicle has deviated from the control conditions for lane following assist or is in a collision-prone state, making the vehicle unsuitable for assist.

[0156] Next, a control method of the driver assistance system 100 according to an embodiment will be described. The control method of the driver assistance system 100 described below can be applied to the driver assistance system 100 according to the embodiment described above. Therefore, unless otherwise specified, the above reference Figures 1 to 13 The description is also applicable to the control method of the driver assistance system 100 of an embodiment.

[0157] Figure 14 1 is a sequence diagram showing a method of outputting a warning in lane following assistance by the driver assistance system 100 according to one embodiment.

[0158] Reference Figure 14 , the driver assistance system 100 of one embodiment sends a first warning signal (1430) to the vehicle 1 to output a warning of the lower stage (once) when the driver does not operate the steering device 40 (1410 is) and a first time has passed (1420 is).

[0159] Furthermore, when a second time has passed after the output of the lower-level warning (Yes at 1440 ), the driver assistance system 100 of one embodiment transmits a second warning signal ( 1450 ) to the vehicle 1 to output a higher-level (secondary) warning.

[0160] The processor 131 determines the steering angle and steering angle change speed of the steering device 40 based on the output of the steering angle sensor 61. When the steering angle and steering angle change speed are maintained within a preset range for a preset time, it is judged that the driver is in a hands-off state and has not operated the steering device 40.

[0161] Processor 131 adjusts the setting ranges for the steering angle and steering angle change rate based on the turn signal used to implement lane following assist. In other words, processor 131 sets the ranges for determining the driver's operation based on the steering angle and steering angle change rate reflected by the turn signal.

[0162] The lower-level warning corresponds to a visual warning, while the higher-level warning corresponds to an auditory warning or at least one of a visual warning and an auditory warning. Specifically, processor 131 generates a first warning signal targeted at the display to cause the display of warning system 72 to output a warning, and generates a second warning signal targeted at the speaker to cause the speaker of warning system 72 to output a warning. According to an embodiment, the second warning signal includes a signal targeted at the display.

[0163] In addition, the driver assistance system 100 sends a second warning signal to the vehicle 1 in a manner that outputs a warning of the hands-off state in stages according to the time when the driver's operation of the steering device 40 is not sensed, and immediately outputs a higher-level (secondary) warning when it is determined that the situation is not suitable for assistance based on at least one data of the image data or the radar data.

[0164] That is, the driver assistance system 100 sends a warning signal (1450) to the vehicle 1 in such a manner that when the steering device 40 is not operated (1460: No) before a first time has passed after the hands-off state is determined (1420: No) and it is determined that the situation is not suitable for assistance (1470: Yes), the upper stage (secondary) warning is immediately output.

[0165] In addition, the driver assistance system 100 sends a warning signal (1450) to the vehicle 1 in such a manner that, when the steering device 40 is not operated (No in 1480) before a second time has passed (No in 1440) after sending the warning signal (1430) for outputting the warning of the lower stage (primary), and when it is determined that the situation is not suitable for assistance (Yes in 1490), the warning of the upper stage (secondary) is immediately output.

[0166] Alternatively, the disclosed embodiments may be implemented as a recording medium storing computer-executable commands. The commands may be stored in the form of program code, which, when executed by a processor, generates a program module to perform the actions of the disclosed embodiments. The recording medium may be computer-readable.

[0167] Computer-readable recording media include all types of recording media that store computer-readable command language, such as ROM (Read Only Memory), RAM (Random Access Memory), magnetic tapes, magnetic disks, flash memories, and optical data storage devices.

[0168] As described above, the disclosed embodiments have been described with reference to the accompanying drawings. Those skilled in the art may implement the present invention in forms different from those of the disclosed embodiments without changing the technical concept or essential features of the present invention. The disclosed embodiments are merely illustrative and the present invention is not limited thereto.

Claims

1. A driver assistance system comprising: A camera is mounted on the vehicle to provide an external field of view of the vehicle and obtain image data; a radar mounted on the vehicle to provide an external sensing field of view of the vehicle and to obtain radar data; and a control unit comprising at least one processor for processing image data obtained by the camera and radar data obtained by the radar, The control unit sends a steering signal to the vehicle based on at least one of the image data or the radar data to achieve lane following assist, senses the driver's operation of the steering device based on the steering angle of the steering device and the steering signal, and sends a warning signal to the vehicle in a manner that outputs warnings in stages according to the time when the operation is not sensed, and determines an unsuitable assistance condition that is not suitable for the lane following assist based on at least one of the image data or the radar data. When it is determined to be the unsuitable assistance condition, the control unit sends a warning signal to the vehicle in a manner that outputs a higher-level warning than the warning output when the driver's operation of the steering device is not sensed.

2. The driver assistance system according to claim 1, wherein: The control unit sends a first warning signal to the vehicle in such a manner that if no operation occurs within a first time, the vehicle is determined to be in a hands-off state and a lower-level warning is output; and sends a second warning signal to the vehicle in such a manner that if no operation occurs within a second time after the lower-level warning is output, an upper-level warning is output.

3. The driver assistance system according to claim 2, wherein: The warnings in the above lower stages are visual warnings. The above-mentioned high-level warning is at least one of a visual warning and an audible warning.

4. The driver assistance system according to claim 2, wherein: The control unit transmits a warning signal to the vehicle so as to immediately output a high-level warning when determining that the situation is not suitable for assistance before the first time has passed or before a second time has passed after the low-level warning has been output.

5. The driver assistance system according to claim 1, wherein: The control unit determines that the situation is not suitable for the assistance when at least one of a situation that deviates from a control condition for implementing the lane following assistance or a collision risk situation occurs.

6. The driver assistance system according to claim 5, wherein: When the lane disappears while the lane is recognized and the lane following assist is being executed, the control unit determines that the situation is out of a control condition for implementing the lane following assist.

7. The driver assistance system according to claim 5, wherein: The control unit determines that the collision risk situation is the collision risk situation when there is a risk of collision with the surrounding objects based on the relative positions of the surrounding objects.

8. The driver assistance system according to claim 5, wherein: The control unit determines that the lane following assist is not suitable when the leading vehicle changes its path while the lane following assist is being performed while the leading vehicle is being followed.

9. The driver assistance system according to claim 5, wherein: When a lane is recognized and the lane following assistance is executed, and a branch point where the lane interval widens is reached, the control unit determines that the assistance is not suitable. The branch point is a critical position within the branch interval where the lane spacing reaches a preset value.

10. The driver assistance system according to claim 5, wherein: When a merging point where the lane distance narrows is reached during lane recognition and lane following assistance is executed, the control unit determines that the assistance is not suitable. The merging point is a critical position within the merging interval where the lane spacing reaches a preset value.

11. A method for controlling a driver assistance system, the driver assistance system comprising: A camera is mounted on the vehicle to provide an external field of view of the vehicle and obtain image data; and a radar, which is provided on the vehicle to have an external sensing field of view of the vehicle and obtain radar data, The control method of the driver assistance system includes: Sending a steering signal to the vehicle based on at least one of the image data or the radar data to implement lane following assistance; sensing a driver's operation of the steering device based on a steering angle of the steering device and the steering signal; sending a warning signal to the vehicle in such a manner that a warning is output in stages according to a time period during which no operation is sensed; Based on at least one of the above-mentioned image data or the above-mentioned radar data, a situation that is not suitable for the above-mentioned lane following assistance is determined to be unsuitable for assistance. When it is determined to be the above-mentioned situation that is not suitable for assistance, a warning signal is sent to the above-mentioned vehicle in a manner such that the output is a higher-level warning compared to the warning output when the driver's operation of the steering device is not sensed.

12. The control method of the driver assistance system according to claim 11, wherein: The step of sending a warning signal to the vehicle in a manner such that a warning is output in stages according to a time when the operation is not sensed includes: sending a first warning signal to the vehicle in such a manner that, if no operation occurs within a first period of time, a hands-off state is determined and a lower-level warning is output; A second warning signal is transmitted to the vehicle so that a higher-level warning is output when no operation occurs within a second time period after the lower-level warning is output.

13. The control method of the driver assistance system according to claim 12, wherein: The warnings in the above lower stages are visual warnings. The above-mentioned high-level warning is at least one of a visual warning and an audible warning.

14. The control method of the driver assistance system according to claim 12, wherein: The step of sending a warning signal to the vehicle in such a manner as to output the warning of the higher level stage includes: A warning signal is sent to the vehicle so that a high-level warning is immediately output when the situation is determined to be unsuitable for assistance before the first time has passed or before a second time has passed after the low-level warning has been output.

15. The control method of the driver assistance system according to claim 11, wherein: Conditions identified as unsuitable for assistance include: When at least one of a situation that deviates from the control conditions for implementing the lane following assist or a collision risk situation occurs, the situation is determined to be the situation not suitable for assisting.

16. The control method of the driver assistance system according to claim 15, wherein: Conditions identified as unsuitable for assistance include: If the lane disappears while the lane is recognized and the lane following assist is being executed, it is determined that the situation has deviated from the control conditions for implementing the lane following assist.

17. The control method of the driver assistance system according to claim 15, wherein: Conditions identified as unsuitable for assistance include: The collision risk situation is determined when there is a risk of collision with the surrounding objects based on the relative positions of the vehicle.

18. The control method of the driver assistance system according to claim 15, wherein: Conditions identified as unsuitable for assistance include: If the leading vehicle changes its path while the lane following assist is being performed while following the leading vehicle, it is determined that the assist is not suitable.

19. The control method of the driver assistance system according to claim 15, wherein: Conditions identified as unsuitable for assistance include: If the lane is recognized and the lane following assistance is executed and a branch point where the lane spacing widens is reached, it is determined that the assistance is not suitable. The branch point is a critical position within the branch interval where the lane spacing reaches a preset value.

20. The control method of the driver assistance system according to claim 15, wherein: Conditions identified as unsuitable for assistance include: If the lane is recognized and the lane following assistance is executed, and the lanes reach a merging point where the distance between lanes becomes narrow, it is determined that the assistance is not suitable. The merging point is a critical position within the merging interval where the lane spacing reaches a preset value.

Images