Driver assistance methods based on drive recorder footage and devices for carrying out these methods

Abstract

To make it possible to realize additional driving assistance with traffic lane detection, object distance estimation, traffic lane deviation determination, and interruption determination on a dashboard camera so as to improve recording performance of the dashboard camera depending on a situation.SOLUTION: The present invention relates to a driver assisting method based on dashboard camera video and a device for executing the same. The driver assisting method based on dashboard camera video may include: a step for detecting a traffic lane with a driver assisting device; and a step for estimating an object distance based on a traffic lane with a driver assisting device.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a driver assistance method based on a driving recorder image and an apparatus for executing the method, more particularly to a driver assistance method based on a driving recorder image for providing a model for detecting lanes quickly and lightly in an environment using limited resources such as a driving recorder, and an apparatus for executing the method. [Background technology]

[0002] With the recent rapid development of sensor technology, artificial intelligence, and computer technology, active driver assistance systems such as Autonomous Emergency Braking Systems, Forward Collision Warning Systems, and Lane Departure Warning Systems are increasingly being installed in cars ranging from luxury vehicles to compact cars. In particular, lane departure warning systems, which warn drivers when they unintentionally cross lanes to prevent accidents, have become so popular that even compact cars are being equipped with the system.

[0003] Active driver assistance systems utilize radar, lidar, ultrasonic and vision sensors, with vision sensors being essential for recognizing and classifying vehicles, pedestrians and traffic signs ahead, as well as for lane detection.

[0004] Research into active driver assistance systems for vehicles themselves is being actively conducted, but research into the aforementioned active driver assistance systems for devices such as drive recorders is still insufficient. In drive recorders, it is necessary to apply these active driver assistance systems in order to make advance decisions and take pictures when an alarm about an accident is sounded or an accident is predicted to occur.

[0005] Currently, due to resource limitations, it is difficult to realize a driving recorder as an active driver assistance system in the vehicle itself. Therefore, research is needed to realize the role of an active driver assistance system on a driving recorder. Summary of the Invention [Problem to be solved by the invention]

[0006] An object of the present invention is to solve all of the above-mentioned problems.

[0007] Another object of the present invention is to implement a device with limited resources, such as a drive recorder, as a driver assistance device and provide a driver assistance service through lane detection.

[0008] Another object of the present invention is to perform lane detection, object distance estimation, lane departure judgment, and cut-in judgment on a drive recorder, thereby providing additional driving assistance and improving the situation-specific recording performance of the drive recorder. [Means for solving the problem]

[0009] A representative configuration of the present invention for achieving the above object is as follows.

[0010] According to one embodiment of the present invention, a driver assistance method based on a driving recorder image may include a step of a driver assistance device detecting a lane, and a step of the driver assistance device estimating an object distance based on the lane.

[0011] Meanwhile, a driver assistance method based on a drive recorder image may include a step of the driver assistance device determining lane departure, and a step of the driver assistance device determining a cut-in.

[0012] The object distance may also be determined based on an object distance estimation table.

[0013] According to another embodiment of the present invention, a driver assistance device for assisting a driver based on a dashcam video may be implemented to detect lanes and estimate object distances based on the lanes.

[0014] On the other hand, the driver assistance device can be implemented to determine lane departure and cut-in.

[0015] The object distance may also be determined based on an object distance estimation table. Effect of the Invention

[0016] According to the present invention, resource-limited devices such as dashcams can be implemented as driver assistance devices to provide driver assistance services through lane detection.

[0017] In addition, according to the present invention, additional driving assistance is possible through lane detection, object distance estimation, lane departure judgment, and cut-in judgment on the drive recorder, thereby improving the situation-specific recording performance of the drive recorder. [Brief description of the drawings]

[0018] [Figure 1] 1 is a conceptual diagram showing a driver assistance device according to an embodiment of the present invention; [Diagram 2] FIG. 4 is a conceptual diagram illustrating an operation of a lane detection unit according to the embodiment of the present invention. [Diagram 3] FIG. 2 is a conceptual diagram illustrating an object distance estimator according to an embodiment of the present invention. [Figure 4] FIG. 2 is a conceptual diagram illustrating an object distance estimator according to an embodiment of the present invention. [Diagram 5] FIG. 4 is a conceptual diagram illustrating an object distance estimation table according to an embodiment of the present invention. [Figure 6] 1 is a conceptual diagram showing a lane departure detection method according to an embodiment of the present invention; [Figure 7] FIG. 2 is a conceptual diagram illustrating an interruption detection method according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0019] The following detailed description of the present invention will be made with reference to the accompanying drawings, which show, by way of example, specific embodiments in which the present invention may be practiced. Such embodiments are described in detail to enable those skilled in the art to fully practice the present invention. It should be understood that the various embodiments of the present invention are different from one another but are not necessarily mutually exclusive. For example, specific shapes, structures and characteristics described herein may be embodied in one embodiment without departing from the spirit and scope of the present invention. It should also be understood that the location or arrangement of individual components within each embodiment may be changed without departing from the spirit and scope of the present invention. Therefore, the following detailed description is not to be taken in a limiting sense, and the scope of the present invention should be understood to include the scope of the claims and all equivalents thereto. In the drawings, like reference numerals indicate the same or similar components throughout the various aspects.

[0020] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, various preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings in order to enable those skilled in the art to which the present invention pertains to easily practice the present invention.

[0021] Hereinafter, the driver assistance device according to the embodiment of the present invention can be interpreted to include not only a driving recorder but also various devices that assist the driver based on lane detection.

[0022] FIG. 1 is a conceptual diagram showing a driver assistance device according to an embodiment of the present invention.

[0023] In FIG. 1, a driver assistance device for assisting a driver in driving is disclosed.

[0024] Referring to FIG. 1, the driver assistance device may include a lane detection unit 110, an object distance estimation unit 120, a lane departure determination unit 130, an intrusion determination unit 140, and a processor 150.

[0025] The lane detection unit 110 can be implemented for lane detection. For example, the lane detection unit 110 can perform lane detection based on a light artificial intelligence model such as YOLO. The lane detection unit 110 can only detect lanes within a critical area close to the vehicle (hereinafter, close area driving lanes) among lanes on which the vehicle is currently driving, without detecting all lanes in the image.

[0026] The object distance estimation unit 120 can be implemented to determine the object distance. The object distance estimation unit 120 can determine a lane width and a vanishing point based on the nearby area driving lane, and estimate the object distance based on the determined lane width and vanishing point. The position of an object located in the image can be estimated based on an object distance estimation table that can estimate the object distance according to the lane width and the vanishing point. The object distance estimation table can include information on the estimated distance for each position in the image according to the lane width and the position of the vanishing point.

[0027] The lane departure determination unit 130 may determine whether or not a vehicle is deviating from a lane based on information about the driving lane in the nearby area and the center of the image. The lane width may be determined based on the driving lane in the nearby area, and the center of the lane may be determined. If the center of the image is outside a specific range based on the center of the lane, the lane departure determination unit 130 may determine that the vehicle is deviating from the lane.

[0028] The cut-in determination unit 140 may perform a determination as to whether another vehicle has cut in and a determination as to whether the vehicle itself has cut in. The determination as to whether another vehicle has cut in may be performed based on whether the vehicle enters the driving lane within a critical distance based on the object distance estimation by the object distance estimation unit 120. The determination as to whether the vehicle itself has cut in may be performed based on whether the vehicle deviates from the lane based on the lane departure by the lane departure determination unit 130 and the object distance estimation by the object distance estimation unit, and whether another vehicle exists within the critical distance.

[0029] The processor 150 can control the operations of the lane detection unit 110 , the object distance estimation unit 120 , the lane departure determination unit 130 , and the cut-in determination unit 140 .

[0030] FIG. 2 is a conceptual diagram showing the operation of the lane detection unit according to the embodiment of the present invention.

[0031] FIG. 2 shows the lane detection operation of the lane detection unit.

[0032] Referring to FIG. 2, the lane detection unit detects lanes in the entire image area, and can detect driving lanes in a nearby area located within a critical distance based on the vehicle.

[0033] In driver assistance devices with limited available resources, such as drive recorders, lane classes can be constructed based on YOLO to detect only nearby driving lanes.

[0034] FIG. 3 is a conceptual diagram illustrating an object distance estimator according to an embodiment of the present invention.

[0035] In FIG. 3, we disclose how the object distance estimator determines lane width and vanishing points.

[0036] Referring to FIG. 3, a lane width 300 may be obtained based on lane detection information extracted from adjacent area driving lanes, and a vanishing point 350 may be determined by connecting the lanes.

[0037] The lane width (a) 300 may be obtained by connecting the bottom endpoint 310 of the vehicle image on the image with the point where the lane meets.

[0038] The vanishing point 350 may be determined by connecting the lanes. The point where the lanes of adjacent areas on the image meet when connected with a straight line may be the vanishing point 350. The height of the vanishing point may be determined by subtracting the height c of the bottom end point 310 of the vehicle image and the length b between the bottom end point 310 of the vehicle image and the vanishing point 350 from the total image length.

[0039] Also, the vanishing point 350 can be estimated based only on information about the lane width 300 and the bottom endpoint 310 of the vehicle image. A vanishing point mapping table exists that can estimate the vanishing point based only on information about the lane width 300 and the bottom endpoint 310 of the vehicle image, and the vanishing point 350 can be determined based on the vanishing point mapping table.

[0040] FIG. 4 is a conceptual diagram illustrating an object distance estimator according to an embodiment of the present invention.

[0041] FIG. 4 discloses a method in which the object distance estimation unit estimates the object distance based on an object distance estimation table.

[0042] Referring to FIG. 4, an object distance estimation table 450 is a table for estimating the distance to an object located at a specific position in an image based on a lane width 400 and a vanishing point 410 .

[0043] The object distance estimation table 450 is a table that estimates the distance to an object located on the image based on information about the lane width 400 and the vanishing point 410, and may be generated based on different circles having different centers.

[0044] FIG. 5 is a conceptual diagram illustrating an object distance estimation table according to one embodiment of the present invention.

[0045] In FIG. 5, a method for generating an object distance estimation table is disclosed.

[0046] Referring to FIG. 5, the object distance estimation table may be generated as a first type object distance estimation table 510 or a second type object distance estimation table 520 and selectively utilized.

[0047] The first type object distance estimation table 510 is a table for determining the object distance based on coordinates on an image, and the second type object distance estimation table 520 is a table for determining the object distance based on a divided space on an image.

[0048] (1) First Type Object Distance Estimation Table 510 The first type object distance estimation table 510 may estimate distance based on concentric circle-based coordinates. Concentric circles having different diameters may be arranged according to the position (height) of the vanishing point 560 on the image. In this case, the diameter of the concentric circle may be relatively larger as it is relatively closer to the vehicle. The arranged concentric circles may become denser as it is farther from the vehicle, and the density of the concentric circles may be determined according to the vanishing point. The distance to an object may be determined based on the concentric circles and the coordinates of each position of the concentric circles.

[0049] (2) Second Type Object Distance Estimation Table 520 When the lane width and vanishing point are determined, a second type object distance estimation table 520 may be generated to divide a space based on the vanishing point on the image to estimate the distance of the object. The space used to determine the distance may be expressed in terms of distance determination space 570.

[0050] In the case of a relatively close distance, a distance determination space 570 having a relatively large area may be estimated to be the same distance, and in the case of a relatively far distance, a distance determination space having a relatively small area may be estimated to be the same distance. The area of ​​the distance determination space 570 may be determined based on the vanishing point 560.

[0051] In addition, the closer the distance is to the center line 550 determined based on the center of the bottom end point of the vehicle image and the vehicle's traveling direction, the larger the area of ​​the distance determination space 570 is estimated to be the same distance, and in the case of a relatively far distance, the smaller the area of ​​the distance determination space 570 is estimated to be the same distance.

[0052] Based on this, the image may be divided into distance determination spaces 570 of different areas, and a second type of object distance estimation table 520 may be generated in a manner in which information about distance is matched to the distance determination spaces corresponding to the different areas.

[0053] According to an embodiment of the present invention, the first type object distance estimation table 510 and the second type object distance estimation table 520 can be used selectively or in combination.

[0054] For example, at a short distance based on the vehicle, since the object is relatively large, the distance can be estimated based on second type object distance estimation table 520, and at a long distance based on the vehicle, since the object is relatively small, the distance can be estimated based on first type object distance estimation table 510. The range in which first type object distance estimation table 510 and second type object distance estimation table 520 are used can be determined based on the height of vanishing point 560.

[0055] As another example, when the number of objects is relatively large based on the density of objects on an image, equal to or greater than a critical density, the second type of object distance estimation table 520 can be used, and when the number of objects is relatively small based on the critical density, the first type of object distance estimation table 510 can be used. When the density of objects is high, if the first type of object estimation table 510 is used, it may be difficult to estimate the distance to a rear object due to interference from a front object. Therefore, the second type of object distance estimation table 520 can be used for approximate distance measurement based on space.

[0056] FIG. 6 is a conceptual diagram showing a lane departure detection method according to an embodiment of the present invention.

[0057] FIG. 6 discloses a method for determining whether or not a lane departure has occurred in the lane departure determining unit.

[0058] 6, the lane departure determination unit can determine whether or not a vehicle is deviating from a lane based on information about the driving lane in the nearby area and the center of the image. As described above, the lane width and the lane center can be determined based on the driving lane in the nearby area. If the image center 610 is outside the deviation range 630 set based on the lane center 620, the lane departure determination unit can determine that the vehicle is deviating from the lane.

[0059] In the case of a vehicle, the vehicle may change lanes in order to change lanes, and in such a case, the lane change may be judged as normal and not as lane departure.

[0060] If the center of the lane deviates from the deviation range 630 and the right lane is recognized as the left lane within the critical time, or the left lane is recognized as the right lane while the image center 610 is within the deviation range 630 based on the lane center 620, the lane deviation determination unit may determine this as a normal lane change and not determine the lane deviation.

[0061] In the lane departure unit, if the image center 610 is outside the departure range 630 based on the lane center 620 and the image center 610 is still outside the departure range 630 based on the lane center 620 even after a critical time has elapsed, the lane departure determination unit can determine that the vehicle has deviated from the lane.

[0062] FIG. 7 is a conceptual diagram illustrating an interruption detection method according to an embodiment of the present invention.

[0063] FIG. 7 discloses a method for determining an interruption by the interruption determination unit.

[0064] 7, the cut-in determination unit may perform a determination as to whether another vehicle cuts in and a determination as to whether the vehicle itself cuts in. The determination as to whether another vehicle cuts in may be performed based on whether the vehicle enters the driving lane within a critical distance based on the object distance estimation unit's object distance estimation.

[0065] The determination as to whether the vehicle itself has cut in may be made based on whether the vehicle has departed from the lane based on the lane departure of the lane departure determination unit and the object distance estimation of the object distance estimation unit, and based on whether another vehicle is present within a critical distance.

[0066] (1) Judgment for other vehicles to cut in In order to determine whether another vehicle has cut in, an object is recognized within a critical distance and a critical time interval based on the object distance estimation table described above. That is, when an object is recognized within a first critical distance and a first critical time interval during distance recognition of the object based on the object distance table, it can be determined that another vehicle has cut in.

[0067] According to an embodiment of the present invention, the critical distance is when the object is located within a specific distance determination space based on the second type object distance estimation table. The first critical distance can be adaptively determined by selecting the specific distance determination space that is determined to be an interruption.

[0068] (2) Judgment on whether your vehicle is cutting in front of you The determination of whether the vehicle itself will cut in may be based on whether another vehicle is present within a second critical distance after the vehicle deviates from the lane based on the lane departure of the lane departure determination unit and the object distance estimation of the object distance estimation unit.

[0069] Similarly, according to an embodiment of the present invention, the second critical distance is when the object is located within a specific distance determination space based on a second type of object distance estimation table. The second critical distance can be adaptively determined by selecting the specific distance determination space that is determined to be an interruption.

[0070] The above-described embodiments of the present invention may be embodied in the form of program instructions that can be executed by various computer components and recorded on a computer-readable recording medium. The computer-readable recording medium may include program instructions, data files, data structures, and the like, alone or in combination. The program instructions recorded on the computer-readable recording medium may be specially designed and constructed for the present invention, or may be available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and hardware devices specially configured to store and execute program instructions, such as ROMs, RAMs, flash memories, and the like. Examples of program instructions include not only machine language codes such as those produced by a compiler, but also high-level language codes that can be executed by a computer using an interpreter, and the like. The hardware devices may be replaced by one or more software modules to perform the processes of the present invention, and vice versa.

[0071] Although the present invention has been described above using specific details such as specific components and limited examples and drawings, this is merely provided to assist in a more general understanding of the present invention, and the present invention is not limited to the above examples. Those skilled in the art to which the present invention pertains may attempt various modifications and changes from such descriptions.

[0072] Therefore, the concept of the present invention should not be limited to the above described embodiments, and all scopes equivalent to or modified equivalently to the scope of the claims below as well as the claims themselves are within the scope of the concept of the present invention.

Claims

1. Driver assistance methods based on drive recorder footage include: a driver assistance device detecting a lane; The driver assistance device estimates an object distance of an object located in an image based on the lane; The driver assistance device determines lane departure; the driver assistance device determining whether or not the vehicle is cutting in; Including, The object distance is: A lane width obtained by connecting the lower end point of the adjacent lane on the image, which is a lane in the critical area close to the vehicle among the lanes in which the vehicle is currently traveling, and the point where the lane meets the vehicle; A vanishing point is a point where the adjacent area traffic lanes on the image meet when extended in a straight line; an object distance estimation table including information on estimated distances by position in the image according to the lane width and the position of the vanishing point; Determined based on A driver assistance method comprising:

2. A driver assistance device that assists a driver based on drive recorder footage is Detect lanes, Estimating an object distance of an object located within the image based on the lane; Detect lane departure, Implemented to determine interrupts, The object distance is: A lane width obtained by connecting the lower end point of the adjacent lane on the image, which is a lane in the critical area close to the vehicle among the lanes in which the vehicle is currently traveling, and the point where the lane meets the vehicle; A vanishing point is a point where the adjacent area traffic lanes on the image meet when extended in a straight line; an object distance estimation table including information on estimated distances by position in the image according to the lane width and the position of the vanishing point; A driver assistance device characterized in that the driver assistance device is determined based on the following:

Images