Lane display device and method thereof

Abstract

The invention provides a lane display device and a method thereof. The lane display device includes: an image capturing device that captures an image of the front of a vehicle including a road surface; a lane marking detection unit that detects a lane marking formed on the road surface from the image captured by the image capturing device; the display frame is used for displaying lane markings; a steering angle detection unit for detecting a steering angle of the vehicle; a travel route calculation unit that calculates a predicted travel route on the basis of the steering angle; an offset amount calculation unit that calculates an offset amount between the lane center line and the predicted travel route; and a display path determination unit that corrects the display of the lane markings on the basis of the offset amount when the offset amount exceeds a first offset range.

Description

Technical Field

[0001] The invention relates to a lane display device and a method thereof. Background Art

[0002] Conventionally, regarding route information, there are known two methods of displaying a travel route based on camera detection and a method of displaying a travel route based on a steering (steering) angle of a steering wheel.

[0003] Patent document 1 discloses a technology that when the vehicle is judged to be traveling on a curve according to the steering angle of the steering device, the displayed image is offset in the direction of the curve compared to the displayed image when the vehicle is traveling in a straight line.

[0004] Prior art literature

[0005] Patent document 1: Japanese Patent Publication No. 2004-155307

[0006] However, the driving route detected by the camera may be displayed in a biased manner due to variations in image processing. Also, the driving route based on the steering angle can obtain more accurate data than the camera, but it cannot reflect the information of the curve ahead.

[0007] However, accurately displaying lane markings and increasing their recognizability remain challenges. Summary of the invention

[0008] This project aims to solve the above-mentioned problems and improve the visibility of lane markings. In addition, it will help to support the good connection between the city and the local areas, including the city periphery.

[0009] According to an embodiment of the present invention, a lane display device is proposed, which includes: a camera device for photographing the front of a vehicle including a road surface; a lane marking detection unit for detecting lane markings formed on the road surface from an image photographed by the camera device; a display screen for displaying the lane markings; a steering angle detection unit for detecting the steering angle of the vehicle; a route calculation unit for calculating a predicted route based on the steering angle; an offset calculation unit for calculating an offset between a lane centerline and the predicted route; and a display path determination unit for correcting the display of the lane markings based on the offset when the offset exceeds a first offset range.

[0010] According to an embodiment of the present invention, in the lane display device, when the offset is within the first offset range, the display path determination unit does not correct the lane marking.

[0011] According to an embodiment of the present invention, in the lane display device, when the lane marking detection unit does not detect the lane marking, the display path determination unit does not display the lane marking.

[0012] According to an embodiment of the present invention, in the lane display device, when the offset exceeds a second offset range, the display path determination unit shortens the length of the path to be displayed of the lane marking compared to when the offset is within the second offset range.

[0013] According to an embodiment of the present invention, the lane display device further comprises: a steering speed detection unit for detecting the steering speed of the vehicle. When the steering speed is above a threshold, the display path determination unit does not correct the display of the lane markings.

[0014] According to an embodiment of the present invention, the lane display device further comprises: a vehicle speed detection unit for detecting the vehicle speed; and a lateral acceleration detection unit for detecting the lateral acceleration generated by the vehicle. When the vehicle speed and the lateral acceleration are respectively above respective predetermined thresholds, the display path determination unit does not correct the display of the lane markings.

[0015] According to another embodiment of the present invention, a lane display method is proposed, which includes: photographing the front of a vehicle including a road surface; detecting lane markings formed on the road surface from the photographed image; displaying the lane markings on a display screen; detecting a steering angle of the vehicle and calculating a predicted route based on the steering angle; calculating an offset between a lane centerline and the predicted route; and when the offset exceeds a first offset range, correcting the display of the lane markings based on the offset.

[0016] According to an embodiment of the present invention, in the lane display method, when the offset is within the first offset range, the lane marking is not corrected.

[0017] According to an embodiment of the present invention, in the lane display method, when the lane marking is not detected, the lane marking is not displayed.

[0018] According to an embodiment of the present invention, in the lane display method, when the offset exceeds a second offset range, the length of the path to be displayed of the lane marking is shortened compared to when the offset is within the second offset range.

[0019] According to the above embodiment, the deviation between the lane center line based on the image captured by the camera and the predicted travel route based on the steering is calculated, and the display of the lane markings is corrected according to the degree of the deviation. As a result, the displayed lane markings can be more accurate and the commercial quality is also improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 is a block diagram of a lane display device according to an embodiment of the present invention.

[0021] Figure 2 1 is an example of lane markings displayed by a lane display device according to an embodiment of the present invention.

[0022] Figure 3 2 is a schematic diagram showing a lane marking correction method according to an embodiment of the present invention.

[0023] Figure 4 is another example of lane markings displayed by the lane display device according to an embodiment of the present invention.

[0024] Figure 5 FIG. 4 is a schematic diagram showing another correction method of lane markings according to an embodiment of the present invention.

[0025] Fig. 6A is another example of lane markings displayed by the lane display device according to an embodiment of the present invention.

[0026] Figure 6B An example image showing lane markings before and after correction.

[0027] Figure 7 A diagram showing the relationship between steering speed and correction area.

[0028] Figure 8 Displays a diagram showing the relationship between vehicle speed / lateral acceleration and the correction area.

[0029] Description of Reference Numerals

[0030] 100: Lane display device

[0031] 102: Camera device

[0032] 104: Lane marking detection unit

[0033] 106: Display screen

[0034] 108: Steering angle detection unit

[0035] 116: Steering speed detection unit

[0036] 118: Route calculation unit

[0037] 120: Offset calculation unit

[0038] 122: Display path determination unit

[0039] 130: Steering device

[0040] 140: Control Department

[0041] 142: Vehicle speed detection unit

[0042] 144: Lateral acceleration detection unit

[0043] WS: Windshield

[0044] LM1, LM2: Lane markings

[0045] L1, L2: To display the path

[0046] P: Predicted path

[0047] C: Lane centerline

[0048] R:Lane (road surface)

[0049] D: Offset DETAILED DESCRIPTION

[0050] Reference will now be made in detail to exemplary embodiments of the present invention, examples of which are illustrated in the accompanying drawings. Whenever possible, the same reference numerals are used in the drawings and the description to refer to the same or like parts.

[0051] Figure 1 is a block diagram of a lane display device according to an embodiment of the present invention. Figure 2 is an example of a lane marking displayed by a lane display device according to an embodiment of the present invention.

[0052] like Figure 1 As shown, the lane display device 100 includes at least but is not limited to: a camera 102, a lane marking detection unit 104, a display screen 106, a steering angle detection unit 108, a travel route calculation unit 118, an offset calculation unit 120, and a display path determination display unit 122. The lane display device 100 may further include a control unit 140. The control unit 140 may control the above-mentioned units or operate in coordination with them.

[0053] The control unit 140 can independently control the lane display device 100. The control unit 140 can also be implemented by a processor as a part of the entire vehicle control device, such as the ECU (Electronic Control Unit) unit of the vehicle. The ECU unit can be used to control various sensors of the vehicle. Therefore, the control unit 140 can receive data, and perform various processing and judgments on the data, and then control various actuating components of the vehicle. Various system controls in the vehicle are controlled by the ECU unit. The control unit 140 can be used to control the above-mentioned parts. In addition, when the control unit 140 is used as an ECU unit, it can also be used to control the vehicle, such as controlling the driving support unit to control the driving support function. Therefore, the control unit 140 can control various behavior modes of the vehicle, such as acceleration, deceleration, parking, avoidance, etc., and can also detect the vehicle state or the state of the surrounding area of ​​the vehicle by controlling various sensors and detectors. For those skilled in the art, various control methods of the control unit 140 can be designed according to actual needs.

[0054] Figure 2 is an example of lane markings displayed by a lane display device according to an embodiment of the present invention. That is, Figure 2 In the example, the display screen 106 of the lane display device 100 is projected onto the windshield WS. The display screen 106 may also be an independent display screen different from the windshield WS.

[0055] like Figure 1 and Figure 2 As shown, the camera device 102 is used to shoot the front of the vehicle including the road surface. The camera device 102 is, for example, a vehicle-mounted camera, which can be arranged inside the vehicle and shoot the front of the vehicle, or can be arranged elsewhere outside the vehicle and shoot the front of the vehicle. The front of the vehicle including the road surface photographed by the camera device 102 includes, for example, the road surface in front of the vehicle (including at least one lane), lane markings, and landscape. The camera device 102 can be a camera of any form and structure, and the present invention is not particularly limited.

[0056] The lane marking detection unit 104 can detect the lane markings LM1 and LM2 formed on the road surface (lane) R from the image captured by the camera device 102. As an example, the lane marking detection unit 104 can be operated independently or integrated with the control unit 140 to operate in coordination. For example, the lane markings LM1 and LM2 in the image captured by the camera device 102 are detected by a specific image processing and calculation method.

[0057] The display screen 106 is used to display the lane markings LM1 and LM2. That is, the detected lane markings LM1 and LM2 are displayed on the display screen 106 so that the driver can clearly know where the lane markings LM1 and LM2 are located. Here, the display screen 106 is, for example, Figure 2 As shown, the display screen of the head-up display (HUD) can be projected onto the windshield WS of the vehicle. The display screen 106 can be controlled by the control unit 140 for operations related to the display.

[0058] The steering angle detection unit 108 is used to detect the steering angle of the vehicle. The steering angle detection unit 108 can be implemented, for example, by using a sensor installed on the steering device 130 and the control unit 140. This sensor is used to detect the operating angle (i.e., the steering angle) of the steering device 130 relative to a reference position to determine the degree of turning of the vehicle. The steering device 130 generally refers to the steering wheel of the vehicle. The reference position can generally be the position where the steering device 130 is located at the steering wheel of the vehicle (e.g., the left and right wheels in front) to keep the vehicle moving straight. The steering angle of the vehicle can be obtained by the operating angle (i.e., the steering angle) of the steering device 130 detected by the sensor, or by processing and analyzing the signal by the control unit 140.

[0059] The path calculation unit 118 can calculate the predicted path P of the vehicle based on the steering angle detected by the steering angle detection unit 108. The path calculation unit 118 can calculate the predicted path P independently or in combination with the control unit 140 through a predetermined calculation method.

[0060] The offset calculation unit 120 is used to calculate the offset D between the lane center line C and the predicted travel route P. The offset calculation unit 120 can be independent or integrated with the control unit 140 to calculate the offset D through a predetermined calculation method. In addition, the lane center line C is not displayed on the display screen 106. The lane center line C is only an indicator (reference line) for calculating the offset D. Generally speaking, the lane center line C can be set to the center line of the lane markings LM1 and LM2 on both sides of the lane R where the vehicle is traveling.

[0061] The display path determination unit 122 is used to determine the display of the lane marking lines L1 and L2 after correction. When the offset D calculated by the offset calculation unit 120 exceeds the first offset range, the display path determination unit 122 corrects the lane marking lines LM1 and LM2 based on the offset D to determine the display of the corrected path (i.e., the lane marking lines L1 and L2 after correction).

[0062] Figure 3 FIG. 2 is a schematic diagram showing a lane marking correction method according to an embodiment of the present invention. Figure 3As shown, it shows a schematic diagram of the offset D calculated by the above-mentioned offset calculation unit 120 and the correction offset for correcting the lane markings LM1 and LM2 for this offset D. Here, for example, a first range (such as -a < D < a) can be set. This first range can be set by the vehicle manufacturer, for example, or the driver can set it through a human-machine interface (HMI).

[0063] When the offset D calculated by the above-mentioned offset calculation unit 120 exceeds the first offset range, the display path determination unit 122 can Figure 3 According to the corresponding relationship shown, calculate or find out the corresponding correction offset, such as +i, +j, -i, -j, etc. Then, after correcting the display of the lane markings LM1 and LM2 with this correction offset, it is displayed on the display screen 106. Here, the relationship between the correction offset and the offset presents an increasing step function relationship, but other functional correspondences such as curves can also be used, and the present invention is not particularly limited.

[0064] In addition, when the offset D calculated by the above-mentioned offset calculation unit 120 does not exceed the first offset range, that is, as Figure 3 In the example shown, when the offset D is between a and -a, the display path determination unit 122 will not correct the lane markings LM1 and LM2 based on the offset D.

[0065] According to an embodiment of the present invention, it calculates the offset D between the center of the lane marking (lane center line C) based on the image captured by the imaging device (camera) 102 and the predicted travel route P based on the steering operation, and corrects the display of the lane markings LM1 and LM2 according to the degree of this offset D. Thus, the displayed lane markings LM1 and LM2 can be more accurate, and the marketability can also be improved. On the contrary, when the offset D does not exceed the first range, it means that the degree of display offset is not large, and there is no need to particularly correct the display of the lane markings LM1 and LM2, so that the computational burden on the display path determination unit 122 (and the control unit 140) can be reduced.

[0066] In addition, according to another embodiment of the present invention, as Figure 4 shown, when the lane marking detection unit does not detect the lane markings, the display path determination unit 122 does not display the lane markings. On actual roads, there are some areas where there are no lane markings. Also, in bad weather conditions or when the visibility is poor, the lane markings may not be easily recognizable. At this time, according to the different steering angles, the predicted travel route P may be displayed as protruding into the oncoming lane, causing misidentification by the driver. Therefore, in this case, the display path determination unit 122 can decide not to display the lane markings.

[0067] Therefore, when the vehicle is traveling on a road without lane markings or when the lane marking detection unit 104 cannot detect lane markings due to poor visibility in bad weather, the display path determination unit 122 can determine not to display the lane markings. Thereby, incorrect identification of lanes can be avoided and the marketability can be improved.

[0068] Furthermore, according to another embodiment of the present invention, as Figure 5 , Fig. 6A and Figure 6B shown, when the offset D between the center of the lane marking (lane center line C) based on the image captured by the imaging device (camera) 102 and the predicted travel route P based on the steering operation exceeds the second offset range (such as Figure 5 ), compared with the case where the offset D is within the second offset range, the display path determination unit 122 shortens the length of the display path of the lane markings.

[0069] Figure 5 is a schematic diagram showing another correction method of the lane markings illustrated according to an embodiment of the present invention. Furthermore, Figure 5 is an explanatory example showing that the deviation value is greater than the second deviation range, so the lane center line C in Figure 2 is omitted; however, when actually calculating the deviation value D, the lane center line C in the above-mentioned Figure 2 is still used. As Figure 5 shown, it is a schematic diagram showing the offset D calculated by the above-mentioned offset calculation unit 120 and the correction offset for correcting the lane markings LM1 and LM2 for this offset D. Here, for example, the second range (such as -b < D < b) can be set. This second range can be set by the vehicle manufacturer, for example, or the driver can set it through the human-machine interface (HMI). Furthermore, the second range can be basically set to a range that does not overlap with the actual lane markings.

[0070] When the offset D calculated by the above-mentioned offset calculation unit 120 is between the first and second offset ranges, the display path determination unit 122 can calculate or find the corresponding correction offset according to the Figure 5 shown correspondence relationship, such as +i, +j, -i, -j, etc. Then, after correcting the display of the lane markings LM1 and LM2 with this correction offset, it is displayed on the display screen 106. Similarly, the relationship between the correction offset and the offset also presents an increasing step function relationship, but other function correspondence relationships such as curves can also be adopted, and the present invention is not particularly limited.

[0071] As Fig. 6AAs shown in FIG. 1 , when the offset D does not have the second offset range, the display of the lane markings LM1 and LM2 is corrected, and the paths L1 and L2 to be displayed are displayed, as shown in FIG. Figure 6B However, when the offset D exceeds the second offset range, the corrected path L2 to be displayed may overlap with the lane marking LM2 of the real scene, such as Fig. 6A The display may be shown as the circled marking area A, and may become a display that protrudes into the opposite lane. In this case, such a display may confuse the driver. Therefore, when the offset D exceeds the second offset range, the display path determination unit 122 shortens the length of the corrected display path L1, L2 so that it does not overlap with the lane markings LM1, LM2 of the real scene. Figure 6B The correction display on the right side. As an example, the shortening ratio can be half (50%) of the normal display. In this way, the paths L1 and L2 to be displayed will not overlap with the lane marking LM2 of the real scene.

[0072] According to this embodiment, if the offset is large (exceeding the second offset range), the driver may be confused because the displayed lane marking (displayed path) overlaps with the actual lane marking. Therefore, when the offset exceeds the second offset range, the driver's sense of discomfort can be reduced and commercial quality can be improved by displaying a shorter display (reduced by about 50%) compared to the case where the offset is within the second offset range.

[0073] According to another embodiment of the present invention, the lane display device 100 may further include: a steering speed detection unit 116. The steering speed detection unit 116 is used to detect the steering speed of the steering device 130 of the vehicle operated by the driver. As an example, the steering angle of the steering device 130 can be detected by the steering angle detection unit 108, and the steering speed detection unit 116 can calculate the rate of change of the steering angle per unit time, and then detect the steering speed. As an example, the steering speed detection unit 116 can operate independently or be integrated with the control unit 140 to operate in coordination.

[0074] Figure 7 A schematic diagram showing the steering speed and the correction area. When the steering speed detected by the steering speed detection unit 116 is above the threshold value X, the display path determination unit 122 does not correct the display of the lane markings. At this time, the display screen 106 only displays the lane markings LM1, LM2, etc. On the contrary, when the steering speed detected by the steering speed detection unit 116 is below the threshold value X, the display path determination unit 122 corrects the display of the lane markings LM1, LM2. At this time, the display screen 106 displays the lane markings after correction (i.e., the path to be displayed).

[0075] If the predicted route based on the steering angle when the steering device 130 is operated sharply, the predicted route may deviate greatly from the lane markings. Therefore, the present invention sets a threshold value for the steering speed. When the steering speed exceeds the threshold value, the steering speed detection unit 116 (or together with the control unit 140) can determine that the steering device 130 is being operated sharply. At this time, the display path determination unit 122 can decide to display only the lane markings in the display path without correcting the lane markings. In this way, excessive correction of the lane markings can be prevented, and commercial quality can be improved.

[0076] According to another embodiment of the present invention, the lane display device 100 may further include: a vehicle speed detection unit 142 and a lateral acceleration detection unit 144. This embodiment is an explanation of whether to correct the lane marking display in another situation where the predicted route may deviate greatly from the lane marking.

[0077] The vehicle speed detection unit 142 is used to detect the speed of the vehicle. The vehicle speed detection unit 142 can be implemented, for example, by a vehicle speed sensor and a control unit 140. The vehicle speed sensor is, for example, a sensor disposed near the wheel to detect the rotation speed to obtain the vehicle speed. The vehicle speed can also be detected using various available sensors, such as using a camera to perform image analysis and processing, radar, lidar, and other available technical means, which are not particularly limited by the present invention. When the vehicle speed detection unit 142 detects the vehicle speed, the control unit 140 can compare the detected vehicle speed with a specified vehicle speed threshold, and determine whether the vehicle speed is equal to or less than the specified vehicle speed threshold.

[0078] The lateral acceleration detection unit 144 is used to detect the lateral acceleration generated in the vehicle. The lateral acceleration of the vehicle refers to the acceleration in the direction perpendicular to the direction of travel of the vehicle, which is the acceleration caused by the centrifugal force generated when the vehicle turns. In other words, the vehicle tends to be "thrown away". The greater the lateral acceleration, the easier it is for the vehicle to be "thrown away" from the travel path. Generally speaking, the lateral acceleration can be detected by a lateral acceleration sensor installed in the vehicle. The lateral acceleration detected by the lateral acceleration detection unit 144 can be transmitted to the display path determination unit 122.

[0079] Figure 8 Displays a schematic diagram of the vehicle speed / lateral acceleration and the correction area. Figure 8 As shown, in this embodiment, when the vehicle speed and the lateral acceleration are respectively above their respective prescribed thresholds (prescribed vehicle speed threshold Z and prescribed lateral acceleration threshold Y), that is, Figure 8 In the shaded area shown, the display path determination unit 122 does not correct the display of the lane markings. In this case, the display screen 106 only displays the uncorrected lane markings.

[0080] In the case of high vehicle speed and high lateral acceleration, the possibility of the vehicle traveling on a sharp curve is very high. At this time, the predicted route based on the steering angle when the steering device 130 is sharply operated may cause the predicted route to deviate greatly from the lane markings. Therefore, the present invention sets respective thresholds for high vehicle speed and lateral acceleration. When the vehicle speed exceeds the prescribed vehicle speed threshold Z and the lateral acceleration exceeds the prescribed lateral acceleration threshold Y, the control unit 140 can determine that the steering device 130 is in an intense operation. At this time, the display path determination unit 122 can decide to display only the lane markings on the display path without correcting the lane markings. In this way, excessive correction of the lane markings can be prevented and commercial quality can be improved.

[0081] According to the above embodiment, the offset between the lane center line based on the image captured by the camera and the predicted route based on the steering is calculated, and the display of the lane markings is corrected according to the degree of the offset. As a result, the displayed lane markings can be more accurate and the commercial quality is improved. In addition, the present invention can also not correct the display of the lane markings when the vehicle operates the steering device violently, thereby preventing excessive correction of the lane markings.

[0082] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A lane display device, characterized in that: include: A camera device for photographing the front of the vehicle including the road surface; a lane marking detection unit for detecting lane markings formed on the road surface from the image captured by the camera device; A display screen displays the lane markings; A steering angle detection unit, used to detect the steering angle of the vehicle; a travel route calculation unit, which calculates a predicted travel route based on the steering angle; An offset calculation unit calculates an offset between a lane centerline and the predicted travel route; and The display path determination unit corrects the display of the lane marking based on the offset when the offset exceeds a first offset range.

2. The lane display device according to claim 1, characterized in that: When the offset is within the first offset range, the display path determination unit does not correct the display of the lane marking.

3. The lane display device according to claim 1 or 2, characterized in that: When the lane marking detection unit does not detect the lane marking, the display path determination unit does not display the lane marking.

4. The lane display device according to claim 1 or 2, characterized in that: When the offset exceeds a second offset range, the display path determination unit shortens the length of the path to be displayed of the lane marking, compared to a case where the offset is within the second offset range.

5. The lane display device according to claim 1, characterized in that: Also includes: A steering speed detection unit is used to detect the steering speed of the vehicle. When the steering speed is equal to or greater than a threshold value, the display path determination unit does not correct the display of the lane marking.

6. The lane display device according to claim 1, characterized in that: Also includes: A vehicle speed detection unit, used to detect the speed of the vehicle; and A lateral acceleration detection unit is used to detect the lateral acceleration generated in the vehicle. When the vehicle speed and the lateral acceleration are respectively greater than respective predetermined thresholds, the display path determination unit does not correct the display of the lane markings.

7. A lane display method, characterized in that: include: Taking a picture of the front of the vehicle including the road surface; detecting lane markings formed on the road surface from the captured image; Displaying the lane marking on a display screen; detecting a steering angle of the vehicle, and calculating a predicted travel route based on the steering angle; Calculating an offset between a lane centerline and the predicted travel path; and When the offset exceeds a first offset range, the display of the lane marking is corrected based on the offset.

8. The lane display method according to claim 7, characterized in that: When the offset is within the first offset range, the lane marking is not corrected.

9. The lane display method according to claim 7 or 8, characterized in that: When the lane marking is not detected, the lane marking is not displayed.

10. The lane display method according to claim 7 or 8, characterized in that: When the offset exceeds the second offset range, the length of the path to be displayed of the lane marking is shortened compared to a case where the offset is within the second offset range.

Images