Device and method for displaying virtual lanes during platoon driving

Abstract

The present invention relates to an apparatus and method for displaying a virtual lane during platoon driving. The apparatus may include: a processor that generates a virtual lane by converting lane information of a preceding vehicle into lane information observed from the perspective of the host vehicle during platoon driving; and a display that displays the virtual lane.

Description

[0001] Cross - reference to related applications

[0002] This invention claims the priority and benefits of Korean Patent Application No. 10 - 2019 - 0149939, filed on November 20, 2019, the entire content of which is incorporated herein by reference. Technical field

[0003] The present invention relates to an apparatus and method for displaying a virtual lane during platoon driving, and more particularly, to a technology for displaying a virtual lane using lane information of a leading vehicle. Background art

[0004] The statements in this section merely provide background information related to the present invention and do not constitute prior art.

[0005] Platoon driving is a technology that enables multiple vehicles arranged in a column and spaced apart from each other by a specific distance to perform autonomous driving. During platoon driving, the leading vehicle located at the front of the platoon can control at least one following vehicle that follows the leading vehicle. The leading vehicle can maintain the distance between the multiple vehicles included in the platoon and can exchange the behavior and condition information of the multiple vehicles included in the platoon through vehicle - to - vehicle communication.

[0006] The inter - vehicle distance between the vehicles in the platoon can be maintained very short, so as to improve fuel efficiency. The inter - vehicle distance (in the longitudinal direction) can be controlled to be automatically maintained by a platoon driving controller.

[0007] However, the lateral steering of a vehicle can be designed to be directly controlled by a driver according to the components of sensors and steering devices.

[0008] In this case, we found that although the driver has to perform lateral steering when the inter - vehicle distance is short, a large leading vehicle (LV) (such as Figure 1 a truck traveling together at a close distance (about 5m to 15m) as shown) may block the view of the FV driver, thus imposing a relatively large lateral steering burden on the driver. Summary of the invention

[0009] The present invention provides an apparatus and method for displaying a virtual lane during platoon driving, which can provide virtual lane information (obtained by converting the lane information measured by the leading vehicle into lane information observed from the perspective of the following vehicle) to the driver of the following vehicle, so that even if the view of the driver of the following vehicle is blocked, the driver of the following vehicle can safely drive the following vehicle based on the virtual lane information.

[0010] The technical problems solved by the present invention are not limited to the above problems, and those skilled in the art to which the present invention pertains will clearly understand any other technical problems not mentioned herein from the following description.

[0011] According to one aspect of the present invention, a device for displaying a virtual lane may include a processor that generates a virtual lane by converting the lane information of a preceding vehicle into lane information observed from the perspective of the host vehicle during platoon driving; and a display that displays the virtual lane.

[0012] According to an embodiment of the present invention, the processor may determine whether a failure occurs during platoon driving based on at least one of a communication state, a front sensor state, or an inter-vehicle distance state.

[0013] According to some embodiments of the present invention, the processor may convert the converted lane information observed from the perspective of the host vehicle into lane information observed from the perspective of an augmented reality image.

[0014] According to some embodiments of the present invention, the processor may convert the lane information measured based on the camera coordinate system of the preceding vehicle into information based on the world coordinate system.

[0015] According to some embodiments of the present invention, the processor may rotate the lane information of the preceding vehicle based on the traveling angle of the preceding vehicle and the traveling angle of the host vehicle.

[0016] According to some embodiments of the present invention, the processor may shift the rotated lane information by the lateral distance and the longitudinal distance from the origin of the host vehicle to the origin of the preceding vehicle.

[0017] According to some embodiments of the present invention, the processor may determine whether the route of the host vehicle can be generated based on the measurement state of at least one of the vehicle speed, yaw rate, or steering angle of the host vehicle.

[0018] According to some embodiments of the present invention, the processor may generate the route of the host vehicle based on at least one of the vehicle speed, yaw rate, or steering angle.

[0019] According to some embodiments of the present invention, the processor may perform a control operation to display the route of the host vehicle on the virtual lane, which is generated by conversion from the perspective of the host vehicle.

[0020] According to some embodiments of the present invention, when the vehicle speed is less than a predetermined value, the processor may calculate the radius of the moving route of the host vehicle based on the steering ratio, wheelbase, or steering angle of the host vehicle.

[0021] According to some embodiments of the present invention, when the vehicle speed is greater than or equal to the predetermined value, the processor may calculate the radius of the moving route of the host vehicle using the vehicle speed and yaw rate of the host vehicle.

[0022] According to some embodiments of the present invention, the processor may generate the route of the host vehicle based on the origin of the host vehicle and by using the radius of the vehicle movement route.

[0023] According to some embodiments of the present invention, the processor may display the route of the host vehicle by converting the route of the host vehicle into a route observed from the perspective of an augmented reality image.

[0024] According to some embodiments of the present invention, the processor may determine a lane departure time point by using the lane information of the preceding vehicle and the route of the host vehicle; and when the lane departure time point is less than a predetermined value, provide a lane departure warning.

[0025] According to some embodiments of the present invention, when the distance between at least one point among a plurality of points of the virtual lane and the center point of the route of the host vehicle is less than or equal to a predetermined distance, the processor may determine that a lane departure time point can be determined.

[0026] According to some embodiments of the present invention, among one or more points of the virtual lane that are less than or equal to a predetermined distance from the center point of the route of the host vehicle, the processor may determine the point with the maximum longitudinal distance from the host vehicle as the lane departure prediction point.

[0027] According to some embodiments of the present invention, the processor may calculate a lane departure prediction time by using the vehicle speed of the host vehicle and the distance from the current point of the host vehicle to the lane departure prediction point; and when the lane departure prediction time is less than a predetermined value, provide a lane departure warning.

[0028] According to some embodiments of the present invention, the processor may display a warning message of the lane departure warning in a pop-up form on the virtual lane; and display the departure lane including the lane departure prediction point of the virtual lane differently in terms of color or thickness, or blink-display the departure lane.

[0029] According to some embodiments of the present invention, the display may display the virtual lane, the movement route of the host vehicle, and the lane departure warning in the form of an augmented reality image on the windshield of the host vehicle.

[0030] According to another aspect of the present invention, a method for displaying a virtual lane may include: when driving in a queue, generating a virtual lane by converting the lane information of the preceding vehicle into lane information observed from the perspective of the host vehicle; displaying the virtual lane obtained by the conversion from the perspective of the host vehicle.

[0031] Through the description provided herein, other application fields will become apparent. It should be understood that this specification and specific examples are only for illustrative purposes and are not intended to limit the scope of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] To better understand the present invention, various embodiments of the present invention given by way of example will now be described with reference to the accompanying drawings, in which:

[0033] Figure 1 is a schematic view showing the field of view of the driver of a following vehicle in typical platoon driving;

[0034] Figure 2 is a block diagram showing the configuration of a device for displaying a virtual lane;

[0035] Figure 3A and 3B are schematic views respectively showing coordinate systems for perspective conversion;

[0036] Figure 3C is a schematic view showing a lane displayed in a coordinate system;

[0037] Figure 4A 、 Figure 4B and Figure 4C are schematic views respectively showing the conversion of lane information from the perspective of the leading vehicle to the lane information observed from the perspective of the following vehicle by using the travel angle;

[0038] Figure 5A and Figure 5B are schematic views respectively showing the conversion of lane information from the perspective of the leading vehicle to the lane information observed from the perspective of the following vehicle by the movement of the longitudinal / lateral distance from the leading vehicle;

[0039] Figure 6A 、 Figure 6B and Figure 6C are schematic views respectively showing the conversion of lane information from the perspective of the leading vehicle to the lane information of the following vehicle;

[0040] Figure 7A and Figure 7B are schematic views respectively showing the display of the front lane on the windshield;

[0041] Figure 8A and Figure 8B are schematic views respectively showing methods for generating the route of the host vehicle and determining the lane departure point;

[0042] Figure 9A 、 Figure 9B and Figure 9C are schematic views respectively showing the display of the route of the host vehicle and lane departure warnings;

[0043] Figure 10 is a flowchart showing a method for displaying a virtual lane; and

[0044] Figure 11A computing system is shown.

[0045] The accompanying drawings described herein are for illustrative purposes only and are not intended to limit the scope of the present invention in any way. Detailed Description

[0046] The following description is merely exemplary in nature and is not intended to limit the present invention, its application, or uses. It should be understood that throughout the drawings, corresponding reference numerals indicate like or corresponding parts and features.

[0047] Some embodiments of the present invention will be described in detail below with reference to the exemplary drawings. When adding reference numerals to the components of each drawing, it should be noted that even if the same or equivalent components are shown in other drawings, the same or equivalent components are designated by the same reference numerals. Additionally, when describing the embodiments of the present invention, detailed descriptions of well-known features or functions will be excluded so as not to unnecessarily obscure the gist of the present invention.

[0048] When describing the components of the exemplary embodiments of the present invention, terms such as first, second, "A", "B", (a), (b), etc. may be used. These terms are only used to distinguish one component from another, and these terms do not limit the nature, order, or sequence of the components constituting them. Additionally, unless otherwise defined, all terms used herein (including technical or scientific terms) have the same meaning as commonly understood by those skilled in the technical field to which the present invention pertains. Terms defined in commonly used dictionaries should be understood to have a meaning equivalent to the contextual meaning in the relevant technical field and should not be understood to have an ideal or overly formal meaning, unless explicitly defined as such in the present invention.

[0049] The following will refer to Figures 2 to 11 for a detailed description of the exemplary embodiments of the present invention.

[0050] The lead vehicle (LV) and the following vehicle (FV) included in a platooning vehicle fleet can perform platooning on a road. The LV and FV can travel while maintaining a specific distance between them. During travel, the LV or FV can adjust the distance between the LV and FV. The LV or FV can increase or decrease the inter-vehicle distance according to the driver's operation. The FV performs lateral steering control based on virtual lane information generated using the lane information of the preceding vehicle, and the virtual lane information will be described below.

[0051] Figure 2 A block diagram showing the configuration of a virtual lane display device according to an embodiment of the present invention.

[0052] Refer to Figure 2, the virtual lane display device 100 may include: a communication part 110, a sensing part 120, a processor 130, a display 140, and a storage device 150.

[0053] When driving in a platoon, the virtual lane display device 100 may generate a virtual lane by converting the lane information of the vehicle in front into the lane information observed from the perspective of the host vehicle, and display the virtual lane on the windshield of the host vehicle. Therefore, even if the front lane is blocked by the vehicle in front, the virtual lane is displayed to assist the driver of the host vehicle driving behind in steering control. In this case, the host vehicle may closely follow the vehicle in front and drive behind it.

[0054] According to an embodiment of the present invention, the virtual lane display device 100 may be implemented inside the vehicle. In this case, the virtual lane display device 100 may be integrally implemented with the internal control unit of the vehicle. Alternatively, the virtual lane display device 100 may be separately implemented from the internal control unit of the vehicle, and may be connected to the internal control unit of the vehicle through an additional connection unit.

[0055] The communication part 110 is a hardware device implemented by various circuits to transmit or receive signals through wireless or wired connections, and it may perform V2V communication with an external server, infrastructure, and other vehicles of the vehicle through in-vehicle network communication technology or wireless Internet access or short-range communication technology. In this case, the vehicle network communication technology may include Controller Area Network (CAN) communication technology, Local Interconnect Network (LIN) communication technology, FlexRay communication technology, and vehicle-to-vehicle communication may be performed through the above communication technologies. Additionally, the wireless communication technology may include Wireless LAN (WLAN), Wireless Broadband (Wibro), Wi-Fi, Worldwide Interoperability for Microwave Access (Wimax). The short-range communication technology may include Bluetooth, ZigBee, Ultra-Wideband (UWB), Radio Frequency Identification (RFID), or Infrared Data Association (IrDA).

[0056] For example, the communication part 110 may share platoon driving information through V2V communication between platoon driving vehicles. In this case, the platoon driving information may include surrounding information measured by the sensors of the host vehicle, the steering angle information of the host vehicle, platoon driving speed, vehicle-to-vehicle speed, destination, route, or front lane information.

[0057] The sensing part 120 senses information of the vehicle in front (such as relative speed, distance, or moving distance) and information for generating the route of the host vehicle (host vehicle route) (such as yaw rate, steering angle, or vehicle speed) of the host vehicle. For this purpose, the sensing part 120 includes a front sensor 121 and a vehicle sensor 122.

[0058] The front sensor 121 measures the longitudinal distance and the lateral distance between the rear part of the vehicle in front and the host vehicle (the rear vehicle).

[0059] The vehicle sensor 122 can measure the yaw rate, the steering angle, or the vehicle speed of the host vehicle.

[0060] For this purpose, the sensing part 120 may include an ultrasonic sensor, a radar, a camera, a laser scanner or a corner radar, a LiDAR, an acceleration sensor, a yaw rate sensor, a torque measurement sensor or a wheel speed sensor, and a steering angle sensor.

[0061] The processor 130 can be electrically connected to the communication part 110, the sensing part 120, the display 140, the storage device 150, etc., can electrically control each component, and can be a circuit that executes software commands. Therefore, the processor 130 can perform various data processing and calculations described below. The processor 130 can process signals transmitted between the components of the virtual lane display device 100.

[0062] The processor 130 determines whether a failure occurs during platooning based on at least one of the communication state, the front sensor state, or the inter-vehicle distance state. When the platooning is in a normal state, the processor 130 can generate a virtual lane by converting the lane information of the vehicle in front into the lane information observed from the perspective of the host vehicle (the rear vehicle of the vehicle in front).

[0063] The processor 130 can perform a control operation to convert the lane information observed from the perspective of the host vehicle into the lane information observed from the perspective of an augmented reality image, and display the converted lane information on the display 140. In this case, the display 140 can be implemented on the windshield of the host vehicle. In this case, the lane information provided from the perspective of the host vehicle can be the image information actually captured by the camera, and the augmented reality image can be an image that includes both actual image information and virtual information, and can be provided from the perspective of the driver.

[0064] The processor 130 can convert the lane information measured based on the camera coordinate system of the vehicle in front into information based on the world coordinate system, and can rotate the lane information of the vehicle in front based on the traveling angle of the vehicle in front and the traveling angle of the host vehicle in the world coordinate system. In other words, the processor 130 can rotate the lane information of the vehicle in front by the difference between the traveling angle of the vehicle in front and the traveling angle of the host vehicle.

[0065] In addition, the processor 130 can shift the rotated lane information by the lateral distance and the longitudinal distance from the origin of the host vehicle to the origin of the vehicle in front. Therefore, the lane information from the perspective of the vehicle in front can be converted into the lane information observed from the perspective of the host vehicle in the world coordinate system.

[0066] The processor 130 may determine whether it is possible to generate the route of the host vehicle based on the measurement status of at least one of the vehicle speed, yaw rate, or steering angle. In other words, when at least one of the vehicle speed, yaw rate, or steering angle is measured, the processor 130 may determine that it is possible to generate the route of the host vehicle.

[0067] The processor 130 may perform a control operation to generate the route of the host vehicle based on at least one of the vehicle speed, yaw rate, or steering angle, and display the route of the host vehicle on a virtual lane, which is generated by transformation from the perspective of the host vehicle.

[0068] When the vehicle speed is less than a predetermined value, the processor 130 may calculate the radius of the moving route of the host vehicle based on the steering ratio, wheelbase, and steering angle, and when the vehicle speed is greater than or equal to the predetermined value, the processor 130 may calculate the radius of the vehicle moving route based on the vehicle speed and yaw rate. Thereafter, the processor 130 may generate the route of the host vehicle based on the origin of the host vehicle by using the radius of the vehicle moving route. The processor 130 may perform a control operation to convert the route of the host vehicle into a route observed from the perspective of an augmented reality image, and display the route of the host vehicle on a virtual lane.

[0069] The processor 130 may determine the lane departure time point by using the lane information of the preceding vehicle and the route of the host vehicle, may provide a lane departure warning when the lane departure time point is less than a predetermined value, and may terminate the process without a lane departure warning when the lane departure time point is greater than or equal to the predetermined value.

[0070] When there are coordinates in the coordinates of the virtual lane that make the distance from the center point of the route of the host vehicle within a preset distance range, the processor 130 may determine that it is possible to determine the lane departure time point. In other words, when at least one of one or more points of the virtual lane is less than or equal to a predetermined distance from the center point of the route of the host vehicle, the processor 130 may determine that it is possible to determine the lane departure time point.

[0071] The processor 130 may determine, among the coordinates in the coordinates of the virtual lane that make the distance from the center point of the route of the host vehicle within a preset distance range, the point with the maximum longitudinal distance from the host vehicle as the lane departure prediction point. In other words, the processor 130 may determine, among one or more points of the virtual lane that are less than or equal to a predetermined distance from the center point of the route of the host vehicle, the point with the maximum longitudinal distance from the host vehicle as the lane departure prediction point.

[0072] The processor 130 may calculate the lane departure prediction time by using the vehicle speed of the host vehicle and the distance from the current point of the host vehicle to the lane departure prediction point, and provide a lane departure warning when the lane departure prediction time is less than a predetermined value.

[0073] The processor 130 may output a warning message for lane departure warning in a pop-up form on a virtual lane, and may differently display a lane that deviates from the virtual lane, including the lane departure prediction time point, in terms of color and thickness.

[0074] According to this embodiment, the processor 130 may be implemented as a preceding vehicle lane information conversion unit 131, a host vehicle route generation unit 132, a lane departure warning determination unit 133, and a screen generation unit 134.

[0075] The preceding vehicle lane information conversion unit 131 may convert lane information measured from the perspective of the preceding vehicle into lane information observed from the perspective of the host vehicle by using the difference between the traveling angle of the preceding vehicle and the traveling angle of the host vehicle, as well as the longitudinal distance and lateral distance from the preceding vehicle.

[0076] The host vehicle route generation unit 132 may generate a predicted movement route of the host vehicle by using the sensing results of the vehicle sensor 122.

[0077] The lane departure warning determination unit 133 may determine the lane departure prediction time point by using the lane information of the preceding vehicle and the route of the host vehicle, and may generate a lane departure warning signal when the lane departure prediction time point is less than a threshold value.

[0078] The screen generation unit 134 may form lane information observed from the perspective of the host vehicle by converting the lane information observed from the perspective of the host vehicle into lane information observed from the perspective of an augmented reality image.

[0079] The display 140 may include an input device and an output device. The input device is used to receive a control command from a user; the output device is used to output the operation state and operation result of the virtual lane display device 100. In this case, the input device may include buttons, and may include a mouse, a joystick, a knob, a stylus, etc. Additionally, the input device may include a soft keyboard implemented on the display. The output device may include a display, and may include a voice output device such as a speaker. When the display includes a touch sensor product such as a touch film, a touch sheet, or a touch pad, the display may operate as a touch screen, and the input device and the output device may be implemented in an integrated form. According to the present invention, the output device may display a virtual lane, a host vehicle movement route, or a lane departure warning in front of the driving road from the perspective of an augmented reality image or the normal image of the host vehicle. The output device may be implemented by using an augmented reality image display to output an augmented reality image. The normal image of the host vehicle may include information about a real image without virtual information, and may be provided from the perspective of the host vehicle. The augmented reality image may include an image with virtual additional information, and may be provided from the perspective of the driver.

[0080] In this case, the display may include at least one of a liquid crystal display (LCD), a thin film transistor liquid crystal display (TFT-LCD), an organic light emitting diode (OLED), a flexible display, a field emission display (FED), or a three-dimensional display (3D display).

[0081] The storage device 150 may store the sensing results of the sensing part 120, the measurement information received from the preceding vehicle, and the data or algorithms necessary for the operation of the virtual lane display device 100.

[0082] For example, the storage device 150 may store platooning information, such as the forward lane information measured by the preceding vehicle, which is received from the preceding vehicle in the platoon through the communication part 110. In addition, the storage device 150 may store information about obstacles (such as the preceding vehicle) sensed by the sensing part 120. The storage device 150 may be implemented as at least one storage medium such as a flash memory type, a hard disk type, a micro type, a card type (e.g., a Secure Digital (SD) card or an eXtreme Digital card) memory, a random access memory (RAM), a static RAM (SRAM), a read only memory (ROM), a programmable ROM (PROM), an electrically erasable programmable ROM (EEPROM), a magnetic RAM (MRAM), a disk type memory, or an optical disk type memory.

[0083] As described above, according to the present invention, the lane information measured by the preceding vehicle is converted into lane information observed from the perspective of the following vehicle and displayed on the AR display of the windshield. Therefore, the driver of the following vehicle can guess the shape of the forward lane to perform steering control. In addition, according to the present invention, the predicted movement route of the host vehicle is displayed together, and a lane departure is warned when the lane departure prediction time point is less than a threshold value, so that the driver can pre-identify the lane departure and avoid the lane departure.

[0084] As described above, according to the present invention, even if the driver's field of vision is restricted by the preceding vehicle traveling at a narrow inter-vehicle distance, the driver of the following vehicle following the preceding vehicle can easily perform steering control, thereby reducing the burden on the driver of the following vehicle. In addition, when the following vehicle cannot smoothly follow the preceding vehicle, a warning is issued, thereby improving the safety of the platoon.

[0085] Hereinafter, reference will be made to Figures 3A to 6C a process of converting the forward lane information measured by the preceding vehicle into lane information observed from the perspective of the following vehicle will be described.

[0086] Figure 3A and Figure 3B are schematic diagrams respectively showing coordinate systems for perspective conversion, Figure 3CShows a lane displayed in the world coordinate system according to an embodiment of the present invention. Figure 4A , Figure 4B and Figure 4C Are schematic diagrams respectively showing that, according to an embodiment of the present invention, lane information observed from the perspective of the preceding vehicle is converted into lane information observed from the perspective of the following vehicle by using the traveling angle. Figure 5A and Figure 5B Are schematic diagrams respectively showing that, according to an embodiment of the present invention, lane information observed from the perspective of the preceding vehicle is converted into lane information observed from the perspective of the following vehicle by moving the longitudinal / lateral distance to the preceding vehicle. Figure 6A , Figure 6B and Figure 6C Are schematic diagrams respectively showing that, according to an embodiment of the present invention, lane information from the perspective of the preceding vehicle is converted into lane information observed from the perspective of the following vehicle.

[0087] The forward lane information measured by the preceding vehicle is measured by the coefficients of the cubic equation. In order to convert the forward lane information into lane information observed from the perspective of the following vehicle, the perspective must be rotated or translated. The cubic equation can be expanded and solved. In this case, since there are terms up to the third order for x and y respectively, the process of solving the equation is very complicated. Therefore, according to the present invention, since the forward lane is divided in units of a specific distance, coordinates can be obtained, and numerical analysis-based processing can be performed using the set of coordinates.

[0088] First, as Figure 3A shown, the virtual lane display device 100 converts the lane information of the preceding vehicle represented by the cubic equation into a set of x and y coordinates.

[0089] When the cubic equation of the lane in the camera coordinate system measured by the lane measurement camera of the preceding vehicle is C3x 3 + C2x 2 + C1x + C0 = y, and the lane display target distance is 50 m and the display resolution is 0.1 m, the following equation 1 can be obtained.

[0090] Equation 1:

[0091] c = [C3, C2, C1, C0]

[0092] x cam = [0:0.1:50]

[0093] = [0 0.1 0.2 0.3 … 49.8 49.9 50]

[0094] y cam = polyval(c, x cam )

[0095] z cam = zeros(1,501) = zeros(1, length(x cam ))

[0096] = [0 0 0 … 0 0 0]

[0097] Lane cam = [x cam ; y cam ; z cam

[0098] The result can be converted to the world coordinate system (as Figure 3B shown), as defined by Equation 2 below.

[0099] Equation 2:

[0100] x world = -y cam

[0101] y world = x cam

[0102] Lane world = [-y cam ; x cam ; z cam

[0103] = [x world ; y world ; z world

[0104] Figure 3C shows the lane displayed in the world coordinate system.

[0105] Figure 4A shows the front lane 411 from the perspective of the preceding vehicle 11, Figure 4B shows the example 412 obtained by converting with the same perspective as when observing the front lane in the direction in which the following vehicle 12 is traveling. Figure 4C shows an example of the lane displayed in the world coordinate system, where the lane is rotated by an angle obtained by subtracting the traveling angle of the following vehicle from the traveling angle of the preceding vehicle.

[0106] The virtual lane display device 100 should rotate the lane information observed from the perspective of the preceding vehicle by a value obtained by subtracting the traveling angle of the following vehicle from the traveling angle of the preceding vehicle based on the z-axis, so that the perspective of the front lane observed from the preceding vehicle 11 is the same as the perspective of the front lane observed by the following vehicle 12 in the traveling direction. In this case, the clockwise direction can be understood as the positive (+) direction. ​​​

[0107] If the value obtained by subtracting the traveling angle of the rear vehicle from the traveling angle of the front vehicle is θ, the rotation matrix (RotationMatrix) and the rotation degree of the lane can be defined as in Equation 3.

[0108] Equation 3:

[0109] RotationMatrix = [cos(θ) sin(θ) 0;

[0110] -sin(θ) cos(θ) 0; 0 0 1]

[0112] RotatedLane = RotationMatrix × Lane world

[0113] As Figure 4B shown, after rotating the lane based on the traveling direction of the rear vehicle, the rotated lane should be shifted by the x-direction distance and y-direction distance (OffsetX and OffsetY) from the origin of the rear vehicle 12 to the origin of the front vehicle 11.

[0114] As Figure 5A shown, assuming that the longitudinal distance and lateral distance to the front vehicle measured by the sensor of the rear vehicle 12 are d x and d y , the length of the front vehicle is L, and the origin of the front vehicle is O front , the following Equation 4 can be obtained.

[0115] Equation 4:

[0116] OffsetX = d x + L sin(θ)

[0117] OffsetY = d y + L cos(θ)

[0118] OffsetZ = 0

[0119] As Figure 5B shown, when shifting the rotated lane coordinates by OffsetX and OffsetY, the lane information in the world coordinate system observed from the perspective of the front vehicle is converted into the lane information observed from the perspective of the rear vehicle. The converted lane information (ConvertedLane) is the sum of the rotation amounts of the lane coordinates OffsetX, OffsetY, and OffsetZ, as shown in the following Equation 5.

[0120] Equation 5:

[0121] ConvertedLane = RotatedLane + [OffsetX; OffsetY; OffsetZ]

[0122] Therefore, as Figure 6A shown, after converting the lane information of the vehicle ahead into the lane information observed in the driving direction of the rear vehicle 12 using the traveling angle, as Figure 6B shown, the rotated lane coordinates are shifted by OffsetX and OffsetY. Then, as Figure 6B shown, the forward lane can be generated and displayed with the perspective changed from the vehicle ahead to the rear vehicle. Figure 6C The lane shows the lane converted to be observed from the perspective of the world coordinate system.

[0123] Figure 7A and Figure 7B are schematic diagrams showing the display of the forward lane according to an embodiment of the present invention, respectively.

[0124] Figure 7A shows an example of displaying the lanes 711 and 712 measured by the vehicle ahead, Figure 7B shows an example of displaying the virtual lanes 713 and 714 on the forward augmented reality (AR).

[0125] The virtual lane display device 100 converts the lane coordinates observed from the perspective of the rear vehicle and calculated in the world coordinate system into the coordinates observed from the AR image perspective to display on the vehicle windshield, and displays the converted lane information on the AR display (windshield).

[0126] Figure 8A and Figure 8B are schematic diagrams showing the methods for generating the route of the vehicle itself and determining the lane departure point according to another embodiment of the present invention, respectively. Figure 9A , Figure 9B and Figure 9C are schematic diagrams showing the display of the route of the vehicle itself and the lane departure warning according to an embodiment of the present invention, respectively.

[0127] Referring to Figure 8A and Figure 8B , the method for generating the route of the vehicle itself will be explained in detail.

[0128] When the vehicle speed is less than the threshold, the virtual lane display device 100 can calculate the radius R of the vehicle movement route based on the steering angle, steering ratio, and wheelbase as shown in Equation 6 below.

[0129] Equation 6:

[0130]

[0131] When the vehicle speed exceeds the threshold, the virtual lane display device 100 can calculate the radius R of the vehicle's moving route based on Equation 7 using the vehicle speed and the yaw rate.

[0132] Equation 7:

[0133]

[0134] The virtual lane display device 100 can generate the route of the host vehicle based on the origin of the following vehicle (the midpoint of the foremost end of the vehicle) by using the radius R of the vehicle's moving route. The radius R can be defined as in Equation 8 below.

[0135] Equation 8:

[0136] (x - R) 2 + y 2 = R 2

[0137] In this case, "x" refers to the lateral distance and "y" refers to the longitudinal distance.

[0138] Figure 9A The virtual front lanes 911 and 912 are shown displayed on the windshield, and the route 913 of the host vehicle is additionally displayed. In other words, the virtual lane display device 100 converts the route of the host vehicle obtained in the world coordinate system of the following vehicle into a route observed from the AR image perspective, so that the route of the host vehicle is displayed on the windshield, and the converted lane coordinates can be displayed on the windshield.

[0139] As Figure 8B shown, the virtual lane display device 100 can determine and display the lane departure prediction time point.

[0140] When the measurement information of the preceding vehicle is normal, when the route of the host vehicle can be generated, and when there are coordinates in the virtual lane coordinates whose distance from the center point (-R, 0) of the route of the host vehicle is less than "R", the virtual lane display device 100 determines that the lane departure prediction time point can be determined, and determines the lane departure prediction time point.

[0141] The virtual lane display device 100 can determine the point with the maximum longitudinal distance "y" from the host vehicle among the coordinates of the virtual lane whose distance from the center point (-R, 0) of the route of the host vehicle is less than "R" as the lane departure time point, and can calculate the predicted lane departure time point (the crossing time) using the speed of the host vehicle and the distance to the lane departure point, as shown in Equation 9 below.

[0142] Equation 9:

[0143]

[0144] The virtual lane display device 100 can display and terminate a lane departure warning when the lane departure prediction time point is less than a threshold value, or terminate the lane departure warning without displaying the lane departure warning when the lane departure prediction time point is greater than or equal to the threshold value. Refer to Figure 9B , the virtual lane display device 100 can display a lane departure point 914 on the virtual lane 912, can display a warning message of "Lane departure risk" in a pop-up form, and can flash-display the warning message in a pop-up form. Refer to Figure 9C , the virtual lane display device 100 can display the lane 915 predicted to deviate among the virtual lanes 911 and 912 differently in terms of thickness or color, and can flash-display the lane 915.

[0145] Hereinafter, with reference to Figure 10 A method for displaying a virtual lane according to an embodiment of the present invention will be described in detail. Figure 10 is a flowchart showing a method for displaying a virtual lane.

[0146] Hereinafter, it is assumed that Figure 1 the virtual lane display device 100 executes Figure 10 the process.

[0147] With reference to Figure 10 , the virtual lane display device 100 determines whether the current platooning is normal (S101).

[0148] In this case, the virtual lane display device 100 can determine whether the V2V communication connection is normal, whether the front sensor is normal, or whether the forward view is restricted due to a short inter-vehicle distance to determine whether the platooning is normal. On the contrary, when there is no platooning, or when the front sensor or communication device fails, the virtual lane display device 100 can determine that the platooning is abnormal.

[0149] When the current platooning is abnormal, the virtual lane display device 100 can determine that the logic for displaying the virtual lane is not necessary for the operation or is in an unavailable state, and does not perform the display operation (S115).

[0150] When the platooning is normal, the virtual lane display device 100 determines whether the lane information of the preceding vehicle received from the preceding vehicle is information normally measured for the logic of displaying the virtual lane (S102). In this case, the virtual lane display device 100 can determine whether the lane information of the preceding vehicle for generating the virtual lane is normally received to determine whether the lane information of the preceding vehicle is information normally measured.

[0151] When the lane information of the leading vehicle is not information obtained through normal measurement, the virtual lane display device 100 notifies the driver that it is difficult to display the forward lane (S106). In this case, the virtual lane display device 100 can provide a notification regarding the difficulty in displaying the forward lane in an auditory or visual manner. Thereafter, the virtual lane display device 100 can determine whether it is possible to generate the route of the host vehicle (S107). A process related to the generation of the route of the host vehicle will be described below.

[0152] When the lane information of the leading vehicle is information obtained through normal measurement, the virtual lane display device 100 converts the lane information from the perspective of the leading vehicle into lane information from the perspective of the following vehicle (S103).

[0153] In this case, the virtual lane display device 100 can perform processing by using a set of coordinates obtained when dividing the lane of the host vehicle in units of a specific distance and based on numerical analysis, so as to convert the lane information into lane information from the AR image perspective. Additionally, the virtual lane display device 100 can perform the conversion by using various algorithms.

[0154] Thereafter, the virtual lane display device 100 can convert the generated lane information observed from the perspective of the following vehicle into lane information from the AR image perspective (S104). The lane coordinates obtained in the world coordinate system from the perspective of the following vehicle are converted into lane coordinates from the AR image perspective, such that the lane coordinates are displayed on the windshield.

[0155] The virtual lane display device 100 displays the converted lane information from the AR image perspective on the AR screen (S105). In this case, the virtual lane display device 100 can use the converted lane coordinates from the AR image perspective to display a virtual lane on the windshield, as Figure 7B shown.

[0156] Thereafter, the virtual lane display device 100 determines whether it is possible to generate the route of the host vehicle (S107).

[0157] When it is not possible to generate the route of the host vehicle, the virtual lane display device 100 notifies that it is not possible to display the route of the host vehicle (S116).

[0158] When it is possible to generate the route of the host vehicle, the virtual lane display device 100 generates the route of the host vehicle (S108), converts the generated route of the host vehicle into the route of the host vehicle from the AR image perspective (S109), and displays the converted route of the host vehicle from the AR image perspective on the AR screen (S110). As Figure 9A shown, the virtual lane display device 100 can display the route 913 of the host vehicle together with the virtual lanes 911 and 912.

[0159] After that, the virtual lane display device 100 determines whether it is possible to determine the lane departure prediction time point (S111).

[0160] When it is possible to determine the lane departure prediction time point, the virtual lane display device 100 calculates the lane departure prediction time point (S112).

[0161] The virtual lane display device 100 may determine whether the lane departure prediction time point is less than a threshold value (S113), and when the lane departure prediction time point is less than the threshold value, may display a lane departure warning (S114). As Figure 9B and Figure 9C shown, the virtual lane display device 100 may display the lane departure warning by displaying a lane departure risk warning message in a pop-up form, by changing the color or thickness of the deviated lane, or by flashing the deviated lane.

[0162] When the lane departure prediction time point is greater than or equal to the threshold value, the virtual lane display device 100 terminates the lane departure warning.

[0163] As described above, according to the present invention, when the field of view of a following vehicle following a leading vehicle is blocked by a large leading vehicle (such as a truck) at a short distance during platoon driving, lane information measured by the leading vehicle is received from the leading vehicle through V2V communication, and a virtual lane is generated and displayed based on the lane information measured by the leading vehicle, so that the forward lane blocked by the leading vehicle can be observed. Therefore, the driver of the following vehicle can participate in platoon driving more comfortably and safely.

[0164] Figure 11 A computing system according to an embodiment of the present invention is shown.

[0165] Referring to Figure 11 , the computing system 1000 may include at least one processor 1100, a memory 1300, a user interface input device 1400, a user interface output device 1500, a storage device 1600, and a network interface 1700 that are interconnected through a bus 1200.

[0166] The processor 1100 may be a central processing unit (CPU) or a semiconductor device that processes instructions stored in the memory 1300 or the storage device 1600. Each of the memory 1300 and the storage device 1600 may include various types of volatile or non-volatile storage media. For example, the memory 1300 may include a read-only memory (ROM) and a random access memory (RAM).

[0167] Accordingly, the operations of the methods or algorithms described in connection with the embodiments disclosed in the present invention may be implemented directly in hardware modules, software modules, or a combination thereof executed by the processor 1100. The software modules may reside in a storage medium (e.g., the memory 1300 or the storage device 1600), such as RAM, flash memory, ROM, erasable programmable ROM (EPROM), electronically EPROM (EEPROM), registers, hard disks, removable disks, or CD-ROMs.

[0168] The exemplary storage medium may be connected to the processor 1100. The processor 1100 may read information from the storage medium and may write information to the storage medium. Alternatively, the storage medium may be integrated with the processor 1100. The processor and the storage medium may reside in an application specific integrated circuit (ASIC). The ASIC may reside in the user terminal. Alternatively, the processor and the storage medium may reside as separate components of the user terminal.

[0169] As described above, according to the present invention, virtual lane information obtained by converting lane information measured for a leading vehicle from the perspective of a following vehicle may be provided to the driver of the following vehicle, so that even if the field of view of the driver of the following vehicle is blocked, the driver of the following vehicle is allowed to drive the following vehicle safely based on the virtual lane information.

[0170] In addition, various effects directly or indirectly understood through the present invention may be provided.

[0171] Although the present invention has been described above with reference to exemplary embodiments and the drawings, the present invention is not limited thereto, and those skilled in the art to which the present invention pertains may make various changes and modifications without departing from the spirit and scope of the present invention.

[0172] Accordingly, exemplary embodiments of the present invention are provided to explain the spirit and scope of the present invention, but the present invention is not limited to these exemplary embodiments, such that the spirit and scope of the present invention are not limited by the exemplary embodiments.

Claims

1. A device for displaying a virtual lane, the device comprising: A communication part configured to receive, during platoon driving, lane information of a preceding vehicle measured in a camera coordinate system from the preceding vehicle through V2V communication; A processor configured to convert the lane information of the preceding vehicle into information based on a world coordinate system, rotate the lane information of the preceding vehicle based on the traveling angle of the preceding vehicle and the traveling angle of the host vehicle, shift the rotated lane information by a lateral distance and a longitudinal distance from the origin of the host vehicle to the origin of the preceding vehicle to obtain lane information observed from the perspective of the host vehicle, generate a virtual lane by converting the lane information observed from the perspective of the host vehicle into lane information observed from the perspective of an augmented reality image, and convert the route of the host vehicle into a route observed from the perspective of an augmented reality image; And A display configured to display the virtual lane and the converted route of the host vehicle in the form of an augmented reality image.

2. The device for displaying a virtual lane according to claim 1, wherein, The processor is configured to: Determine whether a failure occurs during platoon driving based on at least one of a communication state, a front sensor state, or an inter-vehicle distance state.

3. The device for displaying a virtual lane according to claim 1, wherein, The processor is configured to: Determine whether a route of the host vehicle can be generated based on a measurement state of at least one of the vehicle speed, yaw rate, or steering angle of the host vehicle.

4. The device for displaying a virtual lane according to claim 3, wherein, The processor is configured to: Generate a route of the host vehicle based on at least one of the vehicle speed, yaw rate, or steering angle.

5. The apparatus for displaying a virtual lane according to claim 4, wherein, The processor is configured to: Display the route of the host vehicle on the virtual lane generated by conversion from the perspective of the host vehicle.

6. The apparatus for displaying a virtual lane according to claim 1, wherein, The processor is configured to: When the vehicle speed of the host vehicle is less than a predetermined value, calculate the radius of the moving route of the host vehicle based on the steering ratio, wheelbase, or steering angle of the host vehicle.

7. The device for displaying a virtual lane according to claim 6, wherein, The processor is configured to: When the vehicle speed of the host vehicle is greater than or equal to the predetermined value, calculate the radius of the moving route of the host vehicle by using the vehicle speed and yaw rate of the host vehicle.

8. The apparatus for displaying a virtual lane according to claim 7, wherein, The processor is configured to: Generate a route of the host vehicle based on the origin of the host vehicle by using the radius of the moving route of the host vehicle.

9. The apparatus for displaying a virtual lane according to claim 1, wherein, The processor is configured to: Determine a lane departure time point by using the lane information of the preceding vehicle and the route of the host vehicle; Provide a lane departure warning when the lane departure time point is less than a predetermined value.

10. The device for displaying a virtual lane according to claim 1, wherein, The processor is configured to: Determine that a lane departure time point can be determined when the distance between at least one point among a plurality of points of the virtual lane and the center point of the route of the host vehicle is less than or equal to a predetermined distance.

11. The device for displaying a virtual lane according to claim 10, wherein, The processor is configured to: Among one or more points of the virtual lane whose distance from the center point of the route of the host vehicle is less than or equal to the predetermined distance, determine the point with the maximum longitudinal distance from the host vehicle as the lane departure prediction point.

12. The device for displaying a virtual lane according to claim 11, wherein, The processor is configured to: Calculate a lane departure prediction time by using the vehicle speed of the host vehicle and the distance from the current point of the host vehicle to the lane departure prediction point; Provide a lane departure warning when the lane departure prediction time is less than a predetermined value.

13. The apparatus for displaying a virtual lane according to claim 12, wherein, The processor is configured to: Display a warning message of the lane departure warning in a pop-up form on the virtual lane; Display the deviated lane including the lane departure prediction point in the virtual lane differently in terms of color or thickness, or blink-display the deviated lane.

14. The apparatus for displaying a virtual lane according to claim 1, wherein, The display is configured to display a virtual lane, the moving route of the vehicle itself, and a lane departure warning in the form of an augmented reality image on the windshield of the vehicle itself.

15. A method for displaying a virtual lane, the method comprising: Receiving, by a communication part, lane information of a preceding vehicle measured based on a camera coordinate system from the preceding vehicle through V2V communication during platoon driving; Converting, by a processor, the lane information of the preceding vehicle into information based on a world coordinate system; Rotating, by the processor, the lane information of the preceding vehicle based on the traveling angle of the preceding vehicle and the traveling angle of the vehicle itself; Shifting, by the processor, the rotated lane information by a lateral distance and a longitudinal distance from the origin of the vehicle itself to the origin of the preceding vehicle to obtain lane information observed from the perspective of the vehicle itself; Generating, by the processor, a virtual lane by converting the lane information observed from the perspective of the vehicle itself into lane information observed from the perspective of an augmented reality image; Converting, by the processor, the route of the vehicle itself into a route observed from the perspective of an augmented reality image; Displaying, by a display, the virtual lane and the converted route of the vehicle itself in the form of an augmented reality image.

Images