Head-up display and control method thereof

Abstract

The present invention relates to a head-up display and a control method thereof. The head-up display of a vehicle involved in the present invention is characterized in that it includes: a display element that outputs light; an optical system that controls the path of the light so that the image formed by the light is output toward a light-transmitting area; and a control unit that controls the display element to form the image, wherein the optical system controls the path of the light so that the virtual image of the image is located on the ground in front of the vehicle after the image passes through the light-transmitting area, wherein the image is generated by using a first viewpoint facing the ground with an eye box as a reference and a second viewpoint facing the three-dimensional space on the ground with the eye box as a reference in a manner that includes a graphic object with a three-dimensional sense based on the ground in the virtual image of the image.

Description

Technical Field

[0001] The present invention relates to a head-up display of a vehicle that outputs information in front of a driver and a control method thereof. Background Art

[0002] A head-up display (HUD) device for a vehicle is a device that transmits information to the driver by projecting an image output from a display in the form of a graphic image onto the windshield (Wind Shield) in front of the driver or a synthesizer through an optical system.

[0003] Here, the optical system may be composed of a plurality of reflectors or lenses for changing the optical path of the image output from the display. Such a head-up display device for a vehicle has the advantage of providing convenience while guiding the driver's immediate reaction.

[0004] In general vehicle head-up display (HUD) devices, the position of the image is fixed at about 2 to 3 meters in front of the user. When driving, the driver's gaze distance is from a short distance to about 300 meters. As a result, the driver drives while looking at a long distance, and in order to confirm the information of the head-up display (HUD) device during driving, it is necessary to adjust the focus of the eyes significantly, which is inconvenient. That is, the driver repeatedly adjusts the focus between the long distance where the main field of view is located and the 3m position where the image is formed.

[0005] Therefore, it is necessary to develop a head-up display device to provide augmented reality for the driving environment so that the driver can obtain the information desired by the driver without changing the focus of the eyes under the gaze point during driving.

[0006] For example, Korean Patent No. 10-2116783 (published on May 29, 2020) relates to a head-up display device for implementing augmented reality, which places an image on the ground to provide real information at the driver's viewpoint. In this way, information is transmitted in three dimensions at the driver's viewpoint by aligning the image with the ground, but for certain information, it is more advantageous to stand on the ground rather than lying on the ground. Summary of the invention

[0007] The present invention provides a method for realizing a graphic object with a three-dimensional sense in an augmented reality head-up display that realizes augmented reality at a driver's viewpoint by positioning an image on the ground.

[0008] More specifically, the present invention provides a method for realizing a graphic object in a form erected on the ground in an image consistent with the ground.

[0009] In addition, the present invention provides a head-up display and a control method thereof that realizes both an image that is consistent with the ground and an image that has a three-dimensional sense of standing on the ground.

[0010] To solve the above-mentioned technical problems, the head-up display and control method thereof involved in the present invention utilize the following processing: converting the ground into a three-dimensional space and generating an image in the three-dimensional space, and then converting it into a two-dimensional plane equivalent to the ground.

[0011] Specifically, the head-up display of a vehicle includes: a display element that outputs light; an optical system that controls the path of the light so that an image formed by the light is output toward a light-transmitting area; and a control unit that controls the display element to form the image, wherein the optical system controls the path of the light so that a virtual image of the image is located on the ground in front of the vehicle after the image passes through the light-transmitting area, and the image is generated by using a first viewpoint facing the ground with an eye-box as a reference, and a second viewpoint facing the three-dimensional space on the ground with the eye-box as a reference, in a manner such that a graphic object having a three-dimensional sense based on the ground is included in the virtual image of the image.

[0012] The first viewpoint may be a viewpoint of a driver recognizing the virtual image on the ground, the second viewpoint may be a viewpoint of a virtual camera, and the image may be generated by making the viewpoint of the driver and the viewpoint of the camera consistent with each other. The graphic object may be formed in the three-dimensional space based on the viewpoint of the camera, and projected onto the two-dimensional plane of the ground based on the viewpoint of the driver.

[0013] The control unit generates the graphic object obtained by projecting the three-dimensional object equivalent to the graphic object onto the ground by utilizing the transformation of projecting the three-dimensional object onto the ground and the characteristics of the optical system that transforms the image into the virtual image.

[0014] The control unit may generate an image by projecting the three-dimensional object onto the ground, and transform the projected image into a video corresponding to the virtual image by utilizing the characteristics of the optical system.

[0015] The control unit may transform the three-dimensional object into the graphic object by using the dot product of a first transformation matrix for projecting the three-dimensional object onto the ground and a second transformation matrix for transforming the virtual image into the image.

[0016] Moreover, in the present invention, the virtual image of the image may include a two-dimensional graphic object formed on a virtual image plane corresponding to the ground, and the graphic object with the stereoscopic effect may include a three-dimensional graphic object standing on the virtual image plane.

[0017] The display element may form a single image in which the two-dimensional graphic object and the three-dimensional graphic object may be selectively generated.

[0018] The two-dimensional graphic object may be information associated with at least one of a lane, a driving speed, and a driving direction indication, and the three-dimensional graphic object may be information associated with at least one of a point of interest (POI), a guide statement, a destination, and surrounding vehicles.

[0019] The light-transmitting region may be formed on a combiner or a windshield.

[0020] In addition, the present invention discloses a head-up display, characterized in that it includes: a display element that outputs an image; an imaging optical system that controls the path of the light to form the image in a light-transmitting area and makes the virtual image of the image located on the ground in front of the vehicle; and a control unit that controls the display element to form the image, the control unit generating the image in a manner that a two-dimensional graphic object corresponding to the ground and a three-dimensional graphic object having a stereoscopic effect based on the ground are arranged on the ground, the two-dimensional graphic object and the three-dimensional graphic object are generated by processing from different viewpoints based on an eye-box.

[0021] In addition, the present invention discloses an image output method for a head-up display, characterized in that it includes: a step of selecting information to be transmitted through the head-up display; and a step of outputting the image using an optical system that configures a virtual image of the image formed by a display element on the ground in front of a vehicle. In the step of outputting the image, the image is generated by including a graphic object with a three-dimensional sense based on the ground in the virtual image of the image, using a first viewpoint facing the ground with an eye-box as a reference, and a second viewpoint facing the three-dimensional space on the ground with the eye-box as a reference.

[0022] The head-up display and control method thereof according to the present invention can realize an image standing on the ground in a display method in which the position of the image corresponds to the ground, thereby providing augmented reality information in the form of a graphic object with a three-dimensional sense on the ground.

[0023] In addition, according to the present invention, since a graphic object having a three-dimensional effect is realized on the ground, it is possible to realize a head-up display that improves readability and can transmit information in a more intuitive form.

[0024] Furthermore, according to the present invention, by using a method of making the driver's viewpoint consistent with the camera's viewpoint, it is possible to realize an image combining a two-dimensional graphic object and a three-dimensional graphic object based on the ground by only simpler data processing. In addition, it is possible to realize image output in different forms on a single display. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 It is a schematic diagram for explaining an image realized by the head-up display according to the present invention.

[0026] Figure 2a It is a diagram showing the image position of the head-up display according to one embodiment of the present invention.

[0027] Figure 2b as well as Figure 2c It shows that Figure 2a Schematic diagrams of different embodiments of the optical design structure of a head-up display.

[0028] Figure 3a as well as Figure 3b It is a schematic diagram showing an example of an image realized by the head-up display according to the present invention.

[0029] Figure 4 This is a schematic diagram showing a method of generating an image by a display element that outputs a virtual image on the ground.

[0030] Figure 5a as well as Figure 5b It is a schematic diagram showing a method for generating a two-dimensional ground image of a three-dimensional object.

[0031] Figure 6 This is a schematic diagram used to explain the theory of a method for generating a two-dimensional ground image by integrating the image of a three-dimensional object.

[0032] Figure 7 It shows the use of Figure 6 A flowchart of an example of a theoretical data processing method.

[0033] Figure 8 is equivalent to Figure 7 Figure 2 shows examples of images at each step.

[0034] Fig. 9 It shows the use of Figure 6 A flowchart of another example of the theoretical data processing method.

[0035] Fig.10 is equivalent to Fig. 9 Figure 2 shows examples of images at each step. DETAILED DESCRIPTION

[0036] Below, the embodiments disclosed in this specification are described in detail with reference to the accompanying drawings, and the same or similar structural elements are marked with the same figure numbers regardless of the figure numbers, and repeated descriptions are omitted. The suffixes "module" and "part" of the structural elements used in the following description are only marked or mixed from the perspective of easy description of the specification, and they themselves do not indicate that they have mutually distinguishable meanings or functions. In addition, when describing the embodiments disclosed in this specification, the specific description of the relevant common knowledge is omitted when it is judged that it will make the main purpose of the embodiments disclosed in this specification unclear. In addition, the drawings are provided only for the easy understanding of the embodiments disclosed in this specification, and it should be understood that the technical concept disclosed in this specification is not limited by the drawings, but includes all changes, equivalents and alternatives included in the concept and technical scope of the present invention.

[0037] Terms including ordinal numbers such as first and second may be used when describing various structural elements, but the structural elements are not limited by the terms. The purpose of the terms is only to distinguish one structural element from another structural element.

[0038] When a structural element is mentioned to be "coupled" or "connected" to another structural element, it may be directly coupled or connected to the other structural element, but it should be understood that there may be other structural elements between them. Conversely, when a structural element is mentioned to be "directly coupled" or "directly connected" to another structural element, it should be understood that there are no other structural elements between them.

[0039] A singular expression includes a plural expression as long as there is no contrary meaning indicated in the context.

[0040] In the present application, it should be understood that terms such as "including" or "having" are only used to indicate the existence of features, numbers, steps, actions, structural elements, components or combinations thereof recorded in the specification, rather than excluding the possibility of existence or addibility of one or more other features, numbers, steps, actions, structural elements, components or combinations thereof.

[0041] The present invention relates to a head-up display and a control method thereof, and more particularly, to a head-up display that outputs information in front of a driver in a vehicle.

[0042] Here, a head-up display (HUD) is a screen display device provided in front of a driver or operator in a car or airplane, and refers to a display that enables the driver or operator to see various information while looking up. In the present invention, the head-up display is applied to a vehicle, and a car is shown for ease of explanation, but the present invention is not necessarily limited to this. For example, the vehicle can be any power device for the driver to ride, and can be a car, motorcycle, electric skateboard, electric bicycle, electric wheelchair, helicopter, airplane, ship, construction equipment, mobile robot, and other various forms.

[0043] As described above, the head-up display and control method thereof according to the present invention can be applied to various types of vehicles as a device that enables a user to recognize various information while looking up. However, for ease of explanation, a car is shown, and the head-up display and control method thereof according to the present invention are described in more detail below.

[0044] Figure 1 It is a schematic diagram for explaining an image realized by the head-up display according to the present invention.

[0045] Figure 1 1 is a virtual image recognized in the sight of a user in a vehicle driving seat, and shows a case where various graphic objects 111 and 112 are output. In this example, the graphic objects are output using the synthesizer 121, but it can also be applied to a method using a windshield.

[0046] In the present invention, the case where an object in the real world (such as a signboard) is presented as a graphic (or image) in an image is named a graphic object. In this case, individual graphic objects can be formed as units of independently distinguishable objects. For example, there can be a graphic object equivalent to a "signboard" as a three-dimensional object and a graphic object equivalent to a "lane" as a two-dimensional object.

[0047] In this case, the surrounding space of the vehicle can be defined as a two-dimensional space and a three-dimensional space. The two-dimensional space can be the driving surface on which the vehicle is driving, and the three-dimensional space 12 can be the three-dimensional space on which the vehicle is driving. The driving surface can be called the ground 11, which can be understood as the surface contacted by the wheels of the vehicle. In addition, the ground in the present invention is not necessarily limited to the driving surface, and can be a road surface, a pedestrian road, etc.

[0048] On the other hand, a user riding in the vehicle can see at least one of the ground 11 and the three-dimensional space 12 .

[0049] According to the head-up display of the present invention, it is possible to output augmented reality information in the form of a virtual image on the ground surface 11. For example, a virtual image is shown on the actual road surface at the viewpoint the user is looking at.

[0050] At this time, the virtual image is represented in a two-dimensional form on the virtual screen located on the ground 11, so it can be called a two-dimensional graphic object 111. According to the figure, the two-dimensional graphic object 111 can be equivalent to at least one of the information such as turning information on the road during driving, lane information, distance information to the vehicle in front, driving direction indication (or navigation information), speed limit indication, lane departure alarm, etc. As such an example, as shown in the figure, the two-dimensional graphic object 111 can be information associated with at least one of the lane, driving speed, and driving direction indication (or navigation information). More specifically, the information of the speed limit of 60 and the driving navigation in the lane are output to the ground with the two-dimensional graphic object 111.

[0051] In another sense, an image is formed by light output from the display element, and a virtual image of the image includes a two-dimensional graphic object formed on a virtual image plane corresponding to the ground 11 .

[0052] Thus, the head-up display of the present invention enables the user to intuitively recognize information in various driving environments without moving the focus of the eyes to other places by making the virtual screen correspond to the ground. That is, the information to be conveyed by the head-up display is displayed in augmented reality on the ground that the user actually looks at during driving.

[0053] On the other hand, referring to this figure, the head-up display according to the present invention can output a graphic object having a stereoscopic effect in the three-dimensional space 12. More specifically, the graphic object having a stereoscopic effect can include a three-dimensional graphic object 112 standing on the virtual image plane.

[0054] The three-dimensional graphic object 112 may be a three-dimensional image located on the ground 11 and at least partially protruding into the three-dimensional space 12, and may be an image recognized by the user in a stereoscopic manner. As such an example, the three-dimensional graphic object 112 may be a standing image erected on the ground. In addition, the three-dimensional graphic object 112 may be information associated with at least one of a point of interest (POI), a guide sentence, a destination, and surrounding vehicles.

[0055] More specifically, the standing image may be any one of a first standing image and a second standing image having mutually different characteristics.

[0056] Here, the first standing image may be a three-dimensional image having a depth value. In this case, the first standing image may be formed by a combination of left-eye and right-eye images, or may be implemented as a hologram.

[0057] Moreover, the second standing image may be an image that does not have an actual depth value but feels like a three-dimensional image due to a visual error phenomenon or the like. Due to this visual error phenomenon, although it is located on a two-dimensional plane, the driver feels that it is standing upright on the ground. In the following description, an example of implementing the standing image as the second standing image is shown. However, the present invention is not necessarily limited to this, and can also be used as part of a method for implementing the first standing image. Referring to this figure, if a gas station is located in front of the right side of the vehicle, the vehicle recognizes information about the gas station, and the head-up display outputs a gas station image at the location where the gas station is located.

[0058] The gas station image may be the three-dimensional graphic object 112 , for example, an image standing upright on the ground 11 .

[0059] Thus, in the present invention, a head-up display is used to generate a two-dimensional graphic object 111 that is consistent with the ground surface 11 , and a three-dimensional graphic object 112 having a stereoscopic effect is output on the ground surface 11 .

[0060] In this way, the present invention provides information to be transmitted by augmented reality on the ground that the user is looking at. In this case, a more intuitive head-up display is realized by selecting two-dimensional graphic objects and three-dimensional graphic objects according to the type of the information. The head-up display and the control method thereof involved in the present invention are described in more detail below with reference to the accompanying drawings.

[0061] Figure 2a FIG. 1 is a diagram showing the image position of a head-up display according to an embodiment of the present invention. Figure 2b as well as Figure 2c It shows that Figure 2a Schematic diagrams of different embodiments of the optical design structure of a head-up display, Figure 3a as well as Figure 3b It is a schematic diagram showing an example of an image realized by the head-up display according to the present invention.

[0062] First, refer to Figure 2a The head-up display of the present invention can present the position of a virtual image 110 that the user 13 can see with his eyes as a three-dimensional viewpoint lying horizontally in a manner corresponding to the ground 11 in front of the driver.

[0063] The position of the image generated by the optical system of a general vehicle head-up display is fixed 2 to 3 meters in front of the driver and is approximately perpendicular to the ground 11. In contrast, the head-up display of the present invention is a three-dimensional augmented reality display, and the virtual image 110 is located on a virtual plane corresponding to the ground 11 in front of the driver's gaze.

[0064] In this case, the head-up display may include a display element and an optical system. The head-up display of the present invention does not directly project onto a screen to generate a real image like a general projector, but reflects through the optical system of the head-up display to generate a virtual image 110 visible to the eyes.

[0065] The display element outputs light, and the optical system controls the path of the light by refraction, reflection, etc., so that an image formed by the light is output toward the light-transmitting area.

[0066] The optical system may be an imaging optical system for forming an image of the display element, and the imaging optical system is configured to diffuse light passing through the light-transmitting area and output a virtual image to the ground. In this case, the light-transmitting area may be formed in a combiner 121 (combiner, see Figure 2b ) or windshield 221 (refer to Figure 2c ).

[0067] First, refer to Figure 2b The head-up display 120 according to the present invention may include a display element 122 and a synthesizer 121 .

[0068] The display element 122 can be called a display panel, a display source, etc. The display element 122 can be any one of LCD (Liquid Crystal Display), LED (Light Emitting Diode), DLP (Digital Light Projector), OLED (Organic Light Emitting Diodes), LD (Laser Diode), and LBS (Laser Beam Scanning) displays.

[0069] The synthesizer 121 is configured to reflect light toward the user's eye side and transmit light toward the outside (front). The synthesizer 121 may be composed of one or more optical elements. For ease of description, it is assumed that a synthesizer composed of one optical element is used below.

[0070] In order to improve the quality of the image or to have an optimal size and performance according to the situation, another optical system (not shown) may be included between the display element 122 and the synthesizer 121. In addition, the synthesizer 122 may be composed of the elements included in the head-up display 120.

[0071] On the other hand, it is also possible to use the windshield of the vehicle as a synthesizer. Figure 2c In order to use the windshield 221 of the vehicle as a synthesizer, the head-up display 220 may include a display element 222 and a reflector 223 .

[0072] Since the display element 222 has the same features as the above-mentioned synthesizer-type display element, its description is replaced by the above content.

[0073] The reflector 223 plays the role as described below, that is, it reflects the light of the display element 222 toward the windshield 221, and focuses the virtual image (virtual image) on the ground in front of the driver. At this time, the windshield 221 can play the role as described below, that is, it reflects the light of the light source reflected by the reflector 223 toward the eye-box, and transmits the light to the outside (front). The eye-box is a spatial area where the user can recognize the image, which refers to the position of the driver's eyes.

[0074] The head-up display 220 includes a structure that projects light from a light source 222 onto the ground through a reflector 223 and a windshield 221, thereby enabling a virtual image to be located on the ground in front of the driver. In this case, the reflector 223 may include a plurality of reflectors as an optical system.

[0075] As described above, in the present invention, a synthesizer or a windshield type head-up display can be applied, and is configured to configure a virtual image 110 on the ground 11. In addition, in the present invention, the image is referred to as Figure 1 As described above, a three-dimensional graphic object 112 having a stereoscopic effect is realized in the virtual image 110 located on the ground 11 (see Figure 1 ).

[0076] As such, in the present invention, instead of having a display element for realizing a two-dimensional graphic object and a display element for realizing a three-dimensional graphic object separately, the two-dimensional graphic object and the three-dimensional graphic object are realized in a single display element. For example, the display element may form a single image, and the two-dimensional graphic object and the three-dimensional graphic object may be generated in the single image. In this case, the two-dimensional graphic object and the three-dimensional graphic object may be generated selectively or simultaneously.

[0077] As an example of the three-dimensional graphic object, Figure 3a In this way, in order to prevent vehicles from speeding at the intersection, an obstacle image 113 is formed on the ground. As another example, Figure 3b In this way, information about the surrounding conditions is output in the form of a guide sentence 114 standing upright relative to the ground. The guide sentence 114 is implemented in a manner of being presented in upright text when viewed in the eye zone, thereby providing high readability for the driver.

[0078] Like in Figure 3a as well as Figure 3b As shown in FIG. 1 , the head-up display of the present invention outputs three-dimensional graphic information on the ground and provides more intuitive information to the driver.

[0079] The main information provided by the vehicle navigation may include path information on the road during driving, lane information, distance information to the vehicle in front, etc. In addition, the ADAS (Advanced Driver-Assistance System) provides the driver with safety-related information. At this time, the corresponding information mainly includes lane information, distance information to the front / side vehicles, emergency information, etc. Similarly, when driving automatically, it is necessary to provide passengers with information about the conditions that are about to occur in the driving subject, i.e. the vehicle, such as turns on the road or lane changes. Path information, as information for guiding the path, may include TBT (turn-by-turn) information for guiding straight driving, turns, etc.

[0080] Such various information can be selected as any one of a two-dimensional graphic object and a three-dimensional graphic object, and displayed on the ground through the head-up display of the present invention.

[0081] At this time, the head-up display may include a control unit (not shown), which provides three-dimensional graphic information on the ground in the form of a virtual image, and controls the display element so that an image equivalent to the virtual image is formed on the display element. The control unit generates the image by using a first viewpoint facing the ground based on an eye-box and a second viewpoint facing the three-dimensional space on the ground based on the eye-box in such a way that the virtual image of the image includes a graphic object with a three-dimensional sense based on the ground. However, the present invention is not necessarily limited to this. For example, the generation of the image can be implemented on other devices such as a server or a computer. In this case, the generated image can be transmitted to the vehicle or the head-up display by communication.

[0082] As described above, in the present invention, a method of aligning different viewpoints is used to derive an image of a display element.

[0083] In this case, the first viewpoint may be a driver's viewpoint for recognizing the virtual image from the ground, the second viewpoint may be a camera viewpoint of a virtual camera, and the image may be generated by making the driver's viewpoint and the camera viewpoint consistent with each other. Below, the image derivation method of this display element is described in more detail.

[0084] More specifically, a method of causing the control unit to output three-dimensional graphics having a stereoscopic effect toward the ground using a head-up display that realizes an image located on the ground will be described in more detail below.

[0085] Figure 4 is a schematic diagram showing an image generation method of a display element that outputs a virtual image to the ground, Figure 5a as well as Figure 5b is a schematic diagram showing a method for generating a two-dimensional ground image of a three-dimensional object, Figure 6 This is a schematic diagram used to explain the theory of a method for generating a two-dimensional ground image by integrating the image of a three-dimensional object.

[0086] In the present invention, an image is presented in a vertical form on the ground from the driver's viewpoint. To this end, a method is used, that is, the two-dimensional space (ground, two-dimensional image plane) in which the image is generated in the head-up display is converted into a three-dimensional space, and a standing image is generated in the three-dimensional space, and then the generated image is converted into the two-dimensional space.

[0087] Figure 4 The steps of presenting an image generated by a display element (LCD) to a driver in a head-up display that positions the image on the ground are shown. First, if an image is output from a display element such as an LCD, the image is transformed into an image displayed on the ground by a transformation matrix M that reflects the optical system characteristics of the head-up display.

[0088] Here, the transformation matrix M is defined as a mapping matrix including an index mapped from the display element (LCD) to the top-view, and converts the image of the display element (LCD) into a top-view image seen from the upper side with the ground as a reference. That is, if the characteristics of the optical system of the head-up display are reflected in the image of the display element (LCD), it is transformed into a top-view image. Here, the characteristics of the optical system of the head-up display may have the characteristics of an imaging optical system that outputs a virtual image located on the ground.

[0089] Then, the overhead image is converted into an image in which the driver observes the image displayed on the ground in a three-dimensional viewpoint by a transformation matrix Dv related to the relationship between the driver's position and the ground on which the image is displayed. At this time, the driver's position refers to a position including eye height.

[0090] Through this image generation method, a user in a driver's seat of a vehicle recognizes an image output from the display element (LCD) on the ground, which can be defined as an image of the driver's viewpoint.

[0091] on the other hand, Figure 5a as well as Figure 5b The present invention shows a part of the process of presenting an image freely generated in a three-dimensional space not limited to the ground to the driver. In other words, the present invention shows the process of converting a three-dimensional object into a two-dimensional ground image.

[0092] First, assuming that the virtual camera 140 is observing a specific object (Obj) 141, the specific object (Obj) 141 is located in the three-dimensional space within the field of view (camera FoV) of the virtual camera. At this time, the specific object (Obj) 141 is transformed into an image 142 of the object in the three-dimensional space projected onto the ground according to the position and angle of the camera through the transformation matrix C that projects the object onto the ground according to the position and angle of the camera (or the driver's viewpoint). More specifically, the camera projection is performed using the transformation matrix C that projects the object within the field of view of the camera onto the ground, and is converted into a top-view image seen from the upper side.

[0093] In this case, since the three-dimensional space seen by the virtual camera is a distortion-free space, Figure 5a The top-view image seen from the upper side after camera projection can be a trapezoid composed of straight lines. Figure 4 The top-view image is curved differently.

[0094] Then, using the matrix Cv representing the relationship of the image of the ground observed from the virtual camera, the top-view image seen from the upper side is realized as an image of the image projected onto the ground by the virtual camera observed from a three-dimensional viewpoint. According to this method, an object in a three-dimensional space can be converted into an image projected onto the ground when observed from a three-dimensional viewpoint by a virtual camera.

[0095] The method of converting a three-dimensional object into a two-dimensional ground image described above can be incorporated into the image generation method of the head-up display of the present invention. That is, the object in the three-dimensional space is transformed into a three-dimensional figure located on the ground. More specifically, Figure 6 A method of integrating an object in a three-dimensional space into the head-up display of the present invention is shown.

[0096] Will refer to Figure 4 The image generation method of the display element which outputs a virtual image to the ground, and the reference Figure 5a as well as Figure 5bThe method of combining the objects in the three-dimensional space with the three-dimensional graphics generation is described. Through this combination, the driver can recognize the three-dimensional graphic object with a sense of three-dimensionality by viewing it through the head-up display from the camera viewpoint of the object in the three-dimensional space. Here, the object in the three-dimensional space can be generated as information to be transmitted to the driver through the head-up display. According to this method, the information to be transmitted as the three-dimensional graphic object can be output to the ground in a three-dimensional manner through the head-up display of the present invention.

[0097] For this reason, the position of the virtual camera in the three-dimensional space can be consistent with the position of the actual driver's eye box, so that the condition that the driver's viewpoint is the same as the camera's viewpoint is established.

[0098] Therefore, the two-dimensional graphic object and the three-dimensional graphic object are generated by processing different viewpoints based on the eye-box. That is, the two-dimensional graphic object is generated only considering the driver's viewpoint, but the three-dimensional graphic object is generated by making the driver's viewpoint and the camera viewpoint consistent with each other based on the eye-box.

[0099] According to the conditions, the three-dimensional graphic object output by the head-up display can be realized by forming it in the three-dimensional space based on the viewpoint of the camera and projecting it onto the two-dimensional plane of the ground based on the viewpoint of the driver.

[0100] More specifically, when using the above conditions, in order to generate a specific object (Obj) to be displayed in three-dimensional space on a head-up display observed by the driver, the image actually output from the display element (LCD) can be derived using the following formula (1).

[0101] LCD=M -1 ·D V -1 ·C V ·C·Obj (1)

[0102] Here, it can be C V =C -1 The reason is that after the appearance of a specific object (Obj) presented according to the position and angle of the virtual camera is projected onto the ground, the image presented in the camera is obtained using the same position and angle.

[0103] In addition, here can be D V =C V The reason for this is that if the virtual camera and the driver's eyes are aligned more accurately in the position of the eye zone, the image conditions for the virtual camera and the driver to observe the ground become the same.

[0104] Therefore, the formula (1) can be rearranged into the following formula (2).

[0105] LCD=M -1 ·C·Obj (2)

[0106] Thus, the method of outputting a three-dimensional graphic object to the ground using the head-up display of the present invention can ultimately become a method of outputting a specific object (Obj) corresponding to the information to be transmitted using M -1 and C to be converted into the image actually output from the display element (LCD). Here, M -1 It may be the inverse transformation matrix of the transformation matrix reflecting the optical system characteristics of the head-up display of the present invention, and C may be the transformation matrix for projecting the specific object (Obj) 141 onto the ground according to the position and angle of the camera (or the driver's viewpoint).

[0107] In this way, the control unit of the head-up display utilizes the transformation of projecting a three-dimensional object equivalent to the graphic object onto the ground and the characteristics of the optical system that transforms the image into the virtual image to generate the graphic object obtained by projecting the three-dimensional object onto the ground.

[0108] Theoretically, the present invention proposes a control method for outputting a three-dimensional graphic object to the image of a display element (LCD) on the ground using formula (2). Figures 7 to 10 c, the control method of the head-up display of the present invention is described in more detail.

[0109] Figure 7 It shows the use of Figure 6 A flowchart of an example of a theoretical data processing method, Figure 8 is equivalent to Figure 7 Figure 1 shows examples of what the images of each step actually look like. Fig. 9 It shows the use of Figure 6 A flowchart of another example of the theoretical data processing method, Fig.10 is equivalent to Fig. 9 A diagram of what the image actually looks like at each step.

[0110] First, refer to Figure 7 as well as Figure 8 , for LCD = M -1 ·C·Obj First calculate C·Obj and then compare the result value and M -1 The following describes a method for performing a dot product to derive an image to be generated on a display element (LCD). In this case, C·Obj can be derived by simulation.

[0111] Reference Figure 7, the control method of the head-up display of the present invention first selects information to be transmitted through the head-up display (S110).

[0112] The information to be transmitted may be a reference Figures 1 to 3b The information represented by the two-dimensional graphic objects and three-dimensional graphic objects described. Figure 1 As shown, if a gas station is located in front, the location of the gas station or whether the gas station exists becomes the information to be transmitted, and the gas station image can become a three-dimensional graphic object.

[0113] Next, step ( S120 ) is performed, that is, the image is outputted using an optical system that arranges a virtual image of the image formed by the display element on the ground in front of the vehicle.

[0114] In this case, in the step of outputting the image (S120), the image is generated in such a way that the virtual image of the image includes a graphic object having a three-dimensional effect based on the ground, using a first viewpoint facing the ground based on an eye-box and a second viewpoint facing the three-dimensional space on the ground based on the eye-box.

[0115] The first viewpoint may be a driver's viewpoint from which a user in a driver's seat of a vehicle observes the ground. In addition, the second viewpoint may be a camera viewpoint from which a virtual camera observes the front of the vehicle. In the step of outputting the image, the image is referenced Figure 6 As described above, the image is generated by using a method of making the eye zones of the first viewpoint and the second viewpoint consistent.

[0116] According to the diagram, the step of outputting the image (S120) may include a step of projecting (S121) and a step of transforming (S122). In these steps, an image of a three-dimensional object (or information) that will become the target of the three-dimensional graphic object is projected onto the ground, and the projected image is transformed into an image corresponding to the virtual image by using the characteristics of the optical system.

[0117] For example, the projecting step (S121) may be a step of projecting a target object corresponding to the graphic object onto the ground with the eye-friendly area as a reference. -1 The step of obtaining C·Obj in C·Obj may be a step of obtaining an image of a specific object projected onto the ground according to the position and angle of the camera.

[0118] Figure 8 An image is shown in which an object corresponding to the graphic object is projected onto the ground. In this figure, as a top-view image of the ground viewed from above, an image in which black and white areas are repeated and continuous is shown.

[0119] Next, the transformation step (S122) may be a step of transforming the image of the target object projected onto the ground into an image formed by the display element by reflecting the characteristics of the optical system in such a manner that the graphic object becomes the virtual image.

[0120] When the image derived through such processing is output to a display element, the user recognizes a three-dimensional graphic object (or a three-dimensional image) in a virtual image located on the ground ( S130 ).

[0121] More specifically, by comparing the result value of C·Obj and M -1 Perform a dot product to derive the image to be generated on the display device (LCD). Figure 8 In fact, the image to be output from the LCD has been derived through the above method, and the user recognizes a three-dimensional image from the driver's viewpoint through the head-up display of the present invention.

[0122] On the other hand, in order to execute the head-up display control method, a control unit of the head-up display may be separately provided.

[0123] Next, refer to Fig. 9 as well as Fig.10 , for LCD = M -1 ·C·Obj first finds M -1 C, and then a method of performing a dot product between the result value and Obj to derive an image to be generated in a display element (LCD) is described. In this case, C can be set by simulation, and M can be set by optical system design.

[0124] For example, refer to Fig. 9 In the control method of the head-up display of this example, the step of outputting the image (S220) may include a dot product step of a transformation matrix (S221) and a step of generating a graphic object (S222). In these steps, the control unit of the head-up display transforms the three-dimensional object into the graphic object by using the dot product of a first transformation matrix that projects a three-dimensional object (or information) equivalent to the three-dimensional graphic object onto the ground and a second transformation matrix that transforms the virtual image into the image.

[0125] For example, in the dot product step (S221) of the transformation matrix, the dot product of the first transformation matrix and the second transformation matrix is ​​first calculated. The first transformation matrix may be C, which projects the object onto the ground through the camera, and the second transformation matrix may be the inverse matrix of M, which is mapped from the display element of the head-up display to the top-view.

[0126] Next, in the step of generating the graphic object (S222), the three-dimensional object is transformed into the graphic object using the result value of the dot product.

[0127] Reference Fig.10 According to the control method of this example, the result of transforming an object (Object) located in a three-dimensional space into an image actually output to a display element can be confirmed.

[0128] As a specific example, see Fig.10 In (a), the arrow indicating a left turn is located in the three-dimensional space as a standing image of a three-dimensional object, and at its lower end, the line indicating the direction of travel is arranged horizontally with the ground on the two-dimensional plane. If the control method of this example is used to convert the image of such an object, Fig.10 The image to be output to the display element (LCD) is derived as shown in (b) of the present invention. In addition, through the head-up display of the present invention, the user can Fig.10 As shown in (c), a two-dimensional graphic object representing the driving direction and a three-dimensional graphic object representing a left turn are identified.

[0129] As described above, the head-up display and control method thereof according to the present invention can realize an image standing on the ground in a display method in which the position of the image corresponds to the ground. Thus, it is possible to provide augmented reality information in the form of a graphic object having a three-dimensional sense on the ground.

[0130] The application of the head-up display and the control method thereof as described above is not limited to the structures and methods of the embodiments described above, and all or part of the embodiments may be selectively combined to implement the above embodiments through various modifications.

Claims

1. A head-up display for a vehicle, characterized in that: include: A display element outputs light; an optical system that controls the path of the light so that an image formed by the light is output toward a light-transmitting area; and A control unit controls the display element to form the image. The optical system controls the path of the light so that a virtual image of the image is located on the ground in front of the vehicle after the image passes through the light-transmissive area. The image is generated by using a first viewpoint facing the ground with an eye-pleasing area as a reference and a second viewpoint facing the three-dimensional space on the ground with an eye-pleasing area as a reference, in such a way that a graphic object having a three-dimensional sense based on the ground is included in the virtual image of the image, The graphic object is generated by projecting a first object onto the ground using a first transformation matrix, the first object corresponds to the graphic object, and the first transformation matrix corresponds to the first viewpoint and the second viewpoint.

2. The head-up display of a vehicle according to claim 1, characterized in that: The first viewpoint is a viewpoint of a driver who recognizes the virtual image on the ground, and the second viewpoint is a viewpoint of a virtual camera. The image is generated by making the driver's viewpoint and the camera's viewpoint consistent with each other.

3. The head-up display of a vehicle according to claim 2, characterized in that: The first object is formed in the three-dimensional space with a viewpoint of the camera as a reference, and the projecting includes projecting the first object onto a two-dimensional plane of the ground with a viewpoint of the driver as a reference.

4. The head-up display of a vehicle according to claim 1, characterized in that: The control unit generates the graphic object using the characteristics of the optical system that transforms the image into the virtual image, and the first object is a three-dimensional object.

5. The head-up display of a vehicle according to claim 4, characterized in that: The control unit generates an image in which the three-dimensional object is projected onto the ground, and converts the projected image into a video image corresponding to the virtual image by utilizing the characteristics of the optical system.

6. The head-up display of a vehicle according to claim 4, characterized in that: The control unit transforms the three-dimensional object into the graphic object using the dot product of a first transformation matrix that projects the three-dimensional object onto the ground and a second transformation matrix that transforms the virtual image into the image.

7. The head-up display of a vehicle according to claim 1, characterized in that: The virtual image of the image has a two-dimensional graphic object formed on a virtual image plane corresponding to the ground. The graphic object with the stereoscopic effect is a three-dimensional graphic object standing on the virtual image plane.

8. The head-up display of a vehicle according to claim 7, characterized in that: The display element forms a single image in which the two-dimensional graphic object and the three-dimensional graphic object are generated.

9. The head-up display of a vehicle according to claim 7, characterized in that: The two-dimensional graphic object is information associated with at least one of a lane, a driving speed, and a driving direction indication, and the three-dimensional graphic object is information associated with at least one of a point of interest, a guide statement, a destination, and surrounding vehicles.

10. The head-up display of a vehicle according to claim 1, characterized in that: The light-transmitting region is formed on a combiner or a windshield.

11. A head-up display, characterized in that: include: Display component, output image; an imaging optical system that controls the path of light corresponding to the image so as to form the image through the light-transmissive region and to position a virtual image of the image on the ground in front of the vehicle; and and a control unit for controlling the display element to form the image, The control unit generates the image in such a manner that a two-dimensional graphic object corresponding to the ground and a three-dimensional graphic object having a stereoscopic effect based on the ground are arranged on the ground. The two-dimensional graphic object and the three-dimensional graphic object are generated by processing different viewpoints based on the eye-friendly area. The three-dimensional object is projected onto the ground using a first transformation matrix to generate the three-dimensional graphic object, and the three-dimensional object is equivalent to the three-dimensional graphic object.

12. The head-up display according to claim 11, characterized in that: The control unit generates the three-dimensional graphic object by utilizing the characteristics of the optical system that transforms the image into the virtual image with respect to the ground.

13. A method for controlling a head-up display, characterized in that: include: the step of selecting information to be communicated via said heads-up display; and The step of outputting the image by using an optical system that arranges a virtual image of the image formed by the display element on the ground in front of the vehicle, In the step of outputting the image, the image is generated by using a first viewpoint facing the ground with the eye area as a reference and a second viewpoint facing the three-dimensional space on the ground with the eye area as a reference, in such a way that a graphic object having a three-dimensional sense with the ground as a reference is included in the virtual image of the image, and The graphic object is generated by projecting a first object onto the ground using a first transformation matrix, the first object corresponds to the graphic object, and the first transformation matrix corresponds to the first viewpoint and the second viewpoint.

14. The control method of the head-up display according to claim 13, characterized in that: The step of outputting the image includes the step of transforming the image of the target object projected onto the ground into an image formed by the display element by reflecting the characteristics of the optical system in such a manner that the graphic object becomes the virtual image.

15. The control method of the head-up display according to claim 13, characterized in that: The first object is a three-dimensional object; and In the step of outputting the image, the graphic object is generated by using the dot product of a first transformation matrix that projects the three-dimensional object onto the ground and a second transformation matrix that transforms the virtual image into the image.

Images