Vehicle-mounted augmented reality multi-layer display device

Abstract

According to the vehicle-mounted augmented reality multi-layer display device provided by the invention, a vehicle-mounted HUD image is divided into a plurality of image layers with different image characteristics through different imaging technologies for respective imaging, and the multi-layer images are combined for unified display based on a specific sight line direction, so that a user with the sight line in the specific direction is enabled to have a user experience. When the AR-HUD display effect is accurate, the HUD images are displayed in the corresponding position or direction of the real environment in a superposed mode, when the sight line of the user moves, the superposed images are separated, and the user is prompted to adjust the sight line direction through the change of the relative position relation between the multiple layers of images so as to see the accurate AR-HUD display effect. The user eyes do not need to be tracked and monitored constantly to adjust the image display position, the monitoring and operation burden caused by human eye position change is avoided, and the structure is simple.

Description

Technical Field

[0001] The present invention relates to augmented reality head-up display technology AR-HUD, and in particular to a vehicle-mounted augmented reality multi-layer display device. Background Art

[0002] Head Up Display (HUD), also known as head-up display, uses the principle of optical reflection to project car driving assistance information, navigation information, check control information, and ADAS information in front of the windshield. HUD technology has undergone three generations of upgrades: D-HUD, W-HUD, and AR-HUD. Among them, D-HUD (Dombiner HUD) mirrors information onto a transparent panel (called Dombiner) instead of the windshield, W-HUD (WindsDreen HUD) displays image information directly on the windshield, and AR-HUD is the development of W-HUD, which fuses AR and HUD (Augmented Reality) and integrates image information into the road environment based on the position of the target object and the position of the human eye.

[0003] The relationship between the virtual image position displayed by the AR-HUD device and the viewer's eye position determines the state of imaging in the viewer's eyes. In order for the viewer to see the AR image display effect, the eye position must be within a specific range, called the eye movement range. During driving, because the position of the human eye is extremely unstable, the AR-HUD device needs to adjust the HUD display position in real time according to the tracking of the human eye, so as to realize the straight line of the human eye, HUD display, and road, requiring the device to have strong computing power. In the head-up display component and display control method of patent No. DN111433067A, the image display position in the virtual image area is corrected by obtaining the object information of the spatial object position and the viewpoint information of the driver's viewpoint position and movement amount to reduce the deviation between the object and the AR image, which requires a large computing power of the system. Summary of the invention

[0004] The purpose of the present invention is to provide a vehicle-mounted augmented reality multi-layer head-up display device, which can avoid the deviation between the AR-HUD display and the target object caused by changes in the human eye without tracking and monitoring the user's eyes, thereby reducing the requirements on the computing power of the display device and making the AR head-up display device simple in structure.

[0005] The technical solution of the present invention comprises at least two image units, wherein: A first image unit, used for generating and displaying a first layer image; a second image unit for generating a second layer image showing different imaging characteristics than the first layer image; The first image unit and the second image unit are set based on a specific sight line direction, so that a user whose sight line is in the specific direction sees the first layer image and the second layer image overlapped and displayed in the same set position or direction.

[0006] The present invention uses different imaging technologies to divide the on-vehicle HUD image into multiple layers with different image characteristics for separate imaging. Based on a specific line of sight direction, the multi-layer images are combined for unified display, so that a user whose line of sight is in the specific direction sees the HUD image overlapped and displayed at the corresponding position or direction of the real environment. When the user's line of sight moves, the overlapped images are seen to separate. The user is prompted to adjust the line of sight direction through the change in the relative position relationship between the multi-layer images so as to see the accurate AR-HUD display effect. There is no need to constantly track and monitor the user's eyes to adjust the image display position, thus avoiding the monitoring and calculation burden caused by changes in the position of the human eyes and the structure is simple. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] Figure 1 It is a structural schematic diagram of a display device of the present invention.

[0008] Figure 2 It is a schematic diagram of the structure of another display device of the present invention.

[0009] Figure 3 It is a schematic diagram of increasing the optical distance by the number of reflections in an embodiment.

[0010] Figure 4 Schematic diagram of the structure of generating parallel light by a curved mirror in an embodiment.

[0011] Figure 5 This is a schematic diagram of the image position deviation caused by the left and right movement of the line of sight.

[0012] Figure 6 This is a schematic diagram of the image position deviation caused by the upward and downward movement of the line of sight.

[0013] Figure 7 It is a schematic diagram of the relative relationship between multiple images in two layers.

[0014] Figure 8 It is a schematic diagram of the structure of the array light source of the present invention.

[0015] The reference numerals include: a first image unit 1, a second image unit 2, a windshield 3, a user's eye 4; a first imaging unit imaging component 11, a first imaging unit plane reflector 12, a first imaging unit concave mirror 13; a second imaging unit imaging component 21, a second imaging unit plane reflector 22, a second imaging single plane reflector 23, a second imaging unit concave mirror 24; a light source 25, a graticule 26, a concave mirror 27, a convex lens 28, a plane reflector 29; an array light source 5, a light emitting diode 51, a substrate 52, and a collimating lens 53. DETAILED DESCRIPTION

[0016] In the present invention, the first image unit and the second image unit are relative, and refer to two image units with different imaging technologies. Similarly, the first layer image and the second layer image refer to two images with different characteristics. When there are multiple image units, a certain image unit can be used as a first image unit to generate and display the first layer image, and can also be used as a second image unit to generate and display the second layer image. For example, in a display device with three image units, each image unit has different imaging characteristics. A certain image can be used as a first image unit relative to one of the image units, and can be used as a second image unit relative to another image unit. The following is an example of two imaging units.

[0017] exist Figure 1 In the structural schematic diagram of the head-up display device shown, a first image unit 1 and a second image unit 2 are included, and the first image unit 1 and the second image unit 2 are both arranged behind a windshield 3, wherein the light output by the first image unit 1 directly enters the user's eye 4, and generates a first layer of real image at a close distance behind the windshield, and the light output by the second image unit 2 is projected onto the windshield 3, enters the user's eye 4 after reflection, and generates a second layer of virtual image at a long distance in front of the windshield.

[0018] The first image unit includes indicator lights set on the windshield, or signs posted on the windshield, or marks drawn on the windshield, etc. For example, a row of indicator lights is set along a specific direction at a set position of the windshield, and the lights are on to display a straight line segment, or a straight line segment mark is posted at a set position of the windshield along a specific direction, or a straight line segment is drawn at a set position of the windshield along a specific direction. The first image unit forms a first layer of real image at a relatively close distance through real objects such as indicator lights, signs, or marks, and the real image is displayed behind the windshield. In order to avoid blocking the driver's line of sight, the first image unit is set at the bottom edge of the windshield, and the first image unit can also be set on the dashboard behind the windshield.

[0019] The second image unit includes a HUD device known in the art, and generates image information through the PGU and waveguide imaging system of the HUD. The waveguide projects the image information onto the windshield, and the image information is reflected by the windshield into the user's eyes. Usually, the imaging display position of the W-HUD is above the edge of the hood, and the VID at this distance is usually 2m~2.5m in front of the windshield. Of course, the distance of the second layer of virtual image generated by the HUD can be farther or closer.

[0020] In some specific scenarios, the second image unit may also include an array type visible light source, such as Figure 8 The illustrated light source is a 1×n array light source 5, in which a row of LED light emitting diodes 51 are arranged on a substrate 52. An optical control component 53 is provided on the light emitting diode 51 to control the direction of light emission. Since the windshield 3 is not a plane, it is necessary to adjust the position of each light emitting diode 51 on the substrate 52 according to the curvature of the windshield 3, and arrange them in a corresponding manner to eliminate the imaging distortion caused by the curvature of the windshield 3, so that the light emitted by each row and column of the light source 51 is irradiated on the windshield 3 and displayed as a straight line segment. The optical control component 53 includes a collimating lens or a collimating film, which adjusts the direction of light emission from the light source 51, and can focus the light within the range of plus or minus 35 degrees of the normal line of the collimating film. Obviously, the illustrated array light source 5 can also be an m×n multi-array, where m and n can both be greater than or equal to 2.

[0021] The image formed by the array light source not only avoids the problem of AR-HUD being unable to produce high-intensity light due to light loss caused by optical structure limitations, but also controls the projection position of the image through fixed settings, making the device structure simpler.

[0022] It can be seen that the imaging characteristics of the first image unit and the second image unit are not only different in imaging distance, but also in mirror media. The mirror medium of the first image unit is itself, and the mirror medium of the second image unit is the windshield.

[0023] In addition, the first image unit and the second image unit can also generate images with different characteristics through different light properties. The first image unit outputs parallel light directly into the user's eyes, generating a first layer of virtual image at a long distance in the user's eyes without reflection through the windshield. The image formed by the parallel light in the user's eyes is located at an infinite distance, which is generally considered to be greater than 5 meters. Even if the position of the human eye moves, the position of the image formed by the parallel light will not change with the movement of the human eye. In other words, the diffuse light output by the second image unit is reflected by the windshield into the user's eyes to form a close-range virtual image. The first image unit obtains a virtual image distance farther than the second image unit by outputting parallel light directly into the user's eyes.

[0024] Parallel light can be produced by curved mirrors, such as Figure 3 As shown. Figure 4 The curved mirror in (a) is a concave mirror 27. Concave mirror 27 is a semi-transparent and semi-reflective curved mirror, and one or more layers of semi-transparent film are coated on its concave surface, so that this surface can transmit part of the light and reflect part of the light at the same time. A graticule 26 is set at the focal position of the concave surface, and behind the graticule 26 is a visible light source 25. The graticule 26 includes circular, square, linear, triangular and other graphics, which form corresponding guiding images under the irradiation of light; preferably, the graphics have directionality, such as straight line segments and triangles. Light source 25 can be a light-emitting diode LED. Furthermore, a negative crescent lens and the concave surface of the spherical curved mirror can be used between the concave mirror 27 and the light source 25 to form a Mangin mirror system to improve the collimation of the reflected light.

[0025] After the visible light source 25 passes through the graticule 26, the light is irradiated onto the concave surface of the curved mirror 27, and then the concave surface reflects the light into parallel light and enters the user's eyes. At this time, the human eye cannot recognize the true position of the graticule 26, and can only see its reflected virtual image, and think that it is on the reverse extension line of the light entering the human eye. Since the image light of the graticule 26 entering the user's eyes is a beam of parallel light, the user will feel that the image is located at infinity.

[0026] Because the light reflected by concave mirror 27 into the user's eyes is parallel light, the user can see the virtual image of graticule 26 no matter where he is, up, down, left, or right, and the virtual image is displayed at the same position, that is, the position of the virtual image does not change with the movement of the user's eyes.

[0027] It can be seen that the semi-transparent and semi-reflective concave mirror 27 can form a virtual image of the graticule 26 by reflecting light, and the user can see the environmental image behind the concave mirror 27 through the transmitted light. Obviously, the concave mirror 27 can also be a concave total reflection mirror, with a reflective coating on the concave surface to reflect all the light irradiated on the concave surface to form a virtual image of the graticule 27, but the user cannot see the environmental image behind the concave mirror 27.

[0028] exist Figure 3The curved mirror shown in (b) is a convex lens 28. Convex lens 28 is a collimating lens. A graticule 26 is arranged at the focal position of convex lens 28, and a visible light source 25 is arranged behind it. The image light emitted by light source 25 through graticule 26 can be converted into parallel light output through convex lens 28, and then reflected to the user's eyes through plane mirror 29. Plane mirror 29 is coated with a reflective coating, so that the user can see the virtual image of graticule 26. Obviously, if the plane reflector 29 is a semi-mirror, that is, it reflects a part of the light while transmitting a part of the light, then the user can see the virtual image of graticule 26 and the image behind plane mirror 29 at the same time. If the plane mirror 29 only reflects but does not transmit the incident light, then the user can only see the virtual image of graticule 26, but cannot see the image behind plane mirror 28.

[0029] Laser can also be regarded as a parallel light, but the difference is that the safety laser can be used as a light source 25, and can directly form a second image in conjunction with the reticle 26. For example, a reticle 26 engraved with a triangular pattern is set in front of the laser, and the laser is directly irradiated to the plane mirror and then reflected to the user's eyes to form a triangular image.

[0030] It should be noted that although the virtual image generated by the above-mentioned parallel light and the real image of objects such as indicator lights, signs or markings do not reflect through the windshield and directly enter the user's eyes, the imaging characteristics of the two are quite different.

[0031] The above-mentioned indicator lights, signs or markings and other objects form a real image at a close distance behind the windshield, and their corresponding display position on the road surface background changes with the movement of the user's eyes. In other words, when the user's eye position or direction is different, the real image seen is attached to different positions on the road surface, the imaging distance is the shortest, and it can only be used as a layer of image at a close distance. The above-mentioned parallel light forms a virtual image at a long distance in the user's eyes, and its display position on the road surface background does not change with the movement of the user's eyes. In other words, when the user's eye position or direction is different, the virtual image seen is attached to the same position on the road surface, the imaging distance is far, and it can be used as a layer of image at a long distance.

[0032] It can be seen that the image generated by parallel light not only does not change its display position as the user's eyes move, but also has a long imaging distance and enters the user's eyes directly through the optical mirror of the imaging system without being reflected by the windshield. That is, its mirror medium is also different.

[0033] exist Figure 2In another schematic diagram of the structure of a head-up display device shown, the image light output by the first image unit 1 and the second image unit 2 enters the user's eyes through the reflection of the windshield 3, and forms a virtual image in front of the windshield 3. The first image unit 1 and the second image unit 2 include a common HUD device, and the imaging characteristics thereof can be adjusted by increasing the optical distance of the second image unit 2 or increasing the curvature of the curved mirror, so as to obtain an image distance farther than that of the first image unit 1. The specific scheme is as follows.

[0034] The commonly used optical architecture in HUD is a single free-form mirror plus a plane mirror or a double free-form surface. Regardless of whether it is single or double, under the same FOV, increasing the system optical distance, such as the distance between two lenses and / or the distance between the lens and the PGU, can make the imaging distance VID farther, but it also causes the volume of the HUD to increase. To this end, the imaging distance of the second layer image unit can be increased by increasing the number of reflections or the reflection distance of the image-generating light.

[0035] like Figure 4 The figure shows a schematic diagram of increasing the optical distance by the number of reflections, where: Figure 4 (a) is a schematic diagram of the optical path structure of the first image unit 1, Figure 4 (b) is a schematic diagram of the optical path structure of the second image unit 2. Figure 4 In (a), the plane reflector 12 is set at one focal length of the curved mirror 13, and the light emitted by the PGU imaging component 11 is reflected by the plane mirror 12 to the curved mirror 13, and then enters the user's eye 4; Figure 4 In (b), the plane reflector 23 is set at one focal length of the curved mirror 24, and a plane reflector 22 is set in front of the plane reflector 23. The light emitted by the PGU imaging component 21 is first projected to the plane mirror 22, then reflected by the plane mirror 23, and finally to the curved mirror 24, and then enters the user's eye 4. It can be seen that in Figure 4 (b) The image light emitted by the PGU imaging component 21 is Figure 4 The distance traveled by the image light emitted by the PGU imaging component 11 in (a) is twice the focal length of the curved mirror 24, so that the distance of the second layer image formed in the user's eyes is farther than the distance of the first layer image. Obviously, it is also possible to obtain a longer imaging distance of the second layer image by increasing the reflection distance between the imaging component 21 and the plane reflector 23 without adding the plane reflector 22.

[0036] In addition, under the condition of fixed optical spacing, an image distance farther than the first layer image can be obtained by increasing the magnification of the curved mirror of the second image unit, or a hyperbolic mirror can be used, which can help achieve a larger VID in a smaller HUD volume.

[0037] It can be seen that the difference between the imaging characteristics of the first image unit and the second image unit is that the imaging distances are different, and two images with different imaging distances are generated respectively.

[0038] The first image unit and the second image unit are set according to a specific line of sight direction, so that the user whose line of sight is in the specific direction can see the first layer image and the second layer image overlapped and displayed in the same set position or direction. That is to say, the two layers of images are overlapped and displayed in a specific position or direction in the real environment to achieve the enhanced effect of AR-HUD. Obviously, the specific line of sight direction should be the direction that the driver is accustomed to looking at. Each driver has a different height and sitting posture, and his line of sight direction is also different. In order to provide the applicability of the display device, the display device may also include an adjustment unit, which can adjust the line of sight direction by adjusting the left and right, front and back position, or height, or inclination of the display device to adapt to different drivers.

[0039] When the user's line of sight moves, the overlapping images will separate. The change in the relative position relationship between the multiple layers of images from overlapping to separation prompts the user to adjust the direction of sight to see the accurate AR-HUD display effect.

[0040] exist Figure 5 The diagram shows the left-right movement of the line of sight and the change of the image display. When the line of sight is in the middle position O, the image A and the image B are overlapped and displayed in the same direction D, corresponding to the target position. When the line of sight deviates from the specific direction O and moves left and right to O1 and O2, the user sees that the image A and the image B are offset left and right as they deviate from the direction D. Since the imaging distance of image A is different from that of image B, the distances of the two from the direction D are also different. The closer the imaging distance, the greater the deviation distance, and the farther the imaging distance, the smaller the deviation distance. As a result, the user sees a distance L1 and L2 between the left and right images A and B. The change process of images A and B from overlapping to left and right separation indicates the process of the user's line of sight moving left and right.

[0041] Figure 6 The figure shows the image deviation process seen by the user when the line of sight moves up and down. When the line of sight is at the middle position O, the image A and the image B are seen to overlap and display in the same direction D, corresponding to the target position. When the line of sight deviates from the specific direction O and moves up and down to O1 and O2, the user sees that the image A and the image B are offset up and down as they deviate from the direction D. Since the image distance of image A is different from the image distance of image B, the distances of the two deviations from the direction D are also different. The image distance of image A is small and the deviation distance is large, while the image distance of image B is large and the deviation distance is small. As a result, the user sees a distance L1 and L2 between the overlapping objects A and B. The change process of images A and B from overlapping to separation indicates the process of the user's line of sight moving up and down.

[0042] It can be seen that when the user's line of sight is in a specific direction, the images A and B are displayed together when viewed from the front, which is a planar feature without depth information. When the user's line of sight moves to the left, right, top, or bottom of the image, the image has depth information with spatial stereoscopic characteristics, that is, the misalignment or deviation between images A and B is seen. Through the change of the two-layer image from the planar characteristic to the spatial stereoscopic characteristic, the user is prompted to adjust the deviated line of sight and return to the front angle to obtain the best AR display effect.

[0043] The greater the difference in image distance between image A and image B, the greater the gap between images A and B when the user moves his line of sight, that is, the more sensitive he is to changes in the direction of his line of sight. The image distance difference determines the movable range of the line of sight. The larger the image distance difference corresponds to the smaller the eye movement range, and the smaller the deviation between the HUD image display position seen by the user and the target object. It can be seen that the image distance difference between different layers is crucial. In addition, the smaller the overlap of image A and image B in a specific direction, the smaller the corresponding eye movement range.

[0044] It should be noted that if one of image A or image B is an image generated by parallel light and the other is an image generated by diffuse light, during the movement of the user's line of sight, the position of the image generated by the parallel light beam will remain unchanged, and only the position of the image generated by the diffuse light beam will move. This makes the deviation distance between the two images larger under the same line of sight movement distance, which is more conducive to AR control. Even if the image distances of the two images are the same and the image distance difference is zero, it corresponds to an eye movement range, and the AR display effect of the present invention can still be achieved.

[0045] In addition, the images generated by the first image unit and the second image unit may have different color characteristics. When the user's line of sight is in a specific direction, different colors overlap to display a mixed color. When the line of sight shifts, the color changes. The position of the different colors separated from the image can be used to determine the direction of the line of sight shift. For example, Figure 5 In the example, the color of image A is set to red, and the color of image B is set to green. When the user's line of sight is at O, he sees the yellow color of images A and B overlapping each other. When the user's line of sight shifts to the right to O2, he sees the yellow image separated into left red and right green. When the user's line of sight shifts to the left to O1, he sees the yellow image separated into right red and left green. The position of the red color indicates the direction in which the user's line of sight should be shifted. Similarly, in Figure 6 In the figure, when the red image A is at the top, it is judged that the line of sight has deviated downward, and the user is prompted to shift upward to correct the line of sight. When the red image A is at the bottom, it is judged that the line of sight has deviated upward, and the user is prompted to shift downward to correct the line of sight.

[0046] For the two-layer images generated by the parallel light beam and the diffuse light beam, even in the absence of image distance difference, the change in color can prompt the user that the direction of sight has deviated, and the direction of deviation can be determined based on the relative position of the color of a single image.

[0047] From the color change process, we can see that the closer the image is to the user's eyes, the greater the offset distance of the image displayed during the change of the user's line of sight, that is, the more sensitive it is to the change of the user's line of sight. The farther the image is from the user's eyes, the smaller the offset distance of the image displayed during the change of the user's line of sight, that is, the less affected by the change of the user's line of sight. Therefore, the layer at a close distance can be used as a calibration layer to correct the user's line of sight, and the layer farther from the user's eyes can be used as a target layer to align the target object. Multiple images corresponding to multiple targets can be generated simultaneously on the target layer, and only calibration images are generated on the calibration layer.

[0048] Further, the first image unit may generate a first layer image having one or more images, and the second image unit may generate a second layer image having one or more images, such as Figure 7 As shown, the first image unit generates a calibration layer image with only one image A, and the second image unit generates a target layer image unit containing three images B1, B2, and B3. Image A and image B2 are set based on a specific line of sight, so that a user whose line of sight is in a specific direction sees image A and image B2 overlap and display together, so that image A and image B2 constitute the control factor of the line of sight direction of the display device, and image B1 and image B3 are in an associated position. Through the single image A of the calibration layer and the single image B2 of the target layer, the AR display effect of the overall image of multiple images B1, B2, and B3 of the second layer can be controlled.

[0049] The present invention divides the image displayed by the vehicle HUD into at least two images, and displays them relatively independently in different image layers by using different imaging technologies between the images, and overlaps them in a mutually related manner through a common sight line direction. The following example illustrates a specific application scenario.

[0050] The first application scenario example is the vehicle body boundary line augmented reality head-up display device.

[0051] When passing through narrow roads or meeting other vehicles, the driver needs to slow down or stop to observe the road conditions to determine whether the vehicle can pass. Due to the perspective, the driver can only see the front of the vehicle and the road surface, but not the vehicle body. Therefore, the driver needs to estimate the width of the vehicle body. If the estimation is inaccurate, it will affect safe passage. For inexperienced drivers, it is even more difficult to accurately estimate the width of the vehicle body.

[0052] The vehicle body boundary line display device of the present invention generates line segment images extending forward and backward respectively through the first image unit and the second image unit, and the display direction is consistent with the forward extension line of the vehicle body boundary line on the road surface. Based on a specific line of sight direction, the two line segments are overlapped to be displayed in a straight line direction and fit at the set position of the forward extension line of the vehicle body boundary on the road surface. The first image unit and the second image unit are further adjusted to make the front end of the line segment of the near-distance layer close to the end of the line segment of the far-distance layer, so that the head and tail are close or connected, or even overlapped for better display.

[0053] When the user's line of sight moves left and right, he will see that the overlapping straight line segment is staggered into two line segments. When the user sees that the body line image changes from one straight line segment to two, or the color of the body line image changes, it means that his line of sight is not in a specific direction. At this time, the user needs to adjust the line of sight to see an image with only one body boundary line, or change the image color back to the original color. The up and down movement of the driver's line of sight will not affect the AR display effect of the body line.

[0054] If there is a layer of image that is a line segment generated by parallel light, the image unit can be set based on a specific line of sight direction so that it is displayed fit on the boundary line between the road surface and the car body. No matter how the user's line of sight moves, the line segment will be seen indicating the boundary of the car body on the road surface.

[0055] For a certain vehicle itself, the body width and boundary line position are fixed, and the projection position of the body boundary line image on the windshield is also fixed. Obviously, during normal driving of the vehicle, the driver will constantly change his sitting posture or move his head, which will cause the line of sight to deviate. However, when meeting other vehicles on a narrow road or driving at a low speed, the driver needs to focus his line of sight on the left or right direction of the vehicle body when observing the road ahead. At this time, the line of sight for viewing the virtual image of the body boundary line is within the driver's gaze range. Therefore, the specific line of sight direction of the body boundary line display device is set to be basically the same as the line of sight direction of the driver when driving at this time. When the driver carefully observes the road surface from left to right, he can naturally see the body boundary line displayed on the road surface. The position of the boundary line image projected by the display device onto the windshield is fixed and does not require calculation and adjustment functions, nor does it require a monitoring and tracking system for the user's eyes, making the device structure simple.

[0056] The vehicle body boundary line is divided into left and right sides, and vehicle body line display devices need to be set on the left and right sides of the vehicle body respectively. Each vehicle body line display device has a respective sight direction. By using any one of them alone, the driver can view the vehicle body lines on the road surface on the left and right sides respectively.

[0057] The second application scenario example is the vehicle centerline augmented reality head-up display device.

[0058] The vehicle body centerline display device generates line segment images extending forward and backward respectively through the first image unit and the second image unit, and the display direction is consistent with the forward extension line of the vehicle body centerline on the road surface. Based on a specific line of sight direction, the two line segments are overlapped to make them displayed in a straight line direction and fit into the set position of the forward extension line of the vehicle body centerline on the road surface.

[0059] The centerline of the vehicle body can be combined with the boundary line, and two lane line display devices can be set on the left and right sides based on the line of sight of the centerline of the vehicle body, so that the driver can see the overlapping centerline of the vehicle body while looking forward and can see the lane lines on the left and right sides displayed at the correct position on the road surface. Figure 7 In the embodiment, image A and image B1 are displayed on the forward extension line of the center axis of the vehicle body on the road surface to become the center line of the vehicle body, which is used to control the user's line of sight. The display positions of line segment images B1 and B3 are fitted to the boundary line of the vehicle body on the road surface. Through the line of sight constrained by the center line of the vehicle body, the boundary lines on both sides of the vehicle body on the road surface can be seen at the same time, thereby obtaining an AR display image of the boundary line of the vehicle body in a long-distance and wide scene.

[0060] The third application scenario example is the lane keeping line augmented reality head-up display device.

[0061] If the display position of the vehicle boundary line image is adjusted outward by a certain distance, such as 25 cm, so that it fits with the lane line on the road, it can become a lane keeping line image. When the vehicle is driving, the driver can keep his sight in a specific direction and make the lane keeping line fit with the lane line on the road, so that the vehicle can keep driving in the middle of the lane without deviating.

[0062] If the vehicle visually keeps a certain distance between the left vehicle body boundary line image and the corresponding lane line on the road surface, the vehicle can also be kept in the middle of the lane. That is, by whether a lane keeping line image fits with a real image of a lane line on the same side of the road surface, it can be determined whether the vehicle is driving in the middle of the lane. For vehicles with the driver's seat on the left, in order to facilitate the driver's observation, a lane keeping line display device can be set only on the left side of the vehicle, or the vehicle body boundary line display device on the left side of the vehicle can be used to achieve lane keeping.

[0063] Furthermore, the lane keeping line display device and the vehicle body boundary line display device can be integrated together, and the vehicle body line can be displayed when driving at a low speed on a narrow road, and the lane line can be displayed when driving at a high speed.

[0064] For roads without lane lines, it is impossible to rely on the lane lines on the road surface to keep the vehicle in the middle of the lane. At this time, the vehicle's driving direction can be maintained by combining the center line of the vehicle body and the boundary line of the vehicle body. The distance between the left and right body boundary lines is used as the vehicle body width, and the driver's line of sight is controlled through the AR display of the center line of the vehicle body. When the driver keeps the center line of the vehicle body as a straight line segment image or displays overlapping colors, he controls the direction of the vehicle's driving so that the road surface between the two body boundary lines is in the middle of the road, which can ensure that the vehicle is driving in the middle of the road.

[0065] For a vehicle traveling in the middle of a lane, the position of the vehicle body boundary line is fixed, and the lane line on the road that the driver sees is also fixed. In other words, the relative position relationship between the vehicle body boundary line and the lane keeping line is fixed, that is, the projection position of the vehicle body boundary line image on the windshield and the projection position of the lane keeping line image on the windshield can be fixed. The difference is that the vehicle body line is suitable for close-range observation at low speed or parking, corresponding to the vehicle body boundary range on the road surface at close distance in front of the vehicle head, and the lane keeping line is suitable for long-range observation at high speed, and fits the lane line on the road surface at a long distance. For the same driver, the vehicle body line or lane keeping line can be observed in different scenes and from different sight directions.

[0066] Furthermore, the combined display device may also include a control switch to selectively control the display of the vehicle body boundary line and the lane keeping line. When the user is meeting or driving on a narrow road, the lane keeping line can be turned off and only the display of the vehicle body line can be turned on; if the user is driving at high speed, the vehicle body line can be turned off and the lane keeping line can be turned on. At this time, the driver looks at the road ahead, and the driving line of sight is in the same direction as the line of sight for observing the lane keeping line. The up and down movement of the line of sight will not affect the AR display effect of the lane indicator line.

[0067] The fourth application scenario example is the navigation turn indicator line augmented reality head-up display device.

[0068] The navigation turn display device generates a first layer of short-range left and right extended line segment images through the first image unit, and generates a second layer of long-range left and right extended line segment images through the second image unit. If the front turning intersection is perpendicular to the current road direction, the display direction of the line segment image can be perpendicular to the forward extension line of the center axis of the vehicle body, for example, it can be a horizontal straight line segment with an arrow. If the turning intersection is oblique to the current lane direction, the display direction of the left and right extended line segment images can also be oblique to the forward extension line of the center axis of the vehicle body, so as to be consistent with the direction of the turning intersection.

[0069] Based on a specific sight direction, the first image unit and the second image unit are set so that the user whose sight is in the specific direction sees the two line segments overlap and are displayed as a line segment with an indicating arrow. The first layer image unit and the second layer image unit are synchronously controlled by the driving unit so that the two line segments move up and down synchronously together. The navigation turn display device synchronously adjusts the projection position of the two line segment images on the windshield according to the obtained distance of the front turn intersection so that the horizontal straight line segment with the turn arrow continues to fit the target intersection position.

[0070] It should be noted that, during vehicle driving, the existing AR-HUD navigation adjusts the position of the turn indicator arrow by tracking and monitoring the user's eye position to ensure that the user sees the turn indicator arrow continuously attached to the target intersection. In the present invention, it is the opposite. The navigation display device only continuously adjusts the display position of the turn indicator arrow according to the turning distance. The user needs to constantly track the position of the turn indicator arrow and adjust the line of sight up and down to see the turn indicator arrow continuously attached to the turning intersection. If the user's line of sight shifts up and down, the user will see that one overlapping turn indicator arrow becomes two up and down, or the image color changes to monochrome. At this time, the user needs to adjust the line of sight up and down to return to the direction of seeing only one turn indicator arrow, and the left and right movement of the line of sight will not affect the AR display effect of the turn indicator. It can be seen that the present invention allows the turn indicator arrow to continuously fit the turning intersection by adjusting the line of sight of the user, and does not require the navigation display device to track, monitor and calculate the user's eyes, making its structure simple.

[0071] It is worth noting that as the vehicle gradually approaches the turning intersection, the driver's line of sight focused on the turning intersection gradually moves downward, just in line with the line of sight following the turn indicator arrow. As long as the driver keeps staring at the turning intersection, he can see that the turn indicator arrow is always attached to the turning intersection.

[0072] The fifth application scenario example is the safety distance line augmented reality head-up display device.

[0073] The safe distance is usually one thousandth of the vehicle speed. For example, if the speed is 50km / h, the distance should be 50m. If the speed is 120km / h, the distance should be 120m. This distance is the safe distance. Moreover, when the speed is less than 100km / h, people have enough reaction time. Even if the front car stops suddenly (rear-end collision or hitting an obstacle), the rear car can also brake. Therefore, it can be called the safe reaction time.

[0074] The safe distance display device of the present invention generates a first layer of short-distance left-right extended line segment images through a first image unit, and generates a second layer of long-distance left-right extended line segment images through a second image unit. The line segment images can be horizontal straight line segments whose display direction is perpendicular to the forward extension line of the center axis of the vehicle body. Based on a specific sight line direction, the first image unit and the second image unit are set so that the user whose sight line is in the specific direction can see the two line segments overlap and display as a straight line segment. The vehicle speed acquisition unit obtains the vehicle speed information, and the first layer image unit and the second layer image unit are synchronously controlled by the driving unit so that the two straight line segments move up and down synchronously together. The safe distance display device synchronously adjusts the projection positions of the two line segment images on the windshield according to the acquired vehicle speed, so that the user whose sight line is in the specific direction can see that the display position of the horizontal straight line segment on the road surface corresponds to the vehicle speed. In the process of accelerating or decelerating the vehicle, the display position of the safe distance line on the road surface also moves forward or backward. In this process, the driver's sight line direction naturally moves synchronously with the safe distance line.

[0075] The sixth application scenario example - braking distance line augmented reality head-up display device.

[0076] The braking distance line display device generates a first layer of short-distance left-right extended line segment images through the first image unit, and generates a second layer of long-distance left-right extended line segment images through the second image unit. The line segment image can be a horizontal straight line segment whose display direction is perpendicular to the forward extension line of the center axis of the vehicle body. Based on a specific line of sight direction, the first image unit and the second image unit are set so that the user whose line of sight is in the specific direction sees the two line segments overlap and are displayed as a straight line segment. The vehicle speed and deceleration are obtained through the driving information acquisition unit, and the first layer image unit and the second layer image unit are synchronously controlled by the drive unit so that the two straight line segments move up and down synchronously together.

[0077] According to physics, the current speed and deceleration determine the braking distance of the vehicle. The braking distance line display device calculates the braking distance based on the acquired speed and deceleration, and synchronously adjusts the projection position of the two line segment images on the windshield from time to time, so that users with their sight in a specific direction can see that the display position of the braking distance line image on the road surface corresponds to the current speed and deceleration. Obviously, during the vehicle braking process, the display position of the braking distance line image gradually approaches the vehicle body until it stops in front of the front of the vehicle. During this process, the user's sight direction also moves synchronously with the braking distance line.

[0078] It can be seen that in the application of scenes such as narrow road meeting, lane keeping, navigation steering, safe following and braking, the present invention integrates the display of AR with the sight action during vehicle driving. The display device and the driver both realize the reality enhancement effect of the vehicle-mounted HUD device based on a common sight direction. That is to say, the display device only generates the display image based on a predetermined sight direction and no longer adjusts with the user's eyes. This predetermined sight direction is the sight direction of vehicle driving in various scenes. It can be fixed, such as the vehicle body boundary line and lane keeping line. The AR display and the driving sight are fixed in the same direction; it can also be variable, such as the navigation steering indicator line, the safe vehicle distance line, and the braking distance line. The AR display changes are synchronized with the driving sight change. In addition, this sight change is only in a single direction of left or right or up and down. During driving, the user can see the augmented reality image displayed on the exact position of the road surface through the natural coordination of the sight direction, and the accuracy can be completely controlled and adjusted by the user. The display device does not require additional calculation and adjustment functions, and does not need to monitor and track the user's eye system, making the device structure simple.

Claims

1. A vehicle-mounted augmented reality multi-layer display device, comprising at least two image units, wherein: A first image unit, used for generating and displaying a first layer image; A second image unit, configured to generate a second layer image showing imaging characteristics different from those of the first layer image; The first image unit and the second image unit are set based on a specific sight line direction, so that a user whose sight line is in the specific direction sees the first layer image and the second layer image overlapped and displayed in the same set position or direction.

2. The device according to claim 1, characterized in that: The image unit includes an array light source.

3. The device according to claim 1, characterized in that: The different imaging characteristics include the following: The generated images have different image distances, or the generated images have different display position variability, or the generated images have different colors.

4. The device according to claim 3, characterized in that: The different image distances include the following: The image distances of images generated by different optical distances are different, or the image distances of images generated by curved mirrors with different magnifications are different, or the image distances of images generated by single-curved mirrors and hyperbolic mirrors are different, or the image distances of images displayed by the front and rear isolation of the windshield are different.

5. The device according to claim 3, characterized in that: The different display position variability includes that the display position of the image generated by parallel light does not change with the movement of the user's eyes and the display position of the image generated by diffuse light changes with the movement of the user's eyes.

6. The device according to claim 1 or 2 or 3 or 4 or 5, characterized in that: The first layer image generated by the first image unit and the second layer image generated by the second image unit include one or more line segment images extending forward and backward, and the display direction thereof is consistent with the forward extension line of the vehicle body boundary line on the road surface; or The first layer image generated by the first image unit and the second layer image generated by the second image unit include one or more line segment images extending left and right, and their display directions are perpendicular or oblique to the forward extension line of the center axis of the vehicle body on the road surface.

7. The device according to claim 6, characterized in that: The line segment images extending forward and backward include the following: The vehicle body boundary line image is displayed on the forward extension line of the vehicle body boundary on the road surface; or The vehicle centerline image is displayed on the extension line of the vehicle centerline on the road surface; or The lane keeping line image is displayed on the lane line on the road surface or on the forward extension line of the vehicle body boundary on the road surface.

8. The device according to claim 1 or 7, characterized in that: The device comprises a control switch for selectively controlling the display of a vehicle body boundary line image, a vehicle body center line image, and a lane keeping line image.

9. The device according to claim 6, characterized in that: The line segment images extending left and right include the following: Steering An image of a guide line displayed on the road surface at a turning intersection; or An image of a safe distance line displayed at a set position on the road surface in front of the vehicle; or The braking distance line graphic is displayed at a set position on the road surface in front of the vehicle.

10. The device according to claim 1 or 9, characterized in that: The device comprises a control unit and a driving information acquisition unit. According to the vehicle driving information acquired by the information acquisition unit, the control unit synchronously controls the first image unit and the second image unit to adjust the display position of the line segment image generated by them on the road surface.

Images