Head-mounted display device and see-through display method thereof

Abstract

The application discloses a kind of head-mounted display device and its see-through display method, comprising: collecting external environment image;The mirror image processing is carried out to the external environment image collected;Add secondary display screen on head-mounted display device, and the image information after mirror image is displayed on secondary display screen;The image displayed by secondary display screen is secondarily mirror imaged, and the direction of light path is changed, to be shot to the position that human eye can see after the head-mounted display device is worn on human body, deviates from the position that level direction can be seen.This application realizes see-through display function on head-mounted display device, user can view external environment without taking off head-mounted display device, and realizes the use effect of direct vision watching program, oblique vision insight external environment, it is convenient and practical, while enriching equipment function, program immersion is not disturbed by external environment image display influence, improve user experience.

Description

Technical Field

[0001] This invention belongs to the field of head-mounted display technology, specifically, it relates to a method for displaying an image of the external environment on a head-mounted display device. Background Technology

[0002] Head-mounted displays (HMDs), also known as head-mounted displays, are video playback devices that use an optical system to magnify images on an ultra-micro display screen and project the images onto the user's retina, thus presenting a large-screen image display effect to the user's eyes. They can achieve different display effects such as virtual reality (VR), augmented reality (AR), and mixed reality (MR).

[0003] Traditional head-mounted displays prevent users from seeing their external environment while wearing them. Users must remove the display to move or locate objects. This frequent removal and re-wearing of the display is inconvenient and detracts from the user experience.

[0004] To allow users to observe their external environment while wearing a head-mounted display (HUD), the current solution involves adding a camera to the HUD to capture images of the surrounding environment. These images are then displayed on a screen within the HUD for the user to view. There are two main display methods: One method is display switching. This involves using the same screen to display normal video content (program video or game video) and external environmental footage in shifts. When the screen displays the external environmental footage, the normal video information needs to be turned off. This display method causes an interruption in the display of normal video information, preventing users from watching the complete video content and affecting their viewing experience.

[0005] Secondly, there's the overlay display. This involves displaying the normal video content and the external environment simultaneously on the same screen. Because of the overlapping images, users cannot enjoy the normal video content in high quality, nor can they clearly observe the external environment; therefore, the user experience is far from ideal. Summary of the Invention

[0006] This invention proposes a perspective display method for head-mounted display devices to solve the problem that users cannot observe external environmental images while wearing head-mounted display devices without interrupting normal video playback or maintaining clear video display.

[0007] To solve the above-mentioned technical problems, the present invention adopts the following technical solution: In one aspect, the present invention provides a perspective display method for a head-mounted display device, comprising: Acquire images of the external environment; The acquired images of the external environment are mirrored. A secondary display screen is added to the head-mounted display device to display the mirrored image information on the secondary display screen; The image displayed on the secondary display screen is mirrored a second time, and the direction of the light path is changed to direct the light towards a designated location; the designated location is the position that the human eye can see when the head-mounted display device is worn, deviating from the eye-level direction.

[0008] In some embodiments of this application, a reflector can be used to mirror and reflect the image displayed on the secondary display screen, so that the reflected light is directed toward the designated location.

[0009] In some embodiments of this application, to enhance the user's immersion, the head-mounted display device can be configured to emit video light from its main display screen into the wearer's eyes at eye level. Simultaneously, the video light emitted from the main display screen and the secondary display screen is separated, preventing the user's eyes from seeing the video light directly emitted from the secondary display screen. Since the video light emitted directly from the secondary display screen does not mix with the video light emitted from the main display screen, it ensures that the user will not see the image displayed on the secondary display screen when looking at the user at eye level, thus preventing external environmental images from interfering with the user's normal viewing of programs or game videos.

[0010] In some embodiments of this application, in order to minimize the interference of external environment image display on normal video playback, the image size displayed on the secondary display screen can be configured to be smaller than the image size displayed on the main display screen; at the same time, the designated position is configured to be the position where the external environment image after secondary mirroring can be viewed when the human eye is rotated to the lower side, thereby achieving the effect of viewing normal video content at eye level and observing the external environment by looking downwards.

[0011] In another aspect, the present invention also proposes a head-mounted display device, including a main display screen, a camera, a processor, a secondary display screen, and optical elements; wherein, the main display screen is used to display video images of programs or games, and the video light emitted from it enters the wearer's eyes at eye level; the camera is used to capture environmental images outside the device; the processor is used to mirror the external environmental images captured by the camera; the secondary display screen is used to display the external environmental images after being mirrored by the processor; the optical elements are used to perform secondary mirroring of the image displayed on the secondary display screen and change the direction of the light path to direct it towards a designated position; the designated position is the position that the human eye can see when the head-mounted display device is worn, deviating from the eye level.

[0012] In some embodiments of this application, a reflector can be used as the optical element, installed in front of the sub-display screen and arranged at an angle relative to the sub-display screen, to mirror and reflect the image displayed on the sub-display screen, so that the reflected light is directed to the designated position.

[0013] In some embodiments of this application, a light-shielding plate can also be provided in the head-mounted display device to separate the video light emitted from the main display screen and the secondary display screen, preventing the human eye from seeing the video light directly emitted from the secondary display screen. This ensures that the video light emitted directly from the secondary display screen does not mix with the video light emitted from the main display screen, preventing interference from the captured external environment images when the user is watching programs or game videos normally, thus guaranteeing the user's immersion.

[0014] In some embodiments of this application, the central axis of the main display screen can be configured to be parallel to the central axis of the secondary display screen; the light shield is set between the main display screen and the secondary display screen and parallel to the central axis of the screen, and its area configuration should meet the following conditions: it can completely block the line of sight of the human eye to the secondary display screen and the video light emitted directly through the secondary display screen, so as to ensure the separation effect of the two video light beams.

[0015] In some embodiments of this application, the main display screen may include a left-eye display screen and a right-eye display screen; the secondary display screen may be installed at the lower left of the left-eye display screen or at the lower right of the right-eye display screen, so that the wearer only needs to turn their eyes to the lower left or lower right to view the external environment image emitted after secondary mirroring processing by the optical element, and the screen switching is simple and convenient.

[0016] In some embodiments of this application, the size of the secondary display screen can be configured to be smaller than the size of the left-eye display screen and the right-eye display screen, so as to minimize the interference of external environmental image display on normal video playback.

[0017] Compared with the prior art, the advantages and positive effects of the present invention are mainly reflected in: (1) The present invention uses a camera to collect images of the environment outside the device and displays them in front of the wearer through the built-in display screen of the head-mounted display device. This gives the head-mounted display device a see-through function, allowing the user to view the external environment without removing the head-mounted display device. While ensuring the complete display of the viewing content, the user can move or look for external objects at will, improving the user's experience of interacting with the outside world.

[0018] (2) The present invention solves the problem of decreased user viewing experience caused by the superposition of external environmental images and the video content being played normally by the device by adding a display screen to the head-mounted display device to display the environmental images outside the device.

[0019] (3) This invention performs two mirroring processes on the acquired external environment image and changes the propagation path of its video light to a specified position. By configuring the specified position to be the position that the human eye can see when the human wears the head-mounted display device, which is deviated from the horizontal direction, it can achieve the effect of watching the program (or game) directly and observing the outside world by looking sideways. It is convenient and practical. While enriching the functions of the device, the immersive experience of the program is not affected by the interference of the external environment image display, thus improving the user experience.

[0020] Other features and advantages of the present invention will become clearer after reading the detailed description of the embodiments of the present invention in conjunction with the accompanying drawings. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a design flowchart of an embodiment of the perspective display method for a head-mounted display device proposed in this invention; Figure 2 This is a schematic diagram of the system architecture of one embodiment of the head-mounted display device proposed in this invention; Figure 3 This is a schematic diagram of the overall structure of an embodiment of the head-mounted display device proposed in this invention; Figure 4 This is a schematic diagram showing the positional relationship of a main display screen, a secondary display screen, and a light shield in one embodiment. Detailed Implementation

[0023] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the protection scope of the present invention.

[0024] In the description of this invention, it should be understood that the terms "inner", "outer", "left", "right", "front", "rear", "upper", "lower", "center", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0025] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to direct connection, indirect connection, integral molding, or internal communication of components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0026] The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature.

[0027] like Figure 3 As shown, the head-mounted display device of this embodiment mainly includes two parts: a main body 100 and a headband 200. The main body 100 is used to play normal video information, such as video programs or games; the headband 200 is wrapped around the user's head during use to place the main body 100 in front of the user's eyes.

[0028] The main body 100 of the head-mounted display device primarily houses display elements, an optical system, tactile elements, and an auditory system. The display elements, serving as the main display screen 130, typically include a left-eye display screen 131 and a right-eye display screen 132, such as... Figure 4 As shown, because the main display screen 130 obstructs the viewer's eyes, users cannot see the external environment after wearing the head-mounted display device, which causes many inconveniences in use.

[0029] To achieve see-through display functionality in the head-mounted display device, that is, the ability for the user to see the external environment even after wearing the head-mounted display device, this embodiment configures a camera 111 and a secondary display screen 140 on the main body 100 of the head-mounted display device, combined with... Figure 3 , Figure 4 As shown. The camera 111 is used to capture images of the external environment and display them on the secondary display screen 140. This allows the wearer to see the outside world through the secondary display screen 140, enabling the device to perform a see-through function.

[0030] In some embodiments, the main body 100 of the head-mounted display device typically includes a front shell 110 and a middle shell 120, such as Figure 3 As shown. The front shell 110 is mounted in front of the middle shell 120 and is positioned in front of the user's eyes when worn. The middle shell 120 is a circumferential shell that wraps around the user's eyes when worn.

[0031] In some embodiments, a camera 111 for capturing images of the external environment of the device can be mounted on the front housing 110 of the main body 100. For example, it can be mounted at the top center of the front housing 110 of the main body so that the camera 111 can capture a wider range of environmental images in front of the wearer.

[0032] The secondary display screen 140 can be mounted on the middle shell 120 of the main body 100, such as... Figure 4 As shown, for example, it is installed at a certain position on the periphery of the main display screen 130, and in conjunction with image processing technology and optical path adjustment technology, the acquired external environment image is projected to the wearer's eyes while avoiding the wearer's direct line of sight. This can reduce the interference of the displayed external environment image on the wearer's normal viewing of video content and avoid affecting the user's immersive experience.

[0033] To ensure that the external environment image is clearly displayed while avoiding the wearer's direct line of sight, this embodiment designs the following perspective display method, combined with... Figure 1 , Figure 2 As shown, the main processes include the following: S101. Acquire images of the external environment.

[0034] In this embodiment, a camera 111 installed on the front shell 110 of the head-mounted display device is used to capture environmental images outside the device, generate image information and send it to the processor on the built-in circuit board of the head-mounted display device for image processing, such as noise reduction, image size adjustment, image grayscale adjustment and other conventional processing.

[0035] S102. Mirror the acquired external environment image.

[0036] In this embodiment, the processor can be configured not only to perform routine processing on the acquired external environment images, but also to perform mirror processing on the routinely processed external environment images to form a mirror image.

[0037] S103. Display the mirrored image of the external environment on the secondary display screen 140.

[0038] In some embodiments, the processor may employ an existing digital processing chip in the head-mounted display device, which can also be used to process the acquired images of the external environment, thereby simplifying the circuit structure.

[0039] In other words, the functionality of the existing processor in the head-mounted display device can be expanded. On one hand, it processes the video information that the head-mounted display device needs to play normally; on the other hand, it controls the main display screen 130 inside the head-mounted display device to play video programs or game screens normally through the first display driver circuit. Figure 2 As shown; on the other hand, the acquired external environment image is processed, and the mirrored external environment image is displayed on the secondary display screen 140 added inside the head-mounted display device through the second display driving circuit.

[0040] S104. The image displayed on the secondary display screen 140 is mirrored twice, and the direction of the light path is changed to direct the light towards the designated position.

[0041] In this embodiment, optical element 141 is configured in the head-mounted display device to perform a second mirroring of the image displayed on the sub-display screen 140, and the direction of the light path is changed, combined with... Figure 2 , Figure 4 As shown.

[0042] In some embodiments, the optical element 141 may be arranged as a reflector inside the middle shell 120 of the head-mounted display device and located in front of the screen of the secondary display 140, so as to mirror and reflect the image displayed on the secondary display, so that the reflected light is directed to a designated position.

[0043] Specifically, the reflector can be tilted relative to the sub-display screen 140, and its tilt angle α can be determined by combining the direction of the video light emitted by the sub-display screen 140 and the direction of the specified position.

[0044] To prevent the wearer from being distracted by the external environment image displayed on the secondary display screen 140 while normally watching video content (program videos or game screens, etc.), this embodiment configures the main display screen 130 to emit its video light from the wearer's eye-level direction, and configures the designated position to deviate from the wearer's eye-level direction. That is, when the wearer is looking straight ahead, they see the video content normally played by the head-mounted display device; only when their eyes are angled to the designated position can they see the external environment image captured by the device. This achieves the see-through effect of the head-mounted display device while ensuring that the wearer's immersive experience is not diminished.

[0045] In some embodiments, the designated location may be the lower left or lower right position of the wearer's eyes. For a head-mounted display device where the main display 130 includes a left-eye display 131 and a right-eye display 132, the secondary display 140 can be mounted to the lower left of the left-eye display 131 (e.g., Figure 4(As shown) or the lower right side of the right eye display screen, and separate the video light emitted from the main display screen 130 and the secondary display screen 140, so that the wearer cannot see the video light emitted directly from the secondary display screen 140, and can only see the external environment image reflected by the reflector when looking at the side.

[0046] To reliably separate the video light emitted from the main display screen 130 and the secondary display screen 140, a light shield 142 can be installed between the main display screen 130 and the secondary display screen 140, combined with... Figure 2 , Figure 4 As shown, on the one hand, it avoids the mixing of video light emitted from the secondary display screen 140 and video light emitted from the main display screen 130, which would affect the wearer's viewing quality of normal video content; on the other hand, it prevents the wearer from seeing the secondary display screen 140, thus avoiding the problem of reduced immersive experience caused by the secondary display screen 140.

[0047] In some embodiments, the central axis of the main display screen 130 can be configured to be parallel to the central axis of the secondary display screen 140, such as... Figure 2 As shown. The central axis here refers to the axis perpendicular to the screen and passing through the center point of the screen. This ensures that most of the video light emitted from the main display screen 130 is parallel to most of the video light emitted from the secondary display screen 140. With this main and secondary display screen arrangement, when configuring the light shield 142, a simple flat panel can be selected, positioned parallel to the central axis of the screen between the main display screen 130 and the secondary display screen 140. Simultaneously, the area of ​​the light shield 142 should meet the following condition: it should completely block the line of sight of the wearer towards the secondary display screen 140 and the video light directly emitted through the secondary display screen 140. Therefore, when the wearer is looking sideways, they can only see the external environment image reflected by the mirror.

[0048] Of course, the light shield 142 can also be designed in an n-shape or a square shape to block the video light emitted from the secondary display screen 140 from three or four directions. Under the premise of ensuring that the video light can propagate normally to the reflector, it can effectively prevent the video light emitted from the secondary display screen 140 from mixing with the video light emitted from the main display screen 130, and will not shine into the wearer's eyes.

[0049] The following explanation will be based on the example of a secondary display screen 140 positioned to the lower left of the left eye display screen 131 of a head-mounted display device. Figure 4As shown, an n-shaped light shield 142 can be placed between the main display screen 130 and the secondary display screen 140. The secondary display screen 140 is installed below the n-shaped light shield 142, which blocks the video light emitted from the secondary display screen 140 in the upward, left, and right directions, causing most of the video light to propagate forward and a small portion downward. A reflector 141 is installed in front of the screen of the secondary display screen 140 to reflect the forward-propagating video light and direct it towards the lower left of the wearer's eyes. In this way, when the wearer looks directly at the screen, they can barely see the external environment image reflected by the reflector 141; all their vision is focused on the left-eye display screen 131 and the right-eye display screen 132 directly in front of their eyes, which is beneficial for the wearer's immersive experience in VR games and other applications. When the wearer needs to understand the external environment, they only need to turn their eyes to the lower left to see the external environment image reflected by the reflector 141, thus satisfying their need to perceive the outside world.

[0050] Considering that users do not need to view large images of the external environment to perceive the outside world, the size of the secondary display 140 can be configured to be smaller than the size of the left eye display 131 and / or the right eye display 132, so as to facilitate its deployment in the head-mounted display device.

[0051] This embodiment addresses the issue of reduced immersive experience caused by external environment image display by adding a secondary display screen 140 to the head-mounted display device to display external environment information in real time. At the same time, it adds a light shield 142 and a reflector 141 to the structure. This not only allows VR / AR / MR users to view the external environment in real time while wearing the head-mounted display device, but also does not interrupt the normal playback of VR / AR / MR video content, achieving a hybrid display effect from another dimension and expanding the functionality and interactivity of the head-mounted display device.

[0052] Of course, the above description is not intended to limit the present invention, and the present invention is not limited to the examples given above. Any changes, modifications, additions or substitutions made by those skilled in the art within the scope of the present invention should also fall within the protection scope of the present invention.

Claims

1. A perspective display method for a head-mounted display device, characterized in that, include: Acquire images of the external environment; The acquired images of the external environment are mirrored. A secondary display screen is added to the head-mounted display device to display the mirrored image information on the secondary display screen; A light shield is added between the main display screen and the secondary display screen of the head-mounted display device to physically separate the video light emitted from the main display screen and the secondary display screen, so that the human eye cannot see the video light emitted directly from the secondary display screen. A reflector is placed in front of the sub-display screen and at an angle relative to the sub-display screen. The reflector is used to mirror and reflect the image displayed on the sub-display screen to change the direction of the light path and direct the reflected light to a designated position. The specified position is configured to be the position where the external environment image after secondary mirroring can be viewed when the human eye is rotated to an oblique downward position.

2. The see-through display method for a head-mounted display device according to claim 1, characterized in that, The video beam emitted by the head-mounted display device through its main screen enters the wearer's eyes at eye level.

3. The see-through display method for a head-mounted display device according to claim 2, characterized in that, The image size displayed on the secondary display screen is configured to be smaller than the image size displayed on the main display screen.

4. A head-mounted display device, characterized in that, include: The main display screen, which displays video images of programs or games, emits video light that enters the wearer's eyes at eye level. A camera, used to capture images of the environment outside the device; A processor, which is used to mirror the external environment images captured by the camera; A secondary display screen is used to display an image of the external environment after it has been mirrored by the processor. A light shield, located between the main display screen and the secondary display screen, is used to physically separate the video light emitted from the main display screen and the secondary display screen, so that the human eye cannot see the video light emitted directly from the secondary display screen. A reflector is installed in front of the screen of the secondary display screen and is arranged at an angle relative to the secondary display screen. It is used to mirror and reflect the image displayed on the secondary display screen to change the direction of the light path and direct the reflected light to a designated position. The designated location is the position where, after a person wears a head-mounted display device, their eyes can view the external environment image after secondary mirroring when they rotate to a slightly downward angle.

5. The head-mounted display device according to claim 4, characterized in that, The central axis of the main display screen is parallel to the central axis of the secondary display screen. The light-shielding plate is parallel to the central axis of the screen, and its area is configured to completely block the line of sight of the human eye towards the secondary display screen and the video light emitted directly through the secondary display screen.

6. The head-mounted display device according to claim 4 or 5, characterized in that, The main display screen includes a left-eye display screen and a right-eye display screen; The secondary display screen is installed to the lower left of the left eye display screen or to the lower right of the right eye display screen, so that when the wearer's eyes are turned to the lower left or lower right, they can see the external environment image emitted after secondary mirroring by the reflector.

7. The head-mounted display device according to claim 6, characterized in that, The size of the secondary display screen is smaller than the size of the left-eye display screen and the right-eye display screen.

Images