Perspective display method, device and head-mounted display device

Abstract

The embodiment of the present application provides a kind of perspective display method, device and head-mounted display equipment.The perspective display method is applied to VR equipment, the VR equipment is provided with optical module, the optical module includes the directional light source component, display panel and imaging lens group in turn along the light transmission direction;The perspective display method comprises: obtaining the face change image of wearer;The directional light source component is controlled to alternatively emit first light and second light;The display panel is controlled to display the first face change image corresponding to the first light and the second face change image corresponding to the second light;The first face change image and the second face change image are projected into the different eyes of outside observer by the imaging lens group respectively.

Description

Technical Field

[0001] This application relates to the field of display technology, and more specifically, to a perspective display method, a perspective display device, a head-mounted display device, and a computer-readable storage medium. Background Technology

[0002] In recent years, virtual reality (VR) has developed rapidly, and its application areas have become increasingly widespread. To better enhance the user experience, VR functions have become more diversified. For example, to improve users' interaction with the external environment, some VR products have added a video see-through function, which captures images of the external environment through a camera and displays them on the screen of the virtual reality optical system, allowing users to see the external environment.

[0003] However, users outside the VR headset cannot see the wearer's covered eyes and facial expressions, resulting in an asymmetrical interaction.

[0004] In view of this, a new technical solution is needed to solve the above-mentioned technical problems. Summary of the Invention

[0005] The purpose of this application is to provide a new technical solution for a perspective display method.

[0006] Another objective of this application is to provide a perspective display device.

[0007] Another objective of this application is to provide a head-mounted display device.

[0008] Another object of this application is to provide a computer-readable storage medium.

[0009] In a first aspect, embodiments of this application provide a perspective display method. This perspective display method is applied to a VR device. The VR device is equipped with an optical module, which includes a directional light source assembly, a display panel, and an imaging lens group arranged sequentially along the light transmission direction.

[0010] The perspective display method includes:

[0011] Acquire images of facial changes in the wearer;

[0012] The pointing light source assembly is controlled to alternately emit the first light beam and the second light beam;

[0013] The display panel is controlled to display a first facial change image corresponding to the first light and a second facial change image corresponding to the second light;

[0014] The imaging lens group projects the first facial change image and the second facial change image into different eyes of an external observer, respectively.

[0015] Optionally, the optical module further includes an ambient light sensor;

[0016] The perspective display method further includes:

[0017] Establish a mapping relationship between ambient light brightness values ​​and current values ​​in the directional light source component;

[0018] The actual ambient light brightness value is detected by the ambient light sensor.

[0019] The current value of the directional light source component is controlled according to the mapping relationship and the actual ambient light brightness value.

[0020] Optionally, before acquiring images of the wearer's facial changes, the method further includes: acquiring the wearer's original facial image.

[0021] Optionally, the pointing light source assembly includes a first light source group and a second light source group, and controlling the pointing light source assembly to alternately emit the first light beam and the second light beam specifically includes:

[0022] The frame frequency for controlling the opening and closing of the first light source group and the second light source group is greater than 60Hz.

[0023] Optionally, the pointing light source assembly further includes a modulation light path module, which is disposed away from the light-emitting side of the display panel;

[0024] Controlling the pointing light source assembly to alternately emit the first light beam and the second light beam further includes:

[0025] The first light source group is turned on, and the modulation optical path module modulates the transmission direction of the first light so that it is transmitted towards the left eye of the external observer;

[0026] The first light source group is turned off while the second light source group is turned on. The modulation optical path module modulates the transmission direction of the second light beam so that it is transmitted towards the right eye of the observer.

[0027] Optionally, the pointing light source assembly further includes a light guide portion, which is provided with a first functional film and a second functional film. The first functional film is used to transmit the first light and reflect the second light, and the second functional film is used to transmit the second light and reflect the first light.

[0028] Controlling the pointing light source assembly to alternately emit the first light beam and the second light beam further includes:

[0029] The first light source group is turned on, and the second functional film modulates the transmission direction of the first light so that it is transmitted towards the left eye of the observer.

[0030] The first light source group is turned off while the second light source group is turned on. The first functional film modulates the transmission direction of the second light so that it is transmitted toward the right eye of the observer.

[0031] Optionally, the VR perspective display method further includes: adjusting the position of the display panel within one focal length of the imaging lens group to change the magnification of the first facial change image and the second facial change image.

[0032] Optionally, the VR perspective display method further includes: if no external observer is detected within the wearer's field of vision, controlling the directional light source component to turn off.

[0033] Secondly, embodiments of this application also provide a perspective display device. The perspective display device employs the perspective display method described in the first aspect to achieve perspective display;

[0034] The perspective display device includes:

[0035] The acquisition module is used to acquire images of facial changes in the wearer;

[0036] The first control module is used to control the pointing light source assembly to alternately emit the first light beam and the second light beam;

[0037] The second control module is used to control the display panel to display a first facial change image corresponding to the first light and a second facial change image corresponding to the second light;

[0038] An imaging module is used to image the first facial change image and the second facial change image in the eyes of different external observers.

[0039] Thirdly, embodiments of this application also provide a head-mounted display device. The head-mounted display device includes:

[0040] An optical module is used to transmit light to display images of changes in the wearer's face;

[0041] Memory is used to store executable computer instructions;

[0042] A processor, configured to execute the perspective display method according to the first aspect under the control of the executable computer instructions.

[0043] Fourthly, embodiments of this application also provide a computer-readable storage medium storing computer instructions thereon, which, when executed by a processor, perform the perspective display method described in the first aspect.

[0044] The beneficial effects of this application are as follows:

[0045] According to embodiments of this application, through the integrated application of facial capture, image processing, lighting technology, display technology, and imaging technology, images of changes in the wearer's face can be displayed and imaged. Without affecting the wearer's immersion, external observers can clearly observe the wearer's facial image, achieving symmetrical interaction and enhancing the user experience.

[0046] Other features and advantages of this specification will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description

[0047] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments of this specification and, together with their description, serve to explain the principles of this specification.

[0048] Figure 1 The diagram shown is a schematic of the hardware configuration of the head-mounted display device provided in an embodiment of this application.

[0049] Figure 2 The diagram shown is a flowchart of the perspective display method provided in an embodiment of this application.

[0050] Figure 3 The image shows the state of the optical module provided in the embodiment of this application. Figure 1 .

[0051] Figure 4 The image shows the state of the optical module provided in the embodiment of this application. Figure 2 .

[0052] Figure 5 The diagram shows the structure of the light guide provided in an embodiment of this application. Figure 1 .

[0053] Figure 6 The diagram shows the structure of the light guide provided in an embodiment of this application. Figure 2 .

[0054] Figure 7 The diagram shows the structure of the light source module and light guide provided in an embodiment of this application. Figure 1 .

[0055] Figure 8 The diagram shows the structure of the light source module and light guide provided in an embodiment of this application. Figure 2 .

[0056] Figure 9 This is a schematic block diagram of a perspective display device according to an embodiment of this application;

[0057] Figure 10 This is a schematic block diagram of a head-mounted display device according to an embodiment of this application.

[0058] Explanation of reference numerals in the attached figures:

[0059] 1. Pointing light display screen; 10. Pointing light source assembly; 11. Display panel;

[0060] 101. Light source module; 1011. First light source group; 1012. Second light source group; 102. Modulation optical path module; 1021. Converging lens;

[0061] 1013, Light guide; 1014, Substrate; 1015, Protrusion; 1016, First surface; 1017, Second surface;

[0062] 2. Imaging lens group;

[0063] L1, first ray; L2, second ray. Detailed Implementation

[0064] Various exemplary embodiments of the present application will now be described in detail with reference to the accompanying drawings. It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of the present application.

[0065] The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit the scope of this application and its application or use.

[0066] Technologies and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, such technologies and equipment should be considered part of the specification.

[0067] In all the examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limitations. Therefore, other examples of exemplary embodiments may have different values.

[0068] It should be noted that similar labels and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be discussed further in subsequent figures.

[0069] The perspective display method, apparatus, head-mounted display device, and computer-readable storage medium provided in the embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0070] <Hardware Configuration>

[0071] Figure 1 This is a block diagram of the hardware configuration of a head-mounted display device 1000 provided according to an embodiment of this application.

[0072] like Figure 1 As shown, the head-mounted display device 1000 can be a VR device, such as a VR head-mounted display or VR smart glasses, etc. This application embodiment does not limit this.

[0073] In one embodiment, such as Figure 1 As shown, the head-mounted display device 1000 may include a processor 1100, a memory 1200, an interface device 1300, a communication device 1400, a display device 1500, an input device 1600, a microphone array 1700, and a speaker 1800, etc. The processor 1100 may include, but is not limited to, a central processing unit (CPU), a microprocessor (MCU), etc. The memory 1200 may include, for example, ROM (Read-Only Memory), RAM (Random Access Memory), non-volatile memory such as a hard disk, etc. The interface device 1300 may include, for example, various bus interfaces, such as serial bus interfaces (including USB interfaces), parallel bus interfaces, etc. The communication device 1400 may be capable of wired or wireless communication, for example. The display device 1500 may be, for example, a liquid crystal display (LCD), an LED display, a touch screen, etc. The input device 1600 may include, for example, a touch screen, a keyboard, a gamepad, etc. The microphone array 1700 may be used for inputting voice information. The speaker 1800 may be used for outputting voice information.

[0074] In this embodiment, the memory 1200 of the head-mounted display device 1000 is used to store instructions that control the processor 1100 to operate in order to implement or support the implementation of a control method for the head-mounted display device according to any embodiment. Those skilled in the art can design instructions based on the schemes disclosed in this specification. How the instructions control the processor to operate is well known in the art and will not be described in detail here.

[0075] Those skilled in the art should understand that, although in Figure 1 The present specification shows a number of devices for a head-mounted display device 1000. However, the head-mounted display device 1000 of the embodiments described herein may involve only some of the devices, or may include other devices, which is not limited herein.

[0076] Figure 1 The head-mounted display device 1000 shown is illustrative only and is not intended to limit this specification, its application, or its use.

[0077] It should be noted that, referring to Figure 10The hardware configuration of the head-mounted display device 1000 also needs to include an optical module, which enables symmetrical interaction between the observer and the wearer. For example, the head-mounted display device 1000 is a VR head-mounted display, and the optical module is located on the backlight side of the display. The optical module displays the wearer's facial image information, allowing the wearer's facial image information (including eye and face image information) to be clearly seen by the observer, thus achieving symmetrical interaction between the observer and the wearer.

[0078] Figure 3 and Figure 4 The structure of an optical module according to one embodiment is shown. The optical module includes a directional light source assembly 10, a display panel 11 and an imaging lens group 2 arranged sequentially along the light transmission direction. The light transmission direction is away from the wearer, that is, the light transmission method is towards the external observer.

[0079] The directional light source component 10 serves as the core light source of the optical module, responsible for directionally emitting light containing information about the wearer's facial image. For example, in practical use, after capturing images of changes in the wearer's face (including changes in the eyes / face), the directional light source component 10 is activated to directionally emit light containing information about these changes. For example, the directional light source component 10 employs directional light source technology to achieve the purpose of directional light emission.

[0080] The display panel 11, in conjunction with the directional light source assembly 10, constitutes the basic structure of the directional light source display screen 1. Relying on the light source group and display panel 11, which can change the transmission direction, the directional light source display screen 1 can transmit the first facial change image and the second facial change image to different eyes of an external observer in an alternating pattern of odd and even frames. For example, the display panel 11 can be an LCD display screen. The directional light source display screen 1 is a directional light source LCD display screen.

[0081] The main function of the imaging lens group 2 is to focus and shape light, ensuring that the light is transmitted to the observer's eye along a predetermined path and direction. The imaging lens group 2 enables precise control and adjustment of the light, thereby forming a clear and realistic image in the observer's eyes. In other words, in addition to the directional light source display screen 1 formed by the directional light source assembly 10 and the display panel 11, the optical module also incorporates an imaging lens to present the image of the wearer's facial changes displayed on the directional light source display screen 1 at a position closer to the actual wearer's face, preventing the image from appearing above the face and improving the interactive experience.

[0082] Hereinafter, various embodiments and examples according to this application will be described with reference to the accompanying drawings.

[0083] <Method Implementation>

[0084] Figure 2 This application illustrates a perspective display method according to an embodiment of the present application. This perspective display method can be, for example, by... Figure 1 and Figure 10 The head-mounted display device 1000 shown is implemented.

[0085] like Figure 2 As shown, the perspective display method provided in this embodiment may include the following steps S2100 to S2400, see below. Figure 2 .

[0086] S2100: Acquire images of facial changes in the wearer;

[0087] S2200: Control the pointing light source assembly to alternately emit the first light beam and the second light beam;

[0088] S2300: Control the display panel to display a first facial change image corresponding to the first light and a second facial change image corresponding to the second light;

[0089] S2400: The first facial change image and the second facial change image are projected onto different eyes of an external observer through the imaging lens group.

[0090] The purpose of the perspective display method provided in this application is to enable external observers to observe facial changes in the VR wearer, achieving symmetrical interaction. In other words, the prerequisite for implementing this method is that an external observer can be detected within the VR wearer's field of vision. For example, a 6DOF camera (six degrees of freedom camera) can be set up in the VR headset to detect whether an external observer is within the VR wearer's field of vision.

[0091] When an external observer is present within the VR wearer's field of vision captured by the 6DOF camera, steps S2100 to S2400 are executed. That is, when the 6DOF camera detects an external observer within the wearer's field of vision, the 6DOF camera sends a signal to the acquisition module that can acquire images of the wearer's facial changes and to the first control module that can control the opening and closing of the directional light source component, triggering the outward projection function to be activated. At this time, the acquisition module acquires (captures) the wearer's facial change image information, and the display panel corresponding to the outward projection function lights up to display the VR wearer's facial change image in real time.

[0092] In step S2100, facial change images of the wearer are acquired using some facial capture technology (such as a camera (e.g., an eye-tracking camera and a face-tracking camera), a depth sensor, etc.). These images may include changes in facial expressions, head rotation, etc., to track and reflect the wearer's facial dynamics in real time.

[0093] It should be noted that the captured facial images undergo a series of image processing algorithms to optimize image quality, remove noise, and enhance contrast. Additionally, facial recognition and feature extraction can be performed as needed for subsequent matching or fusion with displayed content.

[0094] In step S2200, after acquiring images of the wearer's facial changes, these images are displayed and imaged using a combination of illumination and display technologies to provide them to an external observer.

[0095] In this step, the light emitted by the directional light source assembly 10 is controlled to achieve illumination. For example, different lighting effects can be created by controlling parameters such as the brightness, color, and direction of the light source, thereby enhancing the three-dimensionality and realism of the facial change image.

[0096] Specifically, this is achieved by controlling the directional light source assembly 10 to alternately emit a first light beam and a second light beam. The first light beam L1 and the second light beam L2 have different transmission directions, so that the first facial change image and the second facial change image can be transmitted to different eyes of the observer. In one example, the first light beam L1 emitted by the directional light source assembly 10 can be controlled to be transmitted towards the left eye of the observer, so that the first facial change image corresponding to the first light beam L1 is transmitted to the left eye of the observer. Similarly, the second light beam L2 emitted by the directional light source assembly 10 can be controlled to be transmitted towards the right eye of the observer, so that the second facial change image corresponding to the second light beam L2 is generated to the right eye of the observer. The reverse is also possible.

[0097] For example, the pointing light source assembly 10 includes a light source module 101, which includes a first light source group 1011 and a second light source group 1012, and controls the first light source group 1011 and the second light source group 1012 to alternately turn on and off. For example, the first light source group 1011 and the second light source group 1012 use LED light sources at different angles, and the first light source group 1011 and the second light source group 1012 can be located on opposite sides of the LCD display screen.

[0098] Reference Figure 3 and Figure 4 Located on the left ( Figure 3 When the LED light source (below) is turned on, the light emitted from the light source display screen 1 is transmitted to the observer's left eye via the imaging lens group 2, and is located on the right eye ( Figure 3When the LED light source (above) is turned on, the light emitted from the light source display screen 1 is transmitted to the right eye of the observer through the imaging lens group 2.

[0099] The light source module 101 is designed to alternately turn on and off to create different lighting conditions or display states. The alternating on and off of the light source module 101: By controlling the precise timing of the circuitry, the first light source group 1011 and the second light source group 1012 can be alternately turned on and off. This alternating operating mode allows the directional light source assembly 10 to exhibit different optical characteristics within different time windows, alternatingly transmitting them to the eyes of an external observer.

[0100] In step S2300, the processed facial change image is combined with the display content and displayed through the display panel 11. That is, in this step, the display is achieved by controlling the display panel 11.

[0101] The choice of display panel 11 depends on the specific application scenario and performance requirements. Display panel 11 can be of type LCD or similar. During the display process, it is necessary to ensure that the image clarity, color accuracy, refresh rate, and other indicators meet the requirements.

[0102] It should be noted that different types of display panels 11 can be controlled in different ways to display images. For example, if the display panel is an LCD, controlling the arrangement of liquid crystal molecules in the LCD panel under the influence of an electric field can allow specific light to pass through at a specific angle.

[0103] For example, in step S2200, after the pointing light source assembly 10 alternately emits the first light L1 and the second light L2, the display panel 11 is controlled to execute step S2300. Specifically, the first light L1 is transmitted to the display panel 11, and the display panel 11 displays the first facial change image, and the second light L2 is transmitted to the display panel 11, and the display panel 11 displays the second facial change image.

[0104] In step S2400, imaging is achieved through the cooperation of the imaging lens group 2 and the eyes of the external observer. Specifically, the display content is projected onto different eyes of the external observer through the imaging lens group 2, so that the external observer can clearly observe the image of the wearer's facial changes.

[0105] For example, refer to Figure 3 and Figure 4 An external observer can see the changes in the wearer's face at the virtual image location.

[0106] In this embodiment, by comprehensively applying facial capture, image processing, lighting technology, display technology, and imaging technology, the wearer's facial change image can be displayed and imaged. Without affecting the wearer's immersion, an external observer can clearly observe the wearer's facial image, achieving symmetrical interaction and enhancing the user experience.

[0107] Furthermore, the optical module also includes an ambient light sensor, and the perspective display method further includes the following steps:

[0108] S2500: Establish a mapping relationship between ambient light brightness values ​​and current values ​​in the directional light source assembly 10;

[0109] S2600: The ambient light sensor detects the actual ambient light brightness value;

[0110] S2700: Control the current value of the pointing light source component according to the mapping relationship and the actual ambient light brightness value.

[0111] In this embodiment, the optical module also includes an ambient light sensor, which improves the brightness of the directional light source display screen 1.

[0112] Specifically, the ambient light sensor is used to sense the ambient light intensity where the observer is located, monitor the ambient light intensity in real time, and adjust the brightness of the light source display screen 1 to improve the contrast between the display brightness and the ambient light, thereby enhancing the interactive experience.

[0113] In step S2500, the optimal brightness (or luminous intensity) that the directional light source component should achieve under different ambient light levels is determined through experiments or theoretical calculations, and the corresponding current value is set accordingly. This mapping relationship can be stored in the system's memory for real-time querying and application. In other words, the current value of the directional light source component defined according to the ambient light level is pre-calibrated.

[0114] In step S2600: After the 6DOF camera captures an observer within the wearer's line of sight, the ambient light sensor is activated. The ambient light sensor continuously monitors the light intensity of the surrounding environment and transmits the detected brightness value to the control unit of the pointing light source display screen 1. This brightness value changes in real time, reflecting the current ambient lighting conditions.

[0115] In step S2700: After receiving the brightness value from the ambient light sensor, the control unit queries the previously established mapping relationship to find the current value of the directional light source component corresponding to the brightness value. Then, the control unit adjusts the current of the directional light source component to reach the set value, thereby changing the brightness of the light source to adapt to the current ambient lighting conditions.

[0116] For example, the directional light source display screen 1, which comprises the directional light source component 10 and the display panel 11, is a directional light source LCD display screen, and the light source is an LED light source. When the ambient light brightness is higher than a certain threshold, the LED light source current of the directional light source LCD is increased to ensure higher output screen brightness, so that even in high ambient light conditions, the wearer can clearly see the facial change image of the VR wearer; when the ambient light brightness is lower than a certain threshold, the LED light source current of the directional light source LCD is decreased to appropriately reduce the screen brightness, so that the facial change image of the VR wearer does not appear too bright.

[0117] Therefore, in this embodiment, by automatically adjusting the display brightness of the directional light source display screen 1, it can be ensured that external observers can obtain the best visual effect under different lighting conditions, thereby reducing eye fatigue.

[0118] In one embodiment, the following steps are included before acquiring images of the wearer's facial changes:

[0119] S2000: Acquire the wearer's original facial image.

[0120] In this embodiment, the purpose of this step is to acquire a baseline facial image of the wearer without any specific changes (such as facial expression, lighting, angle, etc.). This raw image will serve as a reference point for subsequent comparison and analysis, used to accurately capture and quantify various facial changes. In other words, this step is the benchmark for acquiring the wearer's facial change image in step S2100. This benchmark facilitates the acquisition of the wearer's facial change image.

[0121] In one embodiment, the pointing light source assembly 10 includes a first light source group 1011 and a second light source group 1012, and controlling the pointing light source assembly 10 to alternately emit the first light beam L1 and the second light beam L2 specifically includes:

[0122] The frame frequency for controlling the opening and closing of the first light source group 1011 and the second light source group 1012 is greater than 60Hz.

[0123] In this embodiment, the pointing light source assembly 10 includes a light source module 101, which consists of two spaced-apart light source groups, namely a first light source group 1011 and a second light source group 1012. These two light source groups may contain the same or different types of light sources, such as LEDs, lasers, etc., depending on the application requirements.

[0124] For example, the first light source group 1011 and the second light source group 1012 employ two sets of LED light sources at different angles, located on the left and right sides of the LCD display screen, respectively. (Refer to...) Figure 3 and Figure 4When the LED light source located on the left (lower part of the figure) is turned on, the light emitted from the light source display screen 1 is transmitted to the left eye of the external observer through the imaging lens group 2. When the LED light source located on the right (upper part of the figure) is turned on, the light emitted from the light source display screen 1 is transmitted to the right eye of the external observer through the imaging lens group 2.

[0125] In this specific step, the on and off frequencies of the first light source group 1011 and the second light source group 1012 in the directional light source assembly 10 are limited to avoid being perceived by the human eye of an external observer and to reduce the feeling of flickering.

[0126] In one embodiment, the pointing light source assembly 10 further includes a modulation light path module 102, which is disposed away from the light-emitting side of the display panel.

[0127] Controlling the pointing light source assembly 10 to alternately emit the first light beam and the second light beam further includes:

[0128] S2800: Control the first light source group 1011 to turn on, and the modulation optical path module 102 modulates the transmission direction of the first light L1 so that it is transmitted towards the left eye of the external observer.

[0129] S2900: Control the first light source group 1011 to turn off and the second light source group 1012 to turn on. The modulation optical path module 102 modulates the transmission direction of the second light L2 so that it is transmitted towards the right eye of the external observer.

[0130] In this embodiment, the pointing light source assembly 10 further includes a modulation optical path module 102. The main function of the modulation optical path module 102 is to modulate the direction of the light emitted by the light source module 101. This means that it can control the propagation path of the light, ensuring that the first light ray L1 and the second light ray L2 can be transmitted to the left or right eye of an external observer in a predetermined direction. For example, the modulation optical path module 102 can ensure that when the first light source group 1011 is turned on, the first light ray L1 emitted by it is mainly guided to the left eye of the observer; while when the second light source group 1012 is turned on, the second light ray L2 emitted by it is mainly guided to the right eye of the observer, and vice versa.

[0131] In step S2800, the control unit sends a signal to the first light source group 1011 to start emitting light and generate a first light L1, which is used to form part of the image of the wearer's facial changes.

[0132] Next, the modulation optical path module 102 adjusts its internal optical components (such as lenses, mirrors, polarizers, etc.) according to the control signal to change the transmission direction of the first light ray L1. The purpose of this adjustment is to ensure that the first light ray L1 can be transmitted accurately toward the left eye of the external observer, so that the left eye of the external observer can receive the light from the first light source group.

[0133] In step S2900, after the transmission of the first light ray L1 is completed, the control unit sends a signal to the first light source group 1011 to turn it off and stop emitting light. At the same time, the control unit sends another signal to the second light source group 1012 to start emitting light, generating the second light ray L2 to form another part of the image of the wearer's facial changes.

[0134] After the second light source group 1012 starts emitting light, the modulation optical path module 102 readjusts its internal optical elements to change the transmission direction of the second light ray L2. The purpose of this adjustment is to ensure that the second light ray can be transmitted accurately toward the right eye of the external observer, so that the right eye of the external observer can receive the light from the second light source group.

[0135] In one specific embodiment, refer to Figure 3 and Figure 4 The type of modulation optical path module 102 includes converging lens 1021, that is, the converging lens 1021 is arranged in an array to form modulation optical path module 102. This arrangement can ensure that the light emitted by the light source module 101 is evenly and efficiently distributed to each lens, thereby achieving a wider and more uniform illumination coverage.

[0136] In addition, the convex surface of the converging lens 1021 faces the light source module 101 directly. The convex lens has a converging effect on light. In this embodiment, the convex surface faces the light source module 101, which means that the light emitted by the light source will be effectively converged when it passes through the lens, thereby enhancing the directionality and intensity of the light.

[0137] For example, refer to Figure 3 and Figure 4 When the first light source group 1011 on the left is turned on, the light emitted from the first light source group 1011 is converged and transmitted to the display panel 11 through the converging lens 1021, and then transmitted to the left eye of the external observer through the imaging lens group 2; when the second light source group 1012 on the right is turned on, the light emitted from the second light source group 1012 is converged and transmitted to the display panel 11 through the converging lens 1021, and then transmitted to the right eye of the external observer through the imaging lens group 2.

[0138] It should be noted that the modulation optical path module 102 includes, but is not limited to, the converging lens 1021 defined above. For example, it can also be a prism, other types of deflection mirrors or gratings, as long as it can change the transmission direction of light to a certain extent so that the first light ray L1 and the second light ray L2 are respectively transmitted to the left eye or right eye of the external observer.

[0139] In one embodiment, refer to Figures 3-8 The pointing light source assembly 10 further includes a light guide portion 1013, which is provided with a first functional film and a second functional film. The first functional film is used to transmit the first light and reflect the second light, and the second functional film is used to transmit the second light and reflect the first light.

[0140] Controlling the pointing light source assembly 10 to alternately emit the first light beam and the second light beam further includes:

[0141] The first light source group 1011 is turned on, and the second functional film modulates the transmission direction of the first light so that it is transmitted towards the left eye of the observer.

[0142] The first light source group 1011 is turned off while the second light source group 1012 is turned on. The first functional film modulates the transmission direction of the second light so that it is transmitted towards the right eye of the observer.

[0143] In this embodiment, the pointing light source assembly 10 further includes a light guide 1013. The light guide 1013 serves as a light transmission channel and can effectively guide the light emitted by the light source to the area that needs to be illuminated, thereby improving the uniformity of the first light L1 and the second light L2 emitted.

[0144] When the pointing light source assembly 10 includes a light guide portion 1013, the first light source group 1011 and the second light source group 1012 may be spaced apart along the length direction of the light guide portion 1013, or the first light source group 1011 and the second light source group 1012 may be spaced apart along the width direction of the light guide portion 1013, so as to provide two sets of light sources at different angles to meet the requirements of the pointing light source display screen 1.

[0145] It should be noted that, regardless of which direction the first light source group 1011 and the second light source group 1012 are spaced apart along the light guide section 1013, as long as they are combined with the modulation light path module 102, the display panel 11 and the imaging lens group 2, it is possible to transmit the first light L1 to one eye of the external observer and the second light L2 to the other eye of the external observer.

[0146] It is important to emphasize that in this embodiment, a first functional film and a second functional film are provided on the light guide 1013. The first functional film is designed to transmit a first light ray L1 and reflect a second light ray L2. This means that when the first light ray L1 shines on the first functional film, it is allowed to pass through and continue to propagate, while the second light ray L2 is reflected to achieve subsequent transmission. Conversely, the second functional film has the opposite function to the first functional film: the second functional film transmits the second light ray L2 and reflects the first light ray L1. This means that the second light ray L2 can pass smoothly through the second functional film, while the first light ray L1 is reflected to achieve subsequent transmission. In other words, in this embodiment, by providing a first functional film and a second functional film on the light guide 1013, the light guide 1013 can adjust the transmission direction of the light so that the first light ray and the second light ray are transmitted to the eyes of an external observer accordingly.

[0147] The specific steps of the following embodiment will be explained in detail:

[0148] Specifically, the alternating emission process of the pointing light source assembly 10 involves the coordinated operation of two light source groups (the first light source group and the second light source group) and two functional films (the first functional film and the second functional film).

[0149] The specific steps for adjusting the transmission direction of the first ray L1 are as follows:

[0150] The control unit sends a signal to the first light source group 1011, causing the first light source group 1011 to start emitting light and generate a first ray L1, which contains part of the information of the facial change image.

[0151] When the first light source group 1011 emits light, the second functional film (which can be a thin film with specific optical properties, such as a polarizing film or a beam splitter) will adjust its optical properties according to the control signal to change the transmission direction of the first light ray L1 so that the left eye of the observer can receive the light from the first light source group.

[0152] The specific steps for adjusting the transmission direction of the second ray L2 are as follows:

[0153] After transmitting the first ray L1, the control unit sends a signal to the first light source group 1011 to turn it off and stop emitting light. Immediately afterwards, the control unit sends another signal to the second light source group 1012 to start emitting light, generating a second ray L2. The second ray L2 contains facial change image information different from the first ray, used to form the image seen by the observer's right eye.

[0154] When the second light source group 1012 emits light, the first functional film (similar to the second functional film, such as a polarizing film or a beam-splitting film) adjusts its optical properties according to the control signal to change the transmission direction of the second light ray L2. Specifically, the first functional film causes the second light ray L2 to be transmitted towards the right eye of the external observer. In this way, the right eye of the external observer can receive the light from the second light source group.

[0155] It should also be noted that, when the pointing light source assembly 10 includes a light guide portion with a first functional film and a second functional film, and includes a modulation light path module 102, the specific steps for adjusting the first light transmission direction are as follows:

[0156] When the first light source group 1011 emits light, the first light ray L1 is transmitted through the light guide 1013 and adjusted by the second functional film of the light guide 1013, thereby achieving a basic adjustment of the transmission direction of the first light ray L1. Then, after the first light ray L1 is modulated by the modulation light path module 102, it can be ensured that the first light ray L1 is transmitted towards the left eye direction of the external observer.

[0157] The specific steps for adjusting the transmission direction of the second ray L2 are as follows:

[0158] When the second light source group 1012 emits light, the second light ray L2 is transmitted through the light guide 1013 and adjusted by the first functional film of the light guide 1013, thereby achieving a basic adjustment of the transmission direction of the second light ray L2. Then, after the second light ray L2 is modulated by the modulation light path module 102, it can be ensured that the second light ray L2 is transmitted towards the right eye direction of the external observer.

[0159] In one specific embodiment, refer to Figure 5 and Figure 6 The light guide section 1013 includes a substrate 1014 and protrusions 1015 disposed on the substrate 1014. The protrusions 1015 are arranged in an array on the substrate 1014. The design of the protrusions 1015 can increase the chance of light escaping from the light guide section 1013, that is, improve the light extraction efficiency. Traditional planar light guide plates cause some light to be trapped inside the light guide plate due to the total internal reflection effect, which cannot be effectively utilized. The presence of the protrusions 1015 can break this total internal reflection condition, allowing more light to escape in the designed direction, thereby improving the light transmission efficiency.

[0160] For example, by adjusting the shape, size, height, and array arrangement of the protrusions 1015, precise control of the light propagation path can be achieved, thereby enhancing the directionality of the light.

[0161] The first functional film and the second functional film are respectively applied to two opposite surfaces of the protrusion 1015 (i.e., the first surface 1016 and the second surface 1017). This design enables the protrusion 1015 to selectively transmit and reflect light according to the characteristics of different light rays, thereby achieving directional output of light.

[0162] For example, refer to Figure 5 A second functional film is provided on the first surface 1016 of the protrusion 1015. The second functional film reflects the first light L1. The first light L1 reflected by the second functional film is modulated by the modulation optical path module 102 and transmitted to the display panel 11, and then transmitted to the left eye of the observer through the imaging lens group 2.

[0163] It should be noted that, in order to allow the light emitted from the first light source group 1011 to be guided into the light guide section 1013, a functional film is provided on the surface of the light guide section 1013 closest to the first light source group 1011 for transmitting the light emitted from the first light source group 1011.

[0164] Reference Figure 6 A first functional film is provided on the second surface 1017 of the protrusion 1015. The first functional film reflects the second light L2. The second light L2 reflected by the first functional film is modulated by the modulation optical path module 102 and transmitted to the display panel 11, and then transmitted to the right eye of the observer through the imaging lens group 2.

[0165] It should be noted that, in order to allow the light emitted from the second light source group 1012 to be guided into the light guide section 1013, a functional film is provided on the surface of the light guide section 1013 closest to the second light source group 1012 for transmitting the light emitted from the second light source group 1012.

[0166] In one specific embodiment, refer to Figure 7 and Figure 8 The light source module 101 may include multiple light sources arranged in an array, or the light source module 101 may include at least one light source and a light guide strip for guiding the transmission of light.

[0167] For example, refer to Figure 7 The diagram illustrates a specific embodiment of the light source module 101. Specifically, the light source module 101 includes multiple light sources arranged in an array. That is, the first light source group 1011 includes multiple light sources arranged in an array, and the second light source group 1012 includes multiple light sources arranged in an array. Optionally, the light sources can be LED light sources or laser light sources, etc.

[0168] Reference Figure 8The diagram illustrates another specific embodiment of the light source module 101. Specifically, the light source module 101 includes a light source and a light guide strip located between the light guide portion 1013 and the light source. That is, the first light source group 1011 includes a first light source and a first light guide strip, with the first light guide strip located between the first light source and the light guide portion 1013. The second light source group 1012 includes a second light source and a second light guide strip, with the second light guide strip located between the second light source and the light guide portion 1013. Optionally, the first light source and the second light source can be LED light sources or laser light sources, etc.

[0169] In one embodiment, the VR perspective display method further includes: adjusting the position of the display panel 11 within one focal length of the imaging lens group 2 to change the magnification of the first facial change image and the second facial change image.

[0170] Specifically, the focal length of the imaging lens group 2 is the distance from the center of the lens of the imaging lens group 2 to its focal point (the point where light rays converge after passing through the imaging lens group 2).

[0171] In this implementation, the relative positional relationship between the display panel 11 and the imaging lens group 2 is defined, that is, the display panel 11 is located within one focal length of the imaging lens group 2, and the setting position of the display panel 11 can be adjusted within this distance range.

[0172] When the display panel 11 is placed within one focal length of the imaging lens group 2, according to the lens imaging principle, the imaging lens group 2 will form a magnified, upright virtual image on the same side of the lens. Furthermore, the specific position of the display panel 11 can be set according to actual magnification requirements, making the position of the perceived image appear closer to the wearer's eye position.

[0173] For example, refer to Figure 1 and Figure 2 The virtual image shown is the location of the magnified and upright virtual image of the wearer's face as seen by an external observer.

[0174] In one embodiment, the VR perspective display method further includes: controlling the pointing light source component 10 to turn off when no external observer is detected within the wearer's field of vision.

[0175] In this embodiment, the directional light source assembly 10 is automatically turned off when no observer is detected within the wearer's line of sight. This step not only helps save energy but also reduces unnecessary light output when no one is watching, thus avoiding disturbance or discomfort to the surrounding environment.

[0176] <Device Embodiment>

[0177] Figure 9This is a structural schematic diagram of a perspective display device 3000 according to one embodiment. (See attached diagram.) Figure 9 As shown, the perspective display device includes an acquisition module 3100, a first control module 3200, a second control module 3300, and an imaging module 3400.

[0178] The acquisition module 3100 is used to acquire images of facial changes of the wearer. For example, the acquisition module 3100 can be a face tracking camera or an eye tracking camera.

[0179] The first control module 3200 is used to control the pointing light source assembly 10 to alternately emit a first light beam and a second light beam; for example, the first control module 3200 can be a control unit for controlling the pointing light source assembly, such as a control circuit including a switch.

[0180] The second control module 3300 is used to control the display panel 11 to display a first facial change image corresponding to the first light and a second facial change image corresponding to the second light. For example, the second control module 3300 can be a control unit for controlling the display panel, such as a control circuit that applies an electric field.

[0181] The imaging module 3400 is used to image the first facial change image and the second facial change image in different eyes of an external observer. Exemplarily, the imaging module 3400 and the external observer's binoculars cooperate to image the first facial change image and the second facial change image onto a virtual image location.

[0182] According to an embodiment of this application, a transmissive display device is provided, which allows an external observer to clearly and distinctly observe the wearer's facial change images without affecting the basic display of the virtual reality device (VR display). The facial change images may include images of the wearer's eyes and face.

[0183] <Equipment Example>

[0184] Figure 10 This is a schematic diagram of the hardware structure of a head-mounted display device according to one embodiment. Figure 10 As shown, the head-mounted display device includes an optical module, a processor, and a memory.

[0185] This optical module can be used to transmit light to display images of changes in the wearer's face;

[0186] This memory can be used to store executable computer instructions.

[0187] The processor can be used to execute the perspective display method according to the method embodiments of this application, under the control of the executable computer instructions.

[0188] The head-mounted display device can be as follows: Figure 1 The head-mounted display device 1000 shown may also be a device with other hardware structures, and is not limited here. The head-mounted display device may be, for example, a VR device, and this embodiment of the application does not limit it.

[0189] In one embodiment, each module of the see-through display device of the above head-mounted display device can be implemented by a processor running computer instructions stored in memory.

[0190] According to the head-mounted display device provided in the embodiments of this application, when the various hardware structures cooperate with each other, without affecting the basic display of the virtual reality device (VR display), an external observer can also clearly and distinctly observe the facial change images of the wearer, wherein the facial change images may include the wearer's eye change images and face change images.

[0191] Computer-readable storage media

[0192] This application also provides a computer-readable storage medium storing computer instructions thereon, which are executed by a processor to perform the perspective display method provided in this application.

[0193] This application may be a system, method, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of this application.

[0194] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example—but not limited to—electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination thereof. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0195] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0196] The computer program instructions used to perform the operations of this application may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuits, such as programmable logic circuits, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), are personalized by utilizing state information from the computer-readable program instructions. These electronic circuits can execute the computer-readable program instructions to implement various aspects of this application.

[0197] Various aspects of this application are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0198] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0199] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0200] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions. It will be well known to those skilled in the art that implementation in hardware, implementation in software, and implementation using a combination of software and hardware are equivalent.

[0201] The various embodiments of this application have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical applications, or technological improvements to the embodiments in the market, or to enable others skilled in the art to understand the embodiments disclosed herein. The scope of this application is defined by the appended claims.

Claims

1. A perspective display method, characterized in that, Applied to VR devices, the VR devices are equipped with an optical module, the optical module includes a pointing light source assembly (10), a display panel (11) and an imaging lens group (2) arranged sequentially along the light transmission direction, the pointing light source assembly (10) includes a first light source group (1011) and a second light source group (1012). The perspective display method includes: Acquire images of facial changes in the wearer; The pointing light source assembly (10) is controlled to alternately emit a first light beam and a second light beam; Control the display panel (11) to display a first facial change image corresponding to the first light and a second facial change image corresponding to the second light; The first facial change image and the second facial change image are projected into different eyes of an external observer through the imaging lens group (2); The pointing light source assembly (10) further includes a light guide (1013), which includes a substrate (1014) and a protrusion (1015) disposed on the substrate (1014). The two opposite surfaces of the protrusion (1015) are respectively provided with a first functional film and a second functional film. The first functional film is used to transmit the first light and reflect the second light, and the second functional film is used to transmit the second light and reflect the first light. Controlling the pointing light source assembly (10) to alternately emit the first light beam and the second light beam further includes: The first light source group (1011) is turned on, and the second functional film modulates the transmission direction of the first light so that it is transmitted towards the left eye of the observer. The first light source group (1011) is turned off while the second light source group (1012) is turned on. The first functional film modulates the transmission direction of the second light so that it is transmitted toward the right eye of the observer.

2. The perspective display method according to claim 1, characterized in that, The optical module also includes an ambient light sensor; The perspective display method further includes: Establish a mapping relationship between the ambient light brightness value and the current value in the directional light source component (10); The actual ambient light brightness value is detected by the ambient light sensor. The current value of the directional light source component is controlled according to the mapping relationship and the actual ambient light brightness value.

3. The perspective display method according to claim 1, characterized in that, Before obtaining images of the wearer's facial changes, the process also includes obtaining the wearer's original facial image.

4. The perspective display method according to claim 1, characterized in that, Controlling the pointing light source assembly (10) to alternately emit the first light beam and the second light beam specifically includes: The frame frequency for controlling the opening and closing of the first light source group (1011) and the second light source group (1012) is greater than 60Hz.

5. The perspective display method according to claim 4, characterized in that, The pointing light source assembly (10) further includes a modulation light path module (102), which is disposed away from the light-emitting side of the display panel (11); Controlling the pointing light source assembly (10) to alternately emit the first light beam and the second light beam further includes: The first light source group (1011) is turned on, and the modulation optical path module (102) modulates the transmission direction of the first light so that it is transmitted towards the left eye of the external observer; The first light source group (1011) is turned off while the second light source group (1012) is turned on. The modulation optical path module (102) modulates the transmission direction of the second light so that it is transmitted toward the right eye of the observer.

6. The perspective display method according to claim 1, characterized in that, The perspective display method further includes: Adjust the position of the display panel (11) within one focal length of the imaging lens group (2) to change the magnification of the first facial change image and the second facial change image.

7. The perspective display method according to claim 1, characterized in that, The perspective display method further includes: If no external observer is detected within the wearer's line of sight, the directional light source component (10) is turned off.

8. A perspective display device, characterized in that, The perspective display device achieves perspective display using the perspective display method as described in any one of claims 1-7; The perspective display device includes: The acquisition module (3100) is used to acquire images of facial changes in the wearer; The first control module (3200) is used to control the pointing light source assembly (10) to alternately emit the first light beam and the second light beam; The second control module (3300) is used to control the display panel (11) to display a first facial change image corresponding to the first light and a second facial change image corresponding to the second light; An imaging module (3400) is used to image the first facial change image and the second facial change image in the different eyes of an external observer.

9. A head-mounted display device, characterized in that, The head-mounted display device includes: An optical module is used to transmit light to display images of changes in the wearer's face; Memory is used to store executable computer instructions; A processor, configured to execute the perspective display method according to any one of claims 1-7, under the control of the executable computer instructions.

10. A computer-readable storage medium, characterized in that, It stores computer instructions, which, when executed by a processor, perform the perspective display method according to any one of claims 1-7.

Images