Display device and method for displaying image

Abstract

The invention discloses a display device and a method for displaying an image, and relates to the technical field of optics, and the display device comprises an optical engine which is used for forming a plurality of input light beams which correspond to image points of a virtual image; an expansion panel for diffractively expanding the input light beam received from the optical engine to form an output light beam; a display panel for forming a second image on a display surface of the display panel; the user interface is used for receiving user input; and a control unit for controlling the formation of the first input light beam and the formation of the second image according to a user input received through the user interface, in which the expansion panel is located between the display panel and the viewing area, in which the expansion panel is transparent so that the second image is viewed through the expansion panel. According to the display device and the method for displaying the image, the pressure of eyes when the display device is used can be relieved.

Description

Technical Field

[0001] The present application relates to the field of optical display technology, and in particular to a display device and a method for displaying an image. Background Art

[0002] The lens of the human eye is surrounded by ciliary muscles, which control the focal length of the eye. In order for the eye to focus at close distances, the ciliary muscles need to contract to shorten the focal length of the lens. In this sense, long-term viewing of images displayed on the small touch screen of a traditional mobile phone may cause stress to the eyes. Some scientific studies have even shown that long-term viewing of close-up images may increase the probability of myopia (i.e., nearsightedness). Summary of the invention

[0003] The purpose of the present application is to provide a display device and a method for displaying images, which have multiple display modes, thereby reducing eye pressure when using the display device.

[0004] The scope of protection sought by various embodiments of the present invention is defined by the independent claims. Any embodiments described in this specification that are not within the scope of the independent claims should be interpreted as examples for understanding various embodiments of the present invention.

[0005] The display device can be set to display at least two different types of images, i.e., a virtual image and a surface image. The two types of images can represent, for example, a video, a static image, a graphic and / or a text. In addition, the display device can selectively include a data processing function and / or a communication function. The display device can be, for example, a portable handheld device with a data processing function and a communication function. The display device can be, for example, a smart phone.

[0006] The display device includes an optical engine for forming input light. The input light includes a plurality of input light beams representing respective points of an input image. The direction of each input light beam may correspond to the position of an input image point, and the intensity of the input light beam may correspond to the brightness of the image point. The direction of each input light beam corresponds to the angular position of a virtual image point, and the intensity of the input light beam corresponds to the brightness of the virtual image point.

[0007] The display device includes an expansion panel that forms a plurality of output beams by diffracting and expanding an input beam. When the output beams are emitted into the eyes of an observer, the observer's eyes can see a displayed virtual image. Each point of the virtual image is represented by a collimated output beam, so the virtual image appears to be displayed at a distance from the eyes.

[0008] The display device has at least one display mode that allows the eye to view images with the ciliary muscle relaxed even when the distance between the display device and the eye is small (e.g., less than 0.5 meters). The display device may have, for example, a virtual display mode in which the display device is arranged to display a virtual image at a long distance.

[0009] The display device also includes a display panel for displaying a surface image. The surface image is formed on the display surface of the display panel. The display panel can be, for example, a liquid crystal display (LCD panel) or an LED panel. The display device has at least one display mode, in which the image is formed on the display panel so that the distance between the displayed image and the eye is approximately equal to the distance between the display device and the eye, for example, less than 0.5 meters. The display device can have, for example, a surface display mode, in which the display device is arranged to display the surface image on the display panel. Displaying the surface image can facilitate, for example, providing user input via a touch panel of the display device, because the visual elements of the displayed surface image have a certain lateral position relative to the touch panel.

[0010] When the eye is located in the viewing area of ​​the display device, the displayed virtual image and the displayed surface image can be observed. The viewing area can also be called, for example, a viewing pupil or an eye box.

[0011] The extension panel may be transparent, and the extension panel may be located between the display panel and the viewing area. The display device may include a stack, which in turn includes at least a display panel and a transparent extension panel. For visible light transmitted from the display panel through the coupling output region of the extension panel along the normal direction, the light attenuation of the transparent extension panel may be, for example, less than 80%, less than 50%, less than 20% or even less than 10%. The normal direction is a direction perpendicular to the plane of the extension panel. The corresponding transmittance of the extension panel, for example, for the above-mentioned visible light, may be greater than 20%, greater than 50%, greater than 80% or even greater than 90%. The outcoupling region of the extension panel includes diffraction features, such as microscopic grooves. The diffraction grooves may attenuate the transmitted light to a certain extent, but the attenuation can be limited by selecting the depth and / or width of the diffraction microscopic grooves.

[0012] The viewing area of ​​the display device may be smaller than the sum of the viewing area of ​​the extension panel and the viewing area of ​​the display panel. For example, the viewing area of ​​the extension panel may be greater than 70% of the front area of ​​the display device, and the viewing area of ​​the display panel may also be greater than 70% of the front area of ​​the display device.

[0013] In one embodiment, the display panel and the expansion panel can be, for example, located in the same housing, so that the viewing area of ​​the expansion panel can be, for example, greater than 70% of the housing front area, and the viewing area of ​​the display panel can also be greater than 70% of the housing front area. The transparency of the expansion panel can allow the panel to be placed in the same housing without increasing the housing front area.

[0014] The control unit of the display device may be arranged to change the display mode, for example, based on user input received through a user interface of the device. The device may comprise, for example, a touch panel, and the control unit may be arranged to change the display mode based on gestures detected by the touch panel.

[0015] In one embodiment, the virtual display mode can be used as the default operation mode of the device to reduce eye strain. When receiving user input through the touch panel, the control unit of the device can, for example, temporarily change the operation mode from the virtual display mode to the surface display mode, and after receiving the user input, the control unit can automatically change the operation mode from the surface display mode back to the virtual display mode.

[0016] Prolonged operation in the surface display mode may cause stress to the eyes. The user may wish to limit the use time of the surface display mode to reduce the stress on the ciliary muscles. In one embodiment, the device may be arranged to provide an indication to the user when the operation time in the surface display mode exceeds a predetermined limit. For example, if the cumulative operation time in the surface display mode exceeds a predetermined number of hours per day, the device may be arranged to provide an indication to the user. The indication may also be referred to as, for example, a warning or an alarm.

[0017] The display device may optionally include a shutter panel, which may be located between the display panel and the extension panel. The shutter panel may control the propagation of light from the display panel to the viewing area. The state of the shutter panel may change from a transparent state to a light blocking state to provide a dark background for the virtual image, thereby improving the visibility of the displayed virtual image. The shutter panel may be configured to allow or block light from propagating from the display panel to the viewing area. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0019] Figure 1a A side view of a display device in a virtual display mode provided by an embodiment of the present application;

[0020] Figure 1b A side view of a display device in a surface display mode provided by an embodiment of the present application;

[0021] Figure 1c A side view of a display device in a composite display mode provided by an embodiment of the present application;

[0022] Figure 1d A shutter panel provided in an embodiment of the present application provides a side view of a dark background for a virtual image;

[0023] Figure 1e A shutter panel provided in an embodiment of the present application provides a side view of a dark background for a surface image;

[0024] Figure 2a A state diagram of a display device provided in an embodiment of the present application working in a surface display mode;

[0025] Figure 2b A state diagram of a display device provided in an embodiment of the present application working in a composite display mode;

[0026] Figure 2c A state diagram of a display device provided in an embodiment of the present application working in a virtual display mode;

[0027] Figure 3a A schematic diagram of a display device receiving user input through a touch panel provided in an embodiment of the present application;

[0028] Figure 3b A flow chart of a method for a display device to receive user input provided by an embodiment of the present application;

[0029] Figure 4a A schematic diagram of the structure of an optical engine provided in an embodiment of the present application;

[0030] Figure 4b A schematic diagram of the structure of an extension panel provided in an embodiment of the present application;

[0031] Figure 4c A schematic diagram of the structure of a display panel provided in an embodiment of the present application;

[0032] Figure 4d A schematic diagram of the structure of a shutter panel provided in an embodiment of the present application;

[0033] Figure 5 A schematic diagram of the dimensions of various components in the display device provided in an embodiment of the present application;

[0034] Figure 6a A schematic diagram of an extended panel provided in an embodiment of the present application that forms an output light beam by diffracting and expanding an input light beam;

[0035] Figure 6b A schematic diagram of the dimensions of the expansion panel provided in an embodiment of the present application;

[0036] Figure 6c A schematic diagram of waveguide light propagating in a transparent substrate of an extension panel provided in an embodiment of the present application;

[0037] Figure 7a-7e The input light beam provided for the implementation of the present application corresponds to an image point on the microdisplay that forms an input image;

[0038] Figure 7f An extended output beam of a virtual image point provided by an embodiment of the present application;

[0039] Figure 7g A top view of the angular width of a virtual image provided by an embodiment of the present application;

[0040] Figure 7h A side view of the angular height of a virtual image provided by an embodiment of the present application;

[0041] Figure 8a One of the schematic diagrams of a display device composed of two modules provided in an embodiment of the present application;

[0042] Figure 8b A second schematic diagram of a display device composed of two modules provided in an embodiment of the present application;

[0043] Figure 8c A third schematic diagram of a display device composed of two modules provided in an embodiment of the present application;

[0044] Figure 8d A fourth schematic diagram of a display device composed of two modules provided in an embodiment of the present application;

[0045] Fig. 9 A schematic diagram of a display panel provided in an embodiment of the present application.

[0046] Icon: 500-display device; LCD1-display panel; EPE1-extension panel; STC1-stack; ENG1-optical engine; DISP0-microdisplay; IMG0-input image; P0 P1 P2 P3 P4-image point; IN1-input light; B0-input beam; AX0-optical axis; VIMG1-virtual image; OUT1-first output light; B3-output beam; EYE1-eye of observer; BOX1-viewing area; DSRF-display surface; IMG2-second image; E0 E1 E2-input element; zVIMG-position of virtual image VIMG1; zLCD-position of second image IMG2; zEYE-position of eye EYE1; CNT-control unit; TOU1-touch panel; MEM1-memory 1; DATA1-image data; MEM2-memory 2; DATA2-program data; UIF1-user interface; COM1-user input; RXTX-communication unit; SENG-control signal; SLCD-control signal; HOU1-housing; SUB1-substrate; DOE1-coupling element; DOE2-intermediate element; DO E3-outcoupling element; SHU1-shutter panel; OUT2-second output light; BCK1-background layer; LNS0-collimating optical system; HAND1-user hand; SRF1-first main surface; SRF2-second main surface; MOD1-first module; MOD2-second module; CNT1-first control unit; RXTX1-first communication unit; CNT2-second control unit; RXTX2-second communication unit; LS1-backlight panel; POL1-first polarizer; E3-first electrode; LIQ1-liquid crystal layer; E4-second electrode; POL2-second polarizer. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various different configurations.

[0048] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application for which protection is sought, but merely represents selected embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.

[0049] The display device 500 may include an optical engine ENG1. The optical engine ENG1 may form an input image IMG0 on the micro display DISP0 and collimate light from image points P0 P1 P2 P3 P4 of the input image IMG0 ( Figure 7a-7e ) to form input light IN1. The input light IN1 includes a plurality of collimated input light beams B0, which propagate in different directions with different intensities. The input light IN1 can represent the input image IMG0 and the corresponding virtual image VIMG1.

[0050] The display device 500 can display the virtual image VIMG1 by forming the first output light OUT1. The extension panel EPE1 can form the first output light OUT1 by diffracting the expanded input light IN1. The first output light OUT1 includes a plurality of output light beams B3, which propagate in different directions with different intensities ( Figure 7f ). When the observer's eye EYE1 is located in the viewing area BOX1 of the display device 500 and when the output light OUT1 enters the eye EYE1, the eye EYE1 can see the displayed virtual image VIMG1. The viewing area BOX1 can also be called, for example, an eye box. The viewing area BOX1 refers to a position where the eye EYE1 can be placed to view the entire angular width and the entire angular height of the displayed virtual image VIMG1.

[0051] The direction of each output light beam B3 corresponds to the position of an image point of the virtual image VIMG1 , and the intensity of the output light beam B3 corresponds to the brightness of the image point.

[0052] Each image point of the virtual image VIMG1 may correspond to a substantially collimated output beam, so the eye EYE1 sees the virtual image VIMG1 as being located far away from the eye EYE1. The perceived distance LVIMG between the virtual image VIMG1 and the eye EYE1 may be, for example, greater than 2 meters. The perceived distance LVIMG may even be an infinite distance.

[0053] The display panel LCD1 may be arranged to form a second image IMG2 on a planar display surface DSRF inside the display panel LCD1. The display panel LCD1 may be, for example, a liquid crystal display ( Fig. 9 ). The display panel LCD1 may be, for example, an OLED display comprising an array of organic light emitting diodes. The display panel LCD1 may comprise a two-dimensional array of individually controllable display pixels arranged in rows and columns. The two-dimensional array of display pixels may define a display surface DSRF. The number of rows may be, for example, between 200 and 20,000, and the number of columns may be, for example, between 200 and 20,000.

[0054] The distance LEYE between the second image IMG2 and the eye EYE1 may be substantially equal to the distance between the display device 500 and the eye EYE1. The distance LEYE may be, for example, between 0.1 m and 0.5 m. When the eye EYE1 focuses on the second image IMG2, the distance LEYE needs to be adapted.

[0055] The display panel LCD1 may form a second image IMG2 on a display surface DSRF of the display panel LCD1. A distance between the display surface DSRF and the eye EYE1 may be equal to a distance between the second image IMG2 and the eye EYE1.

[0056] zVIMG may represent the position of the virtual image VIMG1. zLCD may represent the position of the second image IMG2. zEYE may represent the position of the eye EYE1. For example, the position zLCD of the second image IMG2 may be selected as a reference position (zLCD=0).

[0057] The display device 500 may further include a control unit CNT for controlling the operation of the display device 500 .

[0058] The display device 500 may further include a transparent touch panel TOU1 to receive user input by detecting gestures. The touch panel TOU1 may be placed on the extension panel EPE1, for example.

[0059] The display device 500 may further include a memory MEM1 for storing image data DATA1. The image data DATA1 may include one or more digital images. The image data DATA1 may include a video. The display device 500 may form a virtual image VIMG1 and / or a second image IMG2 according to the image data DATA1.

[0060] The display device 500 may further include a memory MEM2 for storing computer program data DATA2. The control unit CNT may control the operation of the display device 500 by executing the computer program data DATA2.

[0061] The display device 500 may include a user interface for receiving user input. The touch panel TOU1 may be arranged as a first user interface. In addition to the touch panel TOU1, the display device 500 may also include an additional user interface UIF1. The user input COM1 may be received through the touch panel TOU1 and / or through the additional user interface UIF1. The user interface UIF1 may include, for example, one or more buttons, a keyboard, a mouse, an acceleration sensor and / or a gravity sensor. For example, the user may also provide user input by changing the direction of the display device 500 and / or shaking the display device 500. The display device 500 may also include a user interface UIF1 instead of the touch panel TOU1.

[0062] The display device 500 may further include a communication unit RXTX for receiving and / or sending data. The communication unit RXTX may, for example, transmit image data DATA1 and / or program data DATA2. The communication unit RXTX may, for example, communicate with the Internet and / or a local area network. The communication unit RXTX may, for example, communicate via wireless and / or wired communication. The communication unit RXTX may, for example, communicate via a mobile communication network and / or via Bluetooth (trademark).

[0063] The control unit CNT may form a control signal SENG for controlling the operation of the optical engine ENG1. The control signal SENG may include, for example, image data DATA1.

[0064] The control unit CNT may form a control signal SLCD for controlling the operation of the display panel LCD1. The control signal SLCD may include, for example, image data DATA1.

[0065] The transparent touch panel TOU1 may form a touch panel signal STOU. The touch panel signal STOU may represent the detected gesture. The touch panel signal STOU may represent the received user input COM1. The touch panel TOU1 may be, for example, a capacitive, resistive, optical and / or ultrasonic touch panel.

[0066] The display device 500 may have different display modes MODE0, MODE1 and MODE2. The display device 500 may have a virtual display mode MODE1, in which only the virtual image VIMG1 is displayed. Figure 1a The operation in the virtual display mode MODE1 is shown.

[0067] The display device 500 may be set to a surface display mode, in which only the surface image IMG2 is displayed. The display device 500 may have a composite display mode, in which in addition to the virtual image VIMG1 the second image IMG2 is also displayed. The display device 500 may be arranged to control the display modes MODE0 MODE1 MODE2 according to a user input COM1. The user input COM1 may be received, for example, via the touch panel TOU1 and / or via the user interface UIF1.

[0068] In one embodiment, the extension panel EPE1, the optical engine ENG1, the display panel LCD1, the touch panel TOU1 and the control unit CNT are all located in the same housing HOU1 of the display device 500 ( Figure 5 ).

[0069] In addition to the display panel LCD1 and the extension panel EPE1, the stack STC1 may also include a touch panel TOU1, a shutter panel SHU1 ( Figure 1d ) and / or background layer BCK1( Figure 1e ).

[0070] SX, SY, SZ may denote orthogonal directions. The extension panel EPE1 may be parallel to the plane defined by the directions SX and SY. The direction SZ may be perpendicular to the extension panel EPE1.

[0071] Figure 1b Operation in the surface display mode MODE2 is shown. The second image IMG2 may be formed on the surface DSRF inside the display panel LCD1. The second image IMG2 may also be referred to as, for example, a surface image or a real image. The extension panel EPE1 may be transparent, and the eye EYE1 may see the displayed second image IMG2 through the extension panel EPE1. The extension panel EPE1 may be transparent, so that the second output light OUT2 formed by the display panel LCD1 may propagate to the viewing area BOX1 through the extension panel EPE1.

[0072] The extension panel EPE1 is located between the display panel LCD1 and the viewing area BOX1 , wherein the extension panel EPE1 is transparent so as to allow the second image IMG2 to be viewed through the extension panel EPE1 .

[0073] The outcoupling area (DOE3) of the extended panel contains diffractive features, such as microscopic grooves. The diffractive grooves may attenuate the transmitted light to some extent, but the attenuation can be limited by selecting the depth of the diffractive microscopic grooves. The extended panel EPE1 may cause attenuation of the output light OUT2 emitted from the display panel LCD1. The size of the extended panel EPE1 can be such that the optical attenuation caused by the extended panel EPE1 is less than 80%, less than 50%, less than 20% or even less than 10%. The attenuation can be determined, for example, by light having a wavelength of 550nm and transmitted through the extended panel EPE1 in the vertical direction SZ.

[0074] The optical attenuation caused by the extended panel EPE1, for example, for the second output light OUT2 transmitted in the direction (SZ) perpendicular to the first main surface SRF1 of the extended panel EPE1 through the substrate SUB1 and the outcoupling element DOE3 of the extended panel EPE1, can be less than 80%, less than 50%, less than 20% or even less than 10%. The corresponding transmittance of the extended panel EPE1 for the output light OUT2 can be greater than 20%, greater than 50%, greater than 80% or even greater than 90%.

[0075] The symbol LTOU may represent the distance between the surface image IMG2 and the outer surface of the touch panel TOU1 in the direction SZ.

[0076] Figure 1c The operation in the composite display mode MODE0 is shown, in which the images VIMG1 and IMG2 are displayed simultaneously. The composite display mode MODE0 may also be referred to as a mixed display mode, for example.

[0077] Reference Figure 1d The display device 500 may optionally include a shutter panel SHU1, which can be controlled by controlling to enable or disable the propagation of the second output light OUT2 from the display panel LCD1 to the viewing area BOX1. The shutter panel SHU1 can improve the visual contrast of the displayed virtual image VIMG1 by providing a controllable dark background for the extension panel EPE1. The control unit CNT can form a control signal SSHU for controlling the operation of the shutter panel SHU1. The shutter panel SHU1 can be implemented by using a controllable liquid crystal layer.

[0078] Reference Figure 1e The display device 500 may further include a background layer BCK1 to prevent the ambient light AMB0 from propagating through the display panel LCD1. The background layer BCK1 may improve the visual contrast of the displayed surface image IMG2.

[0079] Reference Figure 2a, the display device 500 may be arranged to operate in a surface display mode MODE2. The display panel LCD1 may form a second output light OUT2 representing a displayed surface image IMG2. The distance between the displayed surface image IMG2 and the eye EYE1 may be substantially equal to the distance between the display device 500 and the eye EYE1. The surface image IMG2 may include, for example, graphics, photos and / or characters. The surface image IMG2 may include, for example, characters ABC. The surface image IMG2 may also be, for example, a video image displayed by the display panel LCD1.

[0080] Reference Figure 2b , the display device 500 can be arranged to operate in the composite display mode MODE0. The eye EYE1 can simultaneously receive the first output light OUT1 from the extension panel EPE1 and the second output light OUT2 from the display panel LCD1. The second output light OUT2 can be transmitted from the display panel LCD1 to the viewing area BOX1 through the extension panel EPE1.

[0081] Reference Figure 2c , the display device 500 may be arranged to operate in a virtual display mode MODE1. The extension panel EPE1 may form a first output light by diffraction expansion of input light received from the optical engine ENG1. The first output light OUT1 includes output light beams representing image points of the displayed virtual image VIMG1.

[0082] The virtual image VIMG1 may include, for example, graphics, photos and / or characters. The virtual image VIMG1 may include, for example, characters FGH. The virtual image VIMG1 may also be, for example, a video image displayed through the extension panel EPE1.

[0083] Reference Figure 3a , the display device 500 may include a touch panel TOU1 for detecting gestures. The touch panel TOU1 may receive a user input (COM1) by detecting the position of one or more fingers of the user's hand HAND1.

[0084] The display device 500 may be arranged to display a surface image IMG2 in a surface display mode MODE2 or a mixed display mode MODE0. The surface image IMG2 displayed on the display panel LCD1 may include one or more input elements E0 E1 E2. A user may provide a user input COM1 by touching one or more input elements E0 E1 E2. A user may select one or more options, for example, by touching one or more input elements E0 E1 E2.

[0085] The distance (in direction SZ) between display panel LCD1 and touch panel TOU1 may be so small that the detected finger position may be associated with the position of display elements E0 E1 E2 of surface image IMG2. Displaying surface image IMG2 may facilitate receiving user input through touch panel TOU1.

[0086] The combination of the display panel LCD1 and the touch panel TOU1 can be arranged to operate together as a touch screen even in the case where the transparent extension panel EPE1 is located between the display panel LCD1 and the touch panel TOU1. The symbol LTOU can represent the distance between the surface image IMG2 and the outer surface of the touch panel TOU1 in the direction SZ (see Figure 1b ). The distance LTOU can be, for example, less than 3 mm, less than 2 mm, less than 1 mm, less than 0.5 mm or even less than 0.2 mm. A small LTOU distance facilitates associating the position of the user's finger with the position of an element of the displayed surface image IMG2 when the finger position is detected. The thickness of the panels EPE1, SHU1, TOU1 and the thickness of the optional intermediate gaps between the panels EPE1, SHU1, TOU1 can be selected to be small enough to allow the combination of the display panel LCD1 and the touch panel TOU1 to be used as a touch screen. The thickness of the transparent extension panel EPE1 can be selected so that the distance LTOU is, for example, less than 3 mm, less than 2 mm, less than 1 mm, less than 0.5 mm or even less than 0.2 mm.

[0087] In one embodiment, the touch panel TOU1 may also be located between the display panel LCD1 and the extension panel EPE1. The thickness of the extension panel EPE1 may be so small that the touch panel TOU1 can detect the presence of the user's finger through the extension panel EPE1. The thickness of the extension panel EPE1 may be selected to be so small that the touch panel TOU1 can detect a gesture through the extension panel EPE1. The thickness of the extension panel EPE1 may be, for example, less than 1 mm, less than 0.5 mm, or even less than 0.2 mm.

[0088] In one embodiment, the display device 500 may be arranged to set a display mode (MODE0 MODE1 MODE2) according to a user input (COM1) received through the touch panel TOU1.

[0089] Figure 3bThe method of setting the display mode is illustrated with example method steps. The setting of the display mode may start at step 1010. The display device 500 may check whether the touch panel TOU1 is touched at step 1020. If the touch panel is not touched, the method may restart at step 1010. If the touch panel TOU1 is touched at step 1020, the display device 500 may start operating in the surface display mode MODE2 (or composite display mode MODE0) at step 1030. At step 1040, the display device 500 may display one or more input elements E0 E1 E2 as part of the displayed surface image IMG2. The user may select the display mode by touching one of the displayed input elements E0 E1 E2. At step 1050, the touch screen TOU1 may receive the user input COM1 by detecting the finger position of the user's hand HAND1. At step 1060, the display device 500 may set the display mode (MODE0 MODE1 MODE2) according to the user input COM1. At step 1070, the display device 500 may run one or more applications in the selected display mode. The display device 500 can display one or more images when running an application in the selected display mode. The display device 500 can display a video when running an application in the selected display mode. During the running of the application or after the running of the application ends, the display mode can be selected again in step 1010.

[0090] The control unit CNT of the display device 500 may be configured to perform method steps 1010, 1020, 1030, 1040, 1050, 1060, 1070 by executing computer program data DATA2. The control unit CNT may perform these steps together with the functional units TOU1, LCD1, ENG1, for example.

[0091] In one embodiment, when the display device 500 is shaken, the display device 500 may be arranged to temporarily operate in the surface display mode MODE2 (or the composite display mode MODE0). The user interface UIF1 may include, for example, an acceleration sensor for detecting the shaking of the display device 500.

[0092] Reference Figure 4a, the optical engine ENG1 forms an input light IN1, and the input light IN1 includes multiple input light beams B0. The input light beams B0 can propagate in different directions, respectively corresponding to the image points of the input image IMG0. The intensity of the input light beam B0 can correspond to the brightness of the image points of the input image IMG0. The extension panel EPE1 can display the virtual image VIMG1 by diffracting the expanded input light beam B0, so that the displayed virtual image VIMG1 can correspond to the input image IMG0. The optical engine ENG1 may include, for example, a microdisplay DISP0 and a collimating optical system LNS0. The microdisplay DISP0 can display the input image IMG0. The collimating optical system LNS0 can form a collimated input light beam B0 by collimating the light of the image pixels of the input image IMG0. The center of the microdisplay DISP0 and the center of the collimating optical system LNS0 can together define the optical axis AX0 of the optical engine ENG1.

[0093] The input image IMG0 may represent information for display. The input image IMG0 may represent, for example, graphics and / or text. The input image IMG0 may represent, for example, a video. The engine ENG1 may be arranged to generate a still image and / or a video. The engine ENG1 may generate a real input image IMG0 from a digital image. The engine ENG1 may receive one or more digital images, for example from an Internet server or a smartphone.

[0094] The display DISP0 may include a two-dimensional array of display pixels. The display DISP0 may include a two-dimensional array of light-emitting display pixels. The engine ENG1 may include, for example, one or more light-emitting diodes (LEDs). The display DISP0 may also be referred to as, for example, a microdisplay. The display DISP0 may also be referred to as, for example, a spatial intensity modulator. The input image IMG0 may also be referred to as, for example, a main image.

[0095] The input image IMG0 may be a monochrome image or a multi-color image. The input image IMG0 may be, for example, an RGB image, which may include a red (R) partial image, a green (G) partial image, and a blue (B) partial image. The input image IMG0 may be formed by modulating a laser or by modulating light obtained from one or more light emitting diodes.

[0096] The display DISP0 may include, for example, one or more microdisplay imagers, such as liquid crystal on silicon (LCOS), a liquid crystal display (LCD), a digital micromirror device (DMD). The display DISP0 may generate an input image IMG0, for example, at a resolution of 1280×720 (HD). The display DISP0 may generate an input image IMG0, for example, at a resolution of 1920×1080 (Full HD). The display DISP0 may generate an input image IMG0, for example, at a resolution of 3840×2160 (4K UHD). The input image IMG0 may include a plurality of image points P0 P1 P2 ...., and the optical engine ENG1 may include a collimating optical system LNS0 for forming a light beam from each image pixel. The optical engine ENG1 may include a collimating optical system LNS0 for forming a substantially collimated light beam from the light of the image point.

[0097] Reference Figure 4b , the extended panel EPE1 may include a group of diffraction elements DOE1 DOE2DOE3 of GRP1, which are implemented on the waveguide substrate SUB1. The diffraction element group GRP1 may include a diffraction coupling element DOE1, which forms a waveguide light by diffracting the input light IN1. The input light IN1 can be received from the optical engine ENG1. The waveguide light can propagate in the substrate SUB1 in one or more lateral directions (SX SY). The waveguide light can be confined in the substrate SUB1 by total internal reflection (TIR). The extended panel EPE1 may include a coupling element DOE3, which diffracts the waveguide light and outputs it from the substrate SUB1 to form an output light OUT1.

[0098] The extended panel EPE1 may comprise a set of diffractive elements of GRP1 which contribute to the direction of the output light beam.

[0099] The diffraction element group GRP1 may include only two elements, namely, an in-coupling element DOE1 and an out-coupling element DOE3. The in-coupling element DOE1 may form a first waveguide light by diffracting the input light IN1. The first waveguide light may be waveguided in the substrate SUB1. The out-coupling element DOE3 may derive an output light OUT1 from the substrate SUB1 by diffracting the first waveguide light.

[0100] The diffraction element group GRP1 may further include at least one intermediate element DOE2. The coupling-in element DOE1 may form a first waveguide light by diffracting the input light IN1. The intermediate element DOE2 may form an extended waveguide light by diffracting the first waveguide light. The coupling-out element DOE3 may derive an output light OUT1 from the substrate SUB1 by diffracting the extended waveguide light. The first waveguide light and the extended waveguide light may be waveguided in the substrate SUB1.

[0101] However, the diffractive element group GRP1 may comprise two or more intermediate diffractive elements, which may be operated in parallel or in series. In one embodiment, the extended panel EPE1 may comprise four diffractive elements, which contribute to the direction of the output light beam.

[0102] For example, the first intermediate element can form a second waveguide light from the first waveguide light, and the second intermediate element can form a third waveguide light from the second waveguide light. The outcoupling element DOE3 can derive the output light OUT1 from the substrate SUB1 by diffracting the third waveguide light.

[0103] The outcoupling element DOE3 may have a width w3 and a height h3. The dimensions w3 and h3 of the outcoupling element DOE3 may also define the maximum lateral dimension of the output light OUT1. The dimensions w3 and h3 of the outcoupling element DOE3 may also define the maximum lateral dimension of the viewing area BOX1.

[0104] Figure 4c The display panel LCD1 is shown. The active display area of ​​the display panel LCD1 may have a width wLCD and a height hLCD.

[0105] Figure 4d One shutter panel SHU1 is shown. The width of the shutter panel SHU1 may be, for example, greater than or equal to the width wLCD of the display panel LCD1. The height of the shutter panel SHU1 may be, for example, greater than or equal to the height hLCD of the display panel LCD1.

[0106] Reference Figure 5 , the extension panel EPE1 may partially or completely cover the display panel LCD1. The outcoupling element DOE3 of the extension panel EPE1 may partially or completely cover the effective display area of ​​the display panel LCD1.

[0107] The display device 500 may include a housing HOU1. The display panel LCD1 and the extension panel EPE1 may be disposed in the housing HOU1. The housing HOU1 may have a width of wHOU in the direction SX and a height of hHOU in the direction SY.

[0108] The active display area of ​​the display panel LCD1 may, for example, cover at least 50% of the area of ​​the out-coupling element DOE3. The out-coupling element DOE3 may, for example, cover at least 50% of the active display area of ​​the display panel LCD1.

[0109] The width wLCD of the display panel LCD1 may be, for example, greater than or equal to 70% of the width wHOU of the housing HOU1. The height hLCD of the display panel LCD1 may be, for example, greater than or equal to 70% of the height hHOU of the housing HOU1. The width w3 of the outcoupling element DOE3 may be, for example, greater than or equal to 70% of the width wHOU of the housing HOU1. The height h3 of the outcoupling element DOE3 may be, for example, greater than or equal to 70% of the height hHOU of the housing HOU1.

[0110] The display device 500 may be, for example, a portable smartphone. The display device 500 may be, for example, a portable tablet computer. The width wHOU may be in the range of 50 mm to 200 mm, wherein the height hHOU may be in the range of 100% to 300% of the width wHOU. Alternatively, the height hHOU may be in the range of 50 mm to 200 mm, wherein the width wHOU may be in the range of 100% to 300% of the height hHOU. In one embodiment, the same display device 500 may be used in both portrait mode and landscape mode by rotating 90 degrees.

[0111] Reference Figure 6a , the extension panel EPE1 can expand the input light IN1 by diffraction to form the output light OUT1. The width wOUT1 of the output light OUT1 can be greater than the width wIN1 of the input light IN1. The extension panel EPE1 can also be called, for example, a diffraction beam expander or an exit pupil expander. The input light IN1 can be received from the optical engine ENG1. The input light IN1 may include a plurality of substantially collimated input light beams B0, which correspond to image points of the input image IMG0. The output light OUT1 may include a plurality of substantially collimated output light beams B3, which correspond to image points of the displayed virtual image VIMG1. The virtual image VIMG1 may correspond to the input image IMG0.

[0112] The extended panel EPE1 may include a set of diffraction elements of GRP1 implemented on a waveguide substrate SUB1. The extended panel EPE1 may include an in-coupling element DOE1, an intermediate element DOE2 and an out-coupling element DOE3. The in-coupling element DOE1 may form a first waveguide light B1 by diffracting an input light IN1. The intermediate element DOE2 may form a second waveguide light B2 by diffracting the first waveguide light. The out-coupling element DOE3 may derive an output light OUT1 from the substrate SUB1 by diffracting the second waveguide light B2. The light beam B3 of the output light OUT1 represents an image point of the virtual image VIMG1.

[0113] Figure 6b The dimensions of the extension panel EPE1 are shown by way of example.

[0114] The diffractive coupling element DOE1 has a width w1 and a height h1. The coupling element DOE1 comprises a diffraction grating G1. The coupling element DOE1 has diffraction features F1, such as microscopic grooves. The coupling element DOE1 has a grating period d1. The coupling element DOE1 has a grating vector V1. The direction of the grating vector V1 can be specified, for example, by an angle β1 relative to a reference direction (e.g. SX).

[0115] The diffractive intermediate element DOE2 has a width w2 and a height h2. The intermediate element DOE2 comprises a diffraction grating G2. The intermediate element DOE2 has diffractive features F2, such as microscopic grooves. The intermediate element DOE2 has a grating period d2. The intermediate element DOE2 has a grating vector V2. The direction of the grating vector V2 can be specified, for example, by an angle β2 relative to a reference direction.

[0116] The diffraction outcoupling element DOE3 has a width w3 and a height h3. The outcoupling element DOE3 comprises a diffraction grating G3. The outcoupling element DOE3 has diffraction features F3, such as microscopic grooves. The outcoupling element DOE3 has a grating period d3. The outcoupling element DOE3 has a grating vector V3. The direction of the grating vector V3 can be specified, for example, by an angle β3 relative to a reference direction.

[0117] The size of the grating vector depends on the grating period of the diffraction grating of the diffraction element, and the direction of the grating vector depends on the direction of the diffraction grating. For example, the size of the grating vector V1 depends on the grating period d1 of the diffraction grating G1 of the diffraction element DOE1, and the direction β1 of the grating vector V1 depends on the direction of the diffraction feature F1 of the diffraction grating G1.

[0118] The extended panel EPE1 includes a plurality of diffraction elements DOE1, DOE2, DOE3 in the optical path. The coupling-in element DOE1 can receive input light beams B0P1, B0P2, for example, through the first main surface SRF1 of the extended panel EPE1. The extended panel EPE1 can be designed to make the vector sum of the grating vectors of the diffraction elements in the optical path equal to zero (V1+V2+V3=0) to ensure that each output light beam B3aP1, B3aP2 provided by the coupling-out element DOE3a is parallel to the corresponding input light beam B0P1, B0P2 obtained from the optical engine ENG1.

[0119] The diffraction elements DOE1, DOE2, DOE3 can be produced by using photolithography. For example, one or more imprinting tools can be produced by electron beam lithography, and the diffraction gratings of the diffraction elements can be formed by using the one or more imprinting tools. A replica of the diffraction microstructure of the imprinting tool can be formed on the waveguide plate SUB1 by using the imprinting tool as a mold or by pressing the surface of the waveguide plate with the imprinting tool.

[0120] Reference Figure 6c , the waveguide substrate SUB1 of the extension panel EPE1 has a first main surface SRF1 and a second main surface SRF2. The first surface SRF1 may be parallel to the second surface SRF2. tSUB1 may represent a distance between the surfaces SRF1 and SRF2.

[0121] The waveguide light may propagate within the substrate SUB1 by total internal reflection (TIR). The waveguide light (eg, B2) may be repeatedly reflected between the surfaces SRF1 and SRF2 until the waveguide light (B2) is coupled out of the substrate.

[0122] In particular, the collimated input beam B0P2 may represent an image point P2 of the input image IMG0. The waveguide light B2P2 may be formed by diffracting the light of the input beam B0P2. The collimated output beam B3P2 may be formed by diffracting the light B2P2 out of the substrate SUB1. The output beam B3P2 may represent an image point P2 of the displayed virtual image VIMG1.

[0123] The extended panel EPE1 comprises a waveguide substrate SUB1. The diffractive elements DOE1, DOE2, DOE3 may be implemented on the waveguide plate SUB1. The waveguide plate SUB1 comprises a planar waveguide core. In one embodiment, the plate SUB1 may optionally comprise, for example, one or more cladding layers, one or more protective layers and / or one or more mechanical support layers. The thickness of tSUB1 may refer to the thickness of the planar waveguide core of the plate SUB1.

[0124] The substrate SUB1 may have a refractive index n1. The refractive index n1 may, for example, be in the range of 1.5 to 1.9. The waveguide plate SUB1 may include or consist essentially of a transparent solid material. The plate SUB1 may include, for example, glass, polycarbonate or polymethyl methacrylate (PMMA). The diffractive optical elements may be formed, for example, by molding, embossing and / or etching. The diffractive optical elements may be implemented by one or more surface diffraction gratings. In particular, the diffractive optical elements of the extended panel EPE1 may be surface diffraction gratings, which may be implemented on the same main surface (SRF1 or SRF2) of the waveguide plate SUB1. Forming elements on the same main surface may simplify production.

[0125] Figures 7a to 7eIt is illustrated how to form collimated input light beams B0P0, B0P1, B0P2, B0P3, B0P4 by collimating light of image points P0, P1, P2, P3, P4 of input image IMG0. Input light IN1 received from optical engine ENG1 may include multiple input light beams B0P0, B0P1, B0P2, B0P3, B0P4. Point P0 may be the center point of image IMG0 / VIMG1. Points P1, P2, P3, P4 may be corner points of image IMG0 / VIMG1.

[0126] Reference Figure 7f The extension panel EPE1 may form an output beam B3P0 by diffracting and expanding the light of the input beam B0P0. The output beam B3P0 may represent the image point P0 of the input image IMG0 and the image point P0 of the virtual image VIMG1.

[0127] The input image IMG0 may include a center point P0 and four corner points P1, P2, P3, and P4. P1 may represent the upper left corner point. P2 may represent the upper right corner point. P3 may represent the lower left corner point. P4 may represent the lower right corner point. The input image IMG0 may include, for example, graphic characters "F", "G", and "H". The input image IMG0 may represent display information.

[0128] The extended panel EPE1 can form output beams B3P1, B3P2, B3P3, B3P4 by diffracting the light of the expanded input beams B0P1, B0P2, B0P3, B0P4. The output beams B3P1, B3P2, B3P3, B3P4 can represent the image points P1, P2, P3, P4 of the input image IMG0 and the corresponding image points P1, P2, P3, P4 of the virtual image VIMG1.

[0129] The display device 500 may include:

[0130] An optical engine ENG1 for forming an input light IN1 including a plurality of input light beams B0P0 , B0P1 , B0P2 , B0P3 , B0P4 representing an input image IMG0 and a corresponding virtual image VIMG1 ;

[0131] The expansion panel EPE1 is used for expanding the received input light beam B0 by diffraction to form an output light OUT1, wherein the output light OUT1 includes a plurality of output light beams B3P0, B3P1, B3P2, B3P3, B3P4 representing the input image IMG0 and the corresponding virtual image VIMG1.

[0132] In one embodiment, the extended panel EPE1 can form an output beam B3P0 from an input beam P0P0, so that the output beam B3P0 is parallel to the input beam P0P0. The output beam B3P1 can be formed by the input beam P0P1, so that the output beam B3P1 is parallel to the input beam P0P1. The output beam B3P2 can be formed by the input beam P0P2, so that the output beam B3P2 is parallel to the input beam P0P2. The output beam B3P3 can be formed by the input beam P0P3, so that the output beam B3P3 is parallel to the input beam P0P3. The output beam B3P4 can be formed by the input beam P0P4, so that the output beam B3P4 is parallel to the input beam P0P4.

[0133] refer to Figure 7g and 7h , the virtual image VIMG1 may have an angular width ΔφVIMG and an angular height ΔθVIMG. The direction of each beam may be specified by, for example, an azimuth angle (e.g., φB3P1) and an elevation angle (e.g., θB3P1). REF1 may represent a reference plane defined by directions SY and SZ. REF2 may represent a reference plane defined by directions SX and SZ.

[0134] refer to Figure 8a , 8b , 8c, 8d, the display device 500 can be assembled from two modules MOD1 and MOD2. The first module MOD1 may include a transparent extension panel EPE1 for displaying a virtual image VIMG1. The second module MOD2 may include a display panel LCD1 for displaying a surface image IMG2. The first module MOD1 may also include an optical engine ENG1, a first control unit CNT1 and a first communication unit RXTX1. The second module MOD2 may also include a second control unit CNT2 and a second communication unit RXTX2. The first control unit CNT1 and the second control unit CNT2 may communicate with each other through, for example, the first communication unit RXTX1 and the second communication unit RXTX2. Therefore, the first control unit CNT1 can control the operation of the optical engine and the operation of the display panel. Similarly, the second control unit CNT2 can control the operation of the display panel and the operation of the optical engine. The touch panel TOU1 may, for example, be located in the first module MOD1 ( Figure 8a ) or the second module MOD2( Figure 8b ). The second module MOD2 may be, for example, a smart phone including a display panel LCD1 and a touch panel TOU1. The second module MOD2 may be, for example, a detachable attachment of the first module MOD1. The second module MOD2 may be attached to the first module MOD1 by one or more magnets, adhesives and / or one or more screws. Figure 8dThe situation is shown in which the second module MOD2 has been separated from the first module MOD1.

[0135] refer to Fig. 9 , the display panel LCD1 may be, for example, a liquid crystal display. The display panel LCD1 may include, for example, a backlight panel LS1, a first polarizer POL1, a first electrode E3, a liquid crystal layer LIQ1, a second electrode E4, and a second polarizer POL2. The backlight panel LS1 may provide a backlight BL1. The first polarizer may polarize the backlight BL1. At least one of the electrode layers E1 and E2 includes an individually controllable electrode array. The liquid crystal direction of different pixels in the liquid crystal layer LIQ1 may be controlled by an electric field generated between the electrode layers E1 and E2. The liquid crystal may change the polarization state of the backlight BL1 according to its direction. The liquid crystal layer LIQ1 may form a modulated light ML1 by modulating the polarization of the backlight BL1. The second polarizer POL2 may allow or prevent the propagation of the modulated light ML1 according to the polarization state of the modulated light ML1. Therefore, the display panel LCD1 may form a surface image IMG2 on the display panel LCD1. Dark pixels of the image IMG2 may be formed at positions where the polarizer POL2 prevents the modulated light ML1 from propagating. Bright pixels of the image IMG2 may be formed at positions where the polarizer POL2 allows the modulated light ML1 to propagate. The display panel LCD1 may optionally include a color filter array RGB1 to display the image IMG2 as a colorful image. The colorful image IMG2 may be, for example, an RGB image. The RGB image includes a red sub-image, a green sub-image, and a blue sub-image.

[0136] It will be apparent to those skilled in the art that modifications and variations may be made to the apparatus and method of the present invention. These illustrations are schematic. The specific embodiments described above with reference to the accompanying drawings are for illustration only and are not intended to limit the scope of the present invention as defined by the appended claims.

[0137] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A display device, characterized in that: include: An optical engine for forming a plurality of input light beams (B0), the plurality of input light beams (B0) corresponding to image points (P0, P1, P2, P3, P4) of a virtual image (VIMG1); an expansion panel (EPE1) for expanding an input light beam (B0) received from an optical engine (ENG1) by diffraction to form an output light beam (B3); a display panel (LCD1) for forming a second image (IMG2) on a display surface (DSRF) of the display panel (LCD1); User interface (UIF1), used to receive user input (COM1); and a control unit (CNT1) for controlling the formation of a first input light beam (B0) and the formation of a second image (IMG2) according to a user input (COM1) received through a user interface (UIF1), wherein the display device (500) is arranged to form a virtual image (VIMG1) and a second image (IMG2) so that the virtual image (VIMG1) and the second image (IMG2) can be seen from a viewing area (BOX1) of the display device (500), wherein an extension panel (EPE1) is located between the display panel (LCD1) and the viewing area (BOX1), wherein the extension panel (EPE1) is transparent so that the second image (IMG2) can be viewed through the extension panel (EPE1).

2. The display device according to claim 1, characterized in that The extended panel (EPE1) causes attenuation of the output light (OUT2) emitted from the display panel (LCD1), wherein the size of the extended panel (EPE1) is designed so that the attenuation caused by the extended panel (EPE1) is less than 50%.

3. The display device according to claim 2, characterized in that The extension panel (EPE1) covers at least 50% of the active display area of ​​the display panel (LCD1).

4. The display device according to claim 2, characterized in that A virtual display mode (MODE1) is provided. A virtual image (VIMG1) can be observed in the virtual display mode (MODE1), and a second image (IMG2) cannot be observed in the virtual display mode (MODE1).

5. The display device according to claim 3, characterized in that A surface display mode (MODE2) is provided, in which the second image (IMG2) can be observed, and in which the virtual image (VIMG1) cannot be observed.

6. The display device according to claim 4 or 5, characterized in that: A composite display mode (MODE0) is provided, in which the virtual image (VIMG1) and the second image (IMG2) can be observed simultaneously.

7. The display device according to claim 6, characterized in that The control unit (CNT1) is arranged to set a display mode (MODE0, MODE1, MODE2) of the display device (500) according to a user input (COM1) received via a user interface (UIF1).

8. The display device according to claim 6, characterized in that The display device (500) includes a transparent touch panel (TOU1) for receiving user input (COM1).

9. The display device according to claim 8, characterized in that Arranged to display one or more input elements (E) on a display panel (LCD1) and receive user input (COM1) via a touch panel (TOU1).

10. The display device according to claim 1, characterized in that The optical engine (ENG1) includes a micro display (DISP0) for forming an input image (IMG0) and a collimating optical device (LNS0) for forming an input light beam (B0) by collimating pixel light of the input image (IMG0).

11. The display device according to claim 1, characterized in that The display device (500) is a portable handheld device.

12. The display device according to claim 1, characterized in that The invention comprises a first display module (MOD1) and a detachable second display module (MOD2), wherein the first display module (MOD1) comprises a display panel (LCD1), and the second display module (MOD2) comprises an extension panel (EPE1).

13. The display device according to claim 1, characterized in that A background layer (BCK1) is included to prevent ambient light from propagating to a viewing area (BOX1) through a display panel (LCD1), wherein the display panel (LCD1) is located between the background layer (BCK1) and an extension panel (EPE1).

14. The display device according to claim 1, characterized in that A shutter panel (SHU1) is included, located between a display panel (LCD1) and an extension panel (EPE1) to provide a controllable black background for a virtual image (VIMG1).

15. A method for displaying an image using the display device according to any one of claims 1 to 14, characterized in that: include: Displaying a virtual image (VIMG1) by forming an expanded output beam (B3) from an input beam (B0) received from an optical engine (ENG1) using an expansion panel (EPE1); displaying a second image (IMG2) on the display panel (LCD1), as well as, The display of the virtual image (VIMG1) and the display of the second image (IMG2) are controlled according to a user input (COM1) received through the user interface (UIF1).

Images