Display device

Abstract

The invention provides a display device which comprises a display screen, a first camera element, a second camera element and a control module. The display screen is provided with a display surface facing a user. The first camera element is arranged on one side of the display surface of the display screen and is used for tracking the eye position of a user. The second camera element is arranged on one side, back to the display surface, of the display screen and is used for shooting a scenery image of a scenery which is positioned on the sight line of the user and is shielded by the display screen; the control module is suitable for displaying at least one part of the scenery image on the display screen according to the eye position of the user.

Description

Technical Field

[0001] The present invention relates to a display device, and in particular to a display device with a perspective effect. Background Art

[0002] The display applications of augmented reality (AR) and mixed reality (MR) have driven the development of transparent displays. In order to take into account the effects of perspective and display, transparent displays are generally provided with a display pixel area and a perspective area. Since the design of the perspective area will squeeze the layout space of the display pixel area, the display resolution of transparent displays is often lower than that of general displays. In addition, since transparent displays mostly use ambient light as the illumination light source during display, their display brightness will be lower than that of general non-transparent displays, and the display contrast performance will also be poor. Therefore, developing a display device with a perspective effect and high brightness, high contrast and high resolution is still one of the research and development focuses of relevant manufacturers. Summary of the invention

[0003] The present invention is directed to a display device which can take into account both perspective effect and display quality.

[0004] According to an embodiment of the present invention, a display device includes a display screen, a first camera element, a second camera element, and a control module. The display screen is provided with a display surface facing a user. The first camera element is arranged on one side of the display surface of the display screen, and is used to track the eye position of the user. The second camera element is arranged on the side of the display screen facing away from the display surface, and is used to capture a scene image of a scene located in the user's line of sight and blocked by the display screen. The control module is adapted to display at least a portion of the scene image on the display screen according to the eye position of the user.

[0005] Based on the above, in a display device of one embodiment of the present invention, a first camera element and a second camera element are respectively provided on opposite sides of a display screen. The first camera element is used to track the eye position of the user. The second camera element is used to capture the image of the scene that is located on the side of the display screen facing away from the user and is blocked by the display screen. By tracking the current eye position of the user, the control module can correspondingly adjust the scene image captured by the second camera element and present it on the display screen in real time. No matter how the user's eyes move, the scene image on the display screen can accurately present the true appearance of the rear scene blocked by the display screen, allowing the user to have a visual experience of seeing through the rear scene through the display screen. BRIEF DESCRIPTION OF THE DRAWINGS

[0006] Figure 1A and Figure 1B is a schematic diagram of an application of a display device according to an embodiment of the present invention;

[0007] Figure 2 yes Figure 1A and Figure 1B A schematic top view of the display device when in use;

[0008] Figure 3 yes Figure 1A A block diagram of a display device;

[0009] Figure 4 yes Figure 1B A schematic diagram of a scene image captured by a second imaging element;

[0010] Figure 5 is a top view schematic diagram of a display device in use according to an embodiment of the present invention;

[0011] Figure 6 yes Figure 5 A block diagram of a display device;

[0012] Figure 7 yes Figure 1A A schematic diagram of a display device when operating in another display mode;

[0013] Figure 8 yes Figure 1A A top view schematic diagram of a display device operating in another display mode;

[0014] Fig. 9 yes Figure 8 A block diagram of a display device.

[0015] Description of Reference Numerals

[0016] 10, 10A, 10B: display device;

[0017] 100, 100A: display screen;

[0018] 100ds: display surface;

[0019] 100e: display edge;

[0020] 150: projection element;

[0021] 200, 200A: control module;

[0022] 210: eye position calculation unit;

[0023] 230: image processing unit;

[0024] 250: Camera field of view calculation unit;

[0025] 270: display area division unit;

[0026] CAM1: first camera element;

[0027] CAM2: the second camera element;

[0028] DA1: first display area;

[0029] DA2: second display area;

[0030] FC: focal plane;

[0031] FOV, uFOV: field of view;

[0032] h, x1, x2: distance;

[0033] IFIM: Information Image;

[0034] IM: scene image;

[0035] LEYE: left eye;

[0036] OBJ1, OBJ2, OBJ3: objects;

[0037] OBJ1_IM, OBJ2_IM, OBJ3_IM: object images;

[0038] P, P_CAM: position;

[0039] P_EYE: eye position;

[0040] pFOV: partial field of view;

[0041] pIM: partial scene image;

[0042] REYE: right eye;

[0043] SL: Line of sight;

[0044] USR: user;

[0045] α, β: angle. DETAILED DESCRIPTION

[0046] Reference will now be made in detail to exemplary embodiments of the present invention, examples of which are illustrated in the accompanying drawings. Whenever possible, the same reference numerals are used in the drawings and the description to refer to the same or like parts.

[0047] In the accompanying drawings, for the sake of clarity, the thickness of layers, films, panels, regions, etc. is magnified. It should be understood that when an element such as a layer, film, region, or substrate is referred to as being "on" or "connected to" another element, it may be directly on or connected to another element, or an intermediate element may also exist. On the contrary, when an element is referred to as being "directly on" or "directly connected to" another element, there is no intermediate element. As used herein, "connection" may refer to physical and / or electrical connection. Furthermore, "electrical connection" may be the presence of other elements between two elements.

[0048] Reference will now be made in detail to exemplary embodiments of the present invention, examples of which are illustrated in the accompanying drawings. Whenever possible, the same reference numerals are used in the drawings and the description to refer to the same or like parts.

[0049] Figure 1A and Figure 1B is a schematic diagram of an application of a display device according to an embodiment of the present invention. Figure 2 yes Figure 1A and Figure 1B A schematic top view of the display device when in use. Figure 3 yes Figure 1A A block diagram of a display device. Figure 4 yes Figure 1B A schematic diagram of a scene image captured by a second imaging element.

[0050] Please refer to Figure 1A , Figure 1B and Figure 2 The display device 10 includes a display screen 100. The display screen 100 has a display surface 100ds facing the user USR. In this embodiment, the display screen 100 can be set in a screen body (not shown) of a notebook computer. That is, the display device 10 can be used as a display screen of a notebook computer, but is not limited thereto. In other applications, the display device 10 can also be a desktop screen, such as a professional drawing screen or an e-sports screen.

[0051] In this embodiment, the display screen 100 can be a non-transparent display panel, for example, including a non-self-luminous display panel (such as a liquid crystal display (LCD) panel or an electrophoretic display (EPD) panel) or a self-luminous display panel (such as a micro light emitting diode (micro-LED) display panel, an organic light emitting diode (OLED) display panel or a sub-millimeter light emitting diode (mini light emitting diode, mini-LED) display panel), but is not limited to this.

[0052] Therefore, when the user USR operates a laptop computer indoors, some of the furnishings or objects in the environment (such as object OBJ1, object OBJ2, and object OBJ3) cannot be seen because the opaque display screen 100 (or the screen body not shown) blocks the user USR's line of sight SL. As shown in the figure, in this embodiment, object OBJ1, object OBJ2, and object OBJ3 are, for example, a table, a cup, and a book, respectively, but are not limited thereto.

[0053] In order to make the display device 10 have a perspective display effect, the display device 10 is further configured with a first camera element CAM1 and a second camera element CAM2. The first camera element CAM1 is disposed on one side of the display surface 100ds of the display screen 100, and is used to track the eye position P_EYE of the user USR located on one side of the display surface 100ds of the display screen 100. The second camera element CAM2 is disposed on the side of the display screen 100 facing away from the display surface 100ds, and is used to capture the scene image IM (such as the object OBJ1, the object OBJ2 and the object OBJ3) blocked by the display screen 100. Figure 4 shown).

[0054] Please refer to Figure 2 and Figure 3 , further, the display device 10 further includes a control module 200, electrically coupled to the first camera element CAM1, the second camera element CAM2 and the display screen 100. The control module 200 is adapted to display at least a portion of the scene image IM captured by the second camera element CAM2 on the display surface 100ds of the display screen 100 according to the eye position P_EYE of the user USR tracked by the first camera element CAM1.

[0055] First, it should be noted that in this embodiment, no matter how the eyes of the user USR move, the scene images (such as object image OBJ1_IM, object image OBJ2_IM, and object image OBJ3_IM) displayed on the display screen 100 can accurately present the true appearance of the rear scene blocked by the display screen 100, so that the user USR can have a visual experience of seeing through the rear scene through the display screen 100. More specifically, in order to enhance the realism of the perspective effect of the display device 10, the scene images displayed on the display screen 100 are adjusted as the eyes of the user USR move.

[0056] In detail, the control module 200 may include an eye position calculation unit 210 and an image processing unit 230. The eye position calculation unit 210 is adapted to calculate the eye position P_EYE of the user USR according to the user image captured by the first camera element CAM1. The image processing unit 230 is adapted to adjust the scene image according to the relative relationship between the eye position P_EYE calculated by the eye position calculation unit 210 and the position P_CAM (i.e., the shooting position) of the second camera element CAM2.

[0057] Since the second camera element CAM2 of the present embodiment is disposed on the side of the display screen 100 facing away from the user USR, i.e., closer to the scene to be photographed than the position of the user USR, the field of view FOV that can be photographed by the second camera element CAM2 must be larger than the field of view uFOV defined by the eye position P_EYE of the user USR and the display edge 100e of the display screen 100. In addition, in order to satisfy that the scene image on the display screen 100 can be dynamically adjusted as the viewing position of the user USR changes, the field of view that can be photographed by the second camera element CAM2 must cover the possible viewing angle range of the user USR relative to the display screen 100.

[0058] In this embodiment, the control module 200 may further include a camera field of view calculation unit 250, which is used to obtain the field of view FOV of the second camera element CAM2. For example, the camera field of view calculation unit 250 may calculate the field of view FOV of the second camera element CAM2 according to the optical parameters and characteristics (such as focal length and sensor size, but not limited thereto) of the second camera element CAM2, and determine the shape and coverage of the field of view of the second camera element CAM2 in three-dimensional space according to the position P_CAM of the second camera element CAM2 and the field of view FOV.

[0059] Since the field of view FOV that the second camera element CAM2 can capture is larger than the field of view uFOV defined by the eye position P_EYE of the user USR and the display edge 100e of the display screen 100, the image processing unit 230 is further adapted to capture a portion of the scene image IM captured by the second camera element CAM2 within the field of view FOV according to the relative relationship between the eye position P_EYE of the user USR and the display edge 100e of the display screen 100 (i.e., the partial scene image pIM captured by the second camera element CAM2 within the partial field of view pFOV, such as Figure 4 That is to say, in this embodiment, the control module 200 does not directly display the scene image IM captured by the second camera element CAM2 on the display screen 100.

[0060] Furthermore, the image processing unit 230 is further adapted to adjust the partial scene image pIM extracted from the original scene image IM according to the geometric relationship between the eye position P_EYE of the user USR and the position P_CAM of the second camera element CAM2 relative to the captured scene.

[0061] like Figure 2 As shown, for example, the distance between the eye position P_EYE of the user USR and the display screen 100 along the normal direction of the display surface 100ds is x1. The distance between the eye position P_EYE of the user USR and the position of the scene in the real environment (for example, including objects OBJ1 to OBJ3) along the normal direction of the display surface 100ds is x2. The position of the scene here is, for example, the position of the focusing plane FC of the second camera element CAM2 when capturing the scene image IM, and this focusing plane FC passes through objects OBJ1, object OBJ2 and object OBJ3. On the other hand, the eye position P_EYE of the user USR and the position P_CAM of the second camera element CAM2 (i.e., the shooting position) are parallel to the display surface 100ds (for example Figure 2 The distance between the position P_CAM of the second camera element CAM2 and the position P_CAM of the eye of the user USR and the normal direction of the display surface 100ds is h. The angle between the virtual line connecting any position P in the scene and the eye position P_EYE of the user USR and the normal direction of the display surface 100ds is α. The angle between the virtual line connecting the position P_CAM of the second camera element CAM2 and the position P in the scene and the normal direction of the display surface 100ds is β.

[0062] It is particularly important to note that α and β satisfy the following relationship:

[0063]

[0064] This relationship can correct the image deformation in the scene image caused by the difference in the viewing angle and position of the second camera element CAM2 and the eyes of the user USR (e.g., the left eye LEYE or the right eye REYE) in space. Then, the control module 200 can correctly map the corrected partial scene image pIM to the display screen 100 according to the display size and resolution of the display screen 100 to present the correct scene position and size.

[0065] Through the above-mentioned image processing steps, the control module 200 of this embodiment can adjust the scene image captured by the second camera element CAM2 according to the current eye position P_EYE of the user USR and present it in real time on the display screen 100. No matter how the eyes of the user USR move, the scene image on the display screen 100 can accurately present the real appearance of the rear scene blocked by the display screen 100, so that the user USR has the visual experience of seeing through the rear scene through the display screen 100.

[0066] In addition, unlike the current display technology that uses a transparent display panel to achieve a perspective effect, since the present embodiment uses a non-transparent display panel as the display screen 100, the display device 10 can have the advantages of high brightness, high contrast and high resolution in presenting the perspective effect, thereby further enhancing the user USR's sense of perspective of the rear scene during operation.

[0067] Other embodiments will be listed below to illustrate the present disclosure in detail, wherein the same components will be marked with the same symbols, and the description of the same technical content will be omitted. For the omitted parts, please refer to the aforementioned embodiments, which will not be repeated below.

[0068] Figure 5 FIG. 1 is a top view of a display device in use according to an embodiment of the present invention. Figure 6 yes Figure 5 Please refer to the block diagram of the display device. Figure 5 and Figure 6 The display device 10A of this embodiment and Figure 2 The difference between the display device 10A and the display device 10A is that the display mechanism of the display surface is different. For example, in this embodiment, the display device 10A may further include a projection element 150 electrically coupled to the image processing unit 230 of the control module 200.

[0069] The projection element 150 is adapted to project the scene image processed by the image processing unit 230 onto the display surface 100ds of the display screen 100A. That is, in this embodiment, the display screen 100A may be a projection screen of the projection element 150, but is not limited thereto. Therefore, in this embodiment, the control module 200 may correctly map the corrected scene image onto the display screen 100A according to the projection mechanism of the display screen 100A to present the correct scene position and size.

[0070] Since the other parts of the display device 10A are arranged similarly to Figure 2 For detailed description of the display device 10, please refer to the relevant paragraphs of the aforementioned embodiment, which will not be repeated here.

[0071] Figure 7 yes Figure 1A A schematic diagram of a display device operating in another display mode. Figure 8 yes Figure 1A A top view schematically showing a display device operating in another display mode. Fig. 9 yes Figure 8 Please refer to the block diagram of the display device. Figure 7 and Figure 8 , except that the display device 10 is Figure 1A In addition to the operation mode, the display device 10B of this embodiment can further achieve a perspective effect in a local area.

[0072] For example, the display surface 100ds of the display screen 100 may have a first display area DA1 and a second display area DA2, wherein the control module 200 is suitable for displaying different partial scene images (such as object image OBJ1_IM and object image OBJ2_IM) and information image IFIM in the first display area DA1 and the second display area DA2 respectively.

[0073] That is, the virtual line between a portion of the real scene corresponding to the partial scene image (eg, object OBJ2) and the eye position P_EYE of the user USR passes through the first display area DA1 of the display screen 100. Figure 8 It should be noted that, in this embodiment, although the number of the first display area DA1 is one for exemplary description, in other implementations, the number of the first display area DA1 or the second display area DA2 may also be multiple, depending on actual application requirements.

[0074] In this embodiment, the control module 200A may further include a display area division unit 270, which is adapted to divide the display area of ​​the display screen 100 (i.e., the distribution range of the display surface 100ds) into two or more display areas, and locate these display areas on the display screen 100. The image processing unit 230 is also adapted to correspondingly capture a portion of the scene image captured by the second camera element CAM2 according to the position of the first display area DA1 divided by the display area division unit 270 on the display surface 100ds.

[0075] The captured partial scene image can also use the image processing steps mentioned in the above embodiment to perform image deformation correction and adjust the mapping relationship when the image is displayed on the display screen 100. Therefore, the detailed description can be found in the relevant paragraphs of the above embodiment, which will not be repeated here.

[0076] It is particularly noted that the display device 10B with a local area perspective effect can be widely used in the field of design or professional drawing. For example, for a user who is drawing, the tools on the drawing software and the actual drawing materials behind the display screen 100 can be seen at the same time through the multiple split display areas of the display screen 100. For users in a meeting, the meeting notes and the facial expressions of the participants can be seen at the same time through the multiple split display areas of the display screen 100. Regardless of the usage scenario, this local area perspective effect can allow users to have a different experience in understanding the displayed content and the way of interaction.

[0077] In summary, in a display device of one embodiment of the present invention, a first camera element and a second camera element are respectively provided on opposite sides of a display screen. The first camera element is used to track the eye position of the user. The second camera element is used to capture the image of the scene that is located on the side of the display screen facing away from the user and is blocked by the display screen. By tracking the current eye position of the user, the control module can correspondingly adjust the scene image captured by the second camera element and present it on the display screen in real time. No matter how the user's eyes move, the scene image on the display screen can accurately present the true appearance of the rear scene blocked by the display screen, allowing the user to have a visual experience of seeing through the rear scene through the display screen.

[0078] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A display device, characterized in that: include: A display screen having a display surface facing a user; A first camera element is disposed on one side of the display surface of the display screen and is used to track the eye position of the user; A second imaging element is disposed on a side of the display screen facing away from the display surface and is used to capture an image of a scene that is in the user's sight and blocked by the display screen; as well as The control module is adapted to display at least a portion of the scene image on the display screen according to the eye position of the user.

2. The display device according to claim 1, characterized in that The control module comprises: The eye position calculation unit is adapted to calculate the eye position of the user based on the user image captured by the first camera element.

3. The display device according to claim 1, characterized in that The control module comprises: The image processing unit is adapted to adjust the scene image according to the relative relationship between the eye position of the user and the shooting position of the second camera element.

4. The display device according to claim 3, characterized in that The distance between the eye position of the user and the display screen along the normal direction of the display surface is x1, the distance between the eye position of the user and the scene along the normal direction of the display surface is x2, the distance between the eye position and the shooting position of the second camera element along the direction parallel to the display surface is h, the angle between the virtual line connecting the eye position and any position of the scene and the normal direction of the display surface is α, the angle between the virtual line connecting the shooting position of the second camera element and any position of the scene and the normal direction of the display surface is β, and α and β satisfy the following relationship:

5. The display device according to claim 1, characterized in that The control module comprises: A camera field of view calculation unit, used to obtain the field of view of the second imaging element; and The image processing unit is adapted to capture part of the scene image captured by the second camera element within part of the field of view according to the relative relationship between the eye position of the user and the display edge of the display screen.

6. The display device according to claim 5, characterized in that: The image processing unit is further adapted to adjust the partial scene image according to the relative relationship between the eye position of the user and the shooting position of the second camera element.

7. The display device according to claim 6, characterized in that: The distance between the eye position of the user and the display screen along the normal direction of the display surface is x1, the distance between the eye position of the user and the scene along the normal direction of the display surface is x2, the distance between the eye position and the shooting position of the second camera element along the direction parallel to the display surface is h, the angle between the virtual line connecting the eye position and any position of the scene and the normal direction of the display surface is α, the angle between the virtual line connecting the shooting position of the second camera element and any position of the scene and the normal direction of the display surface is β, and α and β satisfy the following relationship:

8. The display device according to claim 1, characterized in that: The display surface of the display screen is provided with a first display area and a second display area, and the control module is suitable for displaying part of the scene image and the information image in the first display area and the second display area respectively, the information image is different from the scene image, and a virtual line between the part of the scene corresponding to the part of the scene image and the eye position of the user passes through the first display area of ​​the display screen.

9. The display device according to claim 8, characterized in that: The control module comprises: A display area division unit, adapted to locate the first display area and the second display area on the display screen; and The image processing unit is adapted to capture a portion of the scene image photographed by the second imaging element according to the position of the first display area on the display screen.

10. The display device according to claim 1, characterized in that: The display screen is a non-transparent display panel.

11. The display device according to claim 1, characterized in that: Also includes: A projection element is adapted to project at least a portion of the scene image onto the display surface of the display screen.

Images