Display device, driving method thereof, and display system

Abstract

The application provides a display device, a driving method thereof and a display system. The display device comprises an image acquisition unit, an image generation unit, a light guide assembly and a light modulation layer. The image generation unit comprises a logic processing unit and a projection unit. The logic processing unit is electrically connected with the image acquisition unit, used for acquiring an environmental image and generating an associated augmented reality image. The projection unit is electrically connected with the logic processing unit, used for emitting light of the augmented reality image. The logic processing unit is further used for identifying each element of the environmental image, generating a corresponding brightness adjustment background image according to the type of each element, and superimposing the augmented reality image to generate an augmented reality superimposed picture. The display device can improve the brightness of the environmental light transmitted through the display device and improve the fusion effect of the AR picture and the real environment.

Description

Technical Field

[0001] This application relates to the field of display technology, and in particular to a display device and an image display method thereof. Background Technology

[0002] With the rapid development of semiconductor technology, the ways humans interact with computers are evolving at an unprecedented pace. Among these advancements, Augmented Reality (AR) can provide users with more multi-dimensional information, and AR glasses are one of the important mediums in the AR field. However, because AR glasses transmit ambient light, they suffer from poor contrast when used in bright environments, making it impossible to display dark AR elements, such as black text, which limits the widespread adoption of AR glasses.

[0003] Currently, to solve the aforementioned technical problems, devices such as electrochromic lenses or twisted nematic liquid crystal light valves are typically used to adjust brightness. To achieve the ideal contrast in AR images, it is necessary to significantly reduce the transmittance of ambient light. However, this results in a perceived darker ambient light, leading to a poorer blending effect between the AR image and the real environment. Summary of the Invention

[0004] This application provides a display device, its driving method, and a display system, aiming to solve the problem in the prior art where the low ambient light transmittance of AR display devices leads to a poor fusion effect between AR images and the real environment.

[0005] To address the aforementioned technical problems, the first technical solution provided in this application is: to provide a display device. The display device includes:

[0006] Image acquisition unit, used to acquire environmental images;

[0007] The image generation unit includes a logic processing unit and a projection unit; the logic processing unit is electrically connected to the image acquisition unit and is used to acquire environmental images and generate associated augmented reality images; the projection unit is electrically connected to the logic processing unit and is used to emit light for the augmented reality images.

[0008] A light guide assembly includes a light guide plate, a coupling unit, and a coupling unit; the coupling unit and the coupling unit are disposed on the same side of the light guide plate; the coupling unit is opposite to the projection unit and is used to couple the light emitted by the projection unit into the light guide plate; the coupling unit is used to couple the light coupled into the light guide plate out of the light guide plate.

[0009] A dimming layer, located on the other side of the light guide plate, is used to control the transmittance of ambient light.

[0010] The logic processing unit is also used to identify each element of the environmental image, generate a corresponding brightness-adjusted background image according to the type of each element, and overlay the augmented reality image to generate an augmented reality overlay image.

[0011] In some embodiments, the logic processing unit is used to extract each element of the environment image, identify the type of each element, and determine the corresponding ambient light brightening level according to the type of each element to generate a corresponding brightness-adjusted background image; wherein, the higher the correlation between the elements of the environment image and the augmented reality image, the greater the corresponding ambient light brightening level.

[0012] In some embodiments, the logic processing unit is further configured to reduce the brightening effect of the region corresponding to the augmented reality image in the brightness adjustment background image.

[0013] In some embodiments, the field of view of the image acquisition unit is greater than the field of view of the output unit; the logic processing unit is further configured to crop and adjust the viewing angle of the environmental image according to the field of view of the output unit to form an environmental background image of the same size as the display area of ​​the augmented reality image.

[0014] In some embodiments, when the display device is in a wearing state, the image acquisition unit is located above the coupling unit and is coaxially arranged.

[0015] In some embodiments, a light guide assembly and a dimming layer constitute a display lens, and the display device includes two display lenses fixed in relative positions;

[0016] Each display lens is equipped with an image acquisition unit, and the two image acquisition units are respectively located on the side of the two display lenses that are close to each other or on the side that are far apart from each other; or,

[0017] Each display lens is equipped with at least two image acquisition units, and the at least two image acquisition units are arranged around the edge of the display lens.

[0018] In some embodiments, the light transmittance of the low transmittance region corresponding to the dimming layer and the coupling unit is 20% to 40%, and the light transmittance of the dimming layer gradually increases to 80% to 95% along the direction away from the low transmittance region.

[0019] To address the aforementioned technical problems, the second technical solution provided in this application is: a driving method for a display device. The display device is the same as described in the above technical solution, and the driving method for the display device includes:

[0020] Real-time acquisition of environmental images;

[0021] Identify the elements of the environmental image and determine the type of each element;

[0022] Augmented reality images of the corresponding environment are generated based on the type of each element;

[0023] Based on the type of each element, determine the corresponding ambient light brightening level and generate the corresponding brightness-adjusted background image;

[0024] An augmented reality image is overlaid onto a brightness-adjusted background image to generate an augmented reality overlay.

[0025] The information of the augmented reality overlay is transmitted to the projection unit, so that the augmented reality overlay is integrated with the environment in real time.

[0026] In some implementations, the step of acquiring environmental images in real time is followed by:

[0027] The environmental image is cropped and the viewing angle is adjusted to form an environmental background image of the same size as the display area of ​​the augmented reality image;

[0028] The steps for determining the corresponding ambient light brightening level based on the type of each element and generating the corresponding brightness-adjusted background image include:

[0029] Based on the type of each element, determine the brightness enhancement level of the ambient light corresponding to each element, and generate a preliminary brightness-adjusted background image;

[0030] Identify the corresponding augmented reality element regions on the initial brightness-adjusted background image of the augmented reality image;

[0031] The brightness of the augmented reality element areas in the initial brightness adjustment background image is reduced to create a brightness-adjusted background image.

[0032] To address the aforementioned technical problems, the third technical solution provided in this application is: to provide a display system. This display system includes the display device as described in the above technical solutions.

[0033] The beneficial effects of this application: Unlike the prior art, this application provides a display device, its driving method, and a display system. The display device includes an image acquisition unit, an image generation unit, a light guide assembly, and a dimming layer. The image generation unit includes a logic processing unit and a projection unit. The logic processing unit is connected to the image acquisition unit and is used to acquire an environmental image acquired by the image acquisition unit and generate an augmented reality image associated with the environmental image. The projection unit is used to receive information from the augmented reality image and emit light to display the image. The light guide assembly includes a light guide plate and a coupling unit and a coupling unit disposed on the same side of the light guide plate. The coupling unit is used to couple light emitted by the projection unit into the light guide plate, and the coupling unit is used to couple light coupled into the light guide plate out of the light guide plate, so as to propagate the light of the augmented reality image to the user's eyes, realizing the fusion of the augmented reality image and the real environment. In the embodiments of this application, the logic processing unit is also used to identify each element of the environmental image and generate a corresponding brightness adjustment background image according to the type of each element. That is, adjusting the brightness of the area corresponding to each element in the brightness adjustment background image can appropriately brighten the ambient light in the corresponding area, thereby improving the problem of dim ambient light. Meanwhile, the logic unit is also used to overlay augmented reality images onto the brightness adjustment background image to generate an augmented reality overlay image. This enables intelligent brightening of ambient light while integrating augmented reality images into the real environment, improving the fusion effect between augmented reality images and the real environment. Attached Figure Description

[0034] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.

[0035] Figure 1 This is a schematic diagram of the structure of the display device provided in the first embodiment of this application;

[0036] Figure 2 yes Figure 1 A cross-sectional view of the display device in the embodiment along the AA direction;

[0037] Figure 3 This is a schematic diagram of an environmental image captured by an image acquisition unit provided in an embodiment of this application;

[0038] Figure 4 This is a schematic diagram of a preprocessed environmental image provided in an embodiment of this application;

[0039] Figure 5 This is a schematic diagram of a brightness-adjustable background image provided in an embodiment of this application;

[0040] Figure 6 This is a schematic diagram of a brightness-adjustable background image provided in another embodiment of this application;

[0041] Figure 7 This is a schematic diagram of an augmented reality image provided in an embodiment of this application;

[0042] Figure 8 This is a schematic diagram of an augmented reality overlay provided in an embodiment of this application;

[0043] Figure 9 This is a schematic diagram of the structure of the display device provided in the second embodiment of this application;

[0044] Figure 10 This is a schematic diagram of the structure of the display device provided in the third embodiment of this application;

[0045] Figure 11 This is a schematic diagram of the structure of the display device provided in the fourth embodiment of this application;

[0046] Figure 12 This is a schematic diagram of the dimming layer provided in an embodiment of this application;

[0047] Figure 13 A schematic flowchart of a driving method for a display device provided in one embodiment of this application;

[0048] Figure 14 This is a schematic flowchart of a driving method for a display device provided in another embodiment of this application;

[0049] Figure 15 This is a schematic diagram of the structure of a display system provided in an embodiment of this application.

[0050] Figure label:

[0051] 100. Display device; 101. Display lens; 10. Image acquisition unit; 20. Image generation unit; 21. Logic processing unit; 22. Projection unit; 30. Light guide assembly; 31. Light guide plate; 32. Coupler unit; 33. Coupler output unit; 40. Dimming layer; 41. Low transmittance area; P1. Environmental image; P11. Preprocessed environmental image; P2. Augmented reality image; P3. Brightness-adjusted background image; P4. Augmented reality overlay image. Detailed Implementation

[0052] The embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0053] In the following description, specific details such as particular system architectures, interfaces, and technologies are presented for illustrative purposes rather than for limiting purposes, in order to provide a thorough understanding of this application.

[0054] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0055] The terms "first," "second," and "third" in this application are for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first," "second," or "third" may explicitly or implicitly include at least one of that feature. In the description of this application, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified. All directional indications (such as up, down, left, right, front, back, etc.) in the embodiments of this application are only used to explain the relative positional relationships and movements between components in a specific orientation (as shown in the figures). If the specific orientation changes, the directional indications also change accordingly. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices.

[0056] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0057] The present application will now be described in detail with reference to the accompanying drawings and embodiments.

[0058] Please see Figure 1 and Figure 2 , Figure 1 This is a schematic diagram of the structure of the display device provided in the first embodiment of this application. Figure 2 yes Figure 1 A schematic cross-sectional view of the display device in the embodiment along the AA direction. In this embodiment, a display device 100 is provided, which includes:

[0059] Image acquisition unit 10 is used to acquire environmental image P1;

[0060] The image generation unit 20 includes a logic processing unit 21 and a projection unit 22; the logic processing unit 21 is electrically connected to the image acquisition unit 10 and is used to acquire an environmental image P1 and generate an associated augmented reality image P2; the projection unit 22 is electrically connected to the logic processing unit 21 and is used to emit light from the augmented reality image P2.

[0061] The light guide assembly 30 includes a light guide plate 31, a coupling unit 32, and a coupling unit 33. The coupling unit 32 and the coupling unit 33 are disposed on the same side of the light guide plate 31. The coupling unit 32 is opposite to the projection unit 22 and is used to couple the light emitted by the projection unit 22 into the light guide plate 31. The coupling unit 33 is used to couple the light coupled into the light guide plate 31 out of the light guide plate 31.

[0062] A dimming layer 40 is disposed on the other side of the light guide plate 31 and is used to control the transmittance of ambient light.

[0063] The image acquisition unit 10 is located on one side of the light guide assembly 30 and is used to acquire environmental images P1 in real time to provide a display environment of the user's surroundings for scene analysis and object recognition, i.e., for environmental element recognition, providing a basis for the generation of AR elements associated with environmental elements. The image acquisition unit 10 can specifically be a camera.

[0064] The logic processing unit 21 is electrically connected to the image acquisition unit 10 and the projection unit 22. The logic processing unit 21 acquires the environmental image P1 acquired by the image acquisition unit 10, preprocesses the environmental image P1, generates an augmented reality image P2 associated with the environmental image P1, and transmits the information of the associated augmented reality image P2 to the projection unit 22 in the form of an electrical signal. The projection unit 22 receives the electrical signal of the augmented reality image P2 and emits light according to the electrical signal, thereby transmitting the augmented reality image P2 as an optical signal to the light guide component 30.

[0065] The projection unit 22 may specifically include a microdisplay and optical components. The microdisplay may specifically be a self-emissive display such as a micro organic light-emitting diode (Micro OLED) display or a micro light-emitting diode (Micro LED) display. The optical components may include lenses, prisms, waveguides, and other components, which can be used to magnify, focus, and project light.

[0066] The light guide assembly 30 includes a light guide plate 31 and coupling-in unit 32 and coupling-out unit 33 disposed on the light guide plate 31. The coupling-in unit 32 is disposed opposite to the projection unit 22, that is, it is placed in the optical path of the emitted light from the projection unit 22 to receive the light emitted by the projection unit 22 and guide the light emitted by the projection unit 22 into the light guide plate 31. The light is then transmitted through the light guide plate 31 to the coupling-out unit 33, which couples the light out of the light guide plate 31 so that the light can propagate to the user's eyes, allowing the user to view the virtual image (augmented reality image P2). The coupling-in unit 32 and coupling-out unit 33 can specifically be a grating structure or a semi-transparent, semi-reflective structure for coupling in and coupling out light. The light guide plate 31 can be an optical waveguide plate, specifically any one of a diffractive optical waveguide substrate, a volume holographic optical waveguide substrate, an array optical waveguide substrate, or a freeform prism waveguide substrate, so as to efficiently propagate image light to the user's eyes through the waveguide structure and maintain transparency to facilitate the transmission of external ambient light to the user's eyes, so that the user can simultaneously view the real environment, thereby realizing the superposition of augmented reality image P2 with the real environment.

[0067] The dimming layer 40 is disposed on the surface of the light guide plate 31 facing the external environment and is used to control the transmittance of ambient light. Specifically, the dimming layer 40 can be an electrochromic film, a photochromic film, or a twisted nematic liquid crystal light valve, or other film devices that can adjust the transmittance of light. By setting the dimming layer 40, the transmittance of the dimming layer 40 can be controlled according to the brightness of the actual environment, so that the brightness of the ambient light after passing through the light guide component 30 can meet the contrast requirements of the AR image and the surrounding environment.

[0068] In this embodiment of the application, the logic processing unit 21 is further configured to identify the environmental image P1 (see Figure 3 Each element of the image is used to generate a corresponding brightness-adjusted background image P3 (see [link]). Figure 5 and Figure 6 ), and overlay the augmented reality image P2 (see Figure 7 ), generating augmented reality overlay images P4 (see Figure 8 ).

[0069] Specifically, after the environmental image P1 is preprocessed, the logic processing unit 21 further identifies each element in the preprocessed environmental image P11, such as scenery, sky, objects, and background. Then, based on the type of each element in the image, it determines the corresponding brightening level, thereby generating a corresponding brightness adjustment background image P3. This brightens the ambient light corresponding to each element in the environmental image P1 by the appropriate level. For example, if the environmental image P1 includes scenery elements and sky elements, since the brightness of the ambient light of the sky element is greater than that of the scenery element, the brightening level of the corresponding area of ​​the sky can be less than that of the corresponding area of ​​the scenery, or the corresponding area of ​​the sky can be left unbrightened. That is, the brightness adjustment background image P3 can brighten the ambient light in zones, thus satisfying both the contrast between the augmented reality image P2 and the surrounding real environment, and also brightening the ambient light of key elements in the displayed environment. Key elements are those that are relatively important and those associated with the augmented reality image P2. Specifically, by overlaying the augmented reality image P2 onto the brightness-adjusted background image, an augmented reality overlay image P4 is generated. This allows for the simultaneous brightening of the ambient light corresponding to each element in the real environment while projecting the augmented reality image, thereby effectively improving the integration of the augmented reality image P2 with the real environment and enhancing the user experience.

[0070] Please see Figure 3 and Figure 4 , Figure 3 This is a schematic diagram of an environmental image captured by an image acquisition unit provided in an embodiment of this application. Figure 4 This is a schematic diagram of a preprocessed environmental image provided in one embodiment of this application. In some embodiments, the field of view of the image acquisition unit 10 is larger than the field of view of the coupling unit 33; the logic processing unit 21 is further configured to crop and adjust the viewing angle of the environmental image P1 according to the field of view of the coupling unit 33, forming an environmental background image with the same size as the display area of ​​the augmented reality image P2.

[0071] Specifically, the field of view (FOV) of the image acquisition unit 10 is 50°~70°, for example, it can be 50°, 55°, 60°, 65° or 70°; the field of view of the coupling unit 33 is 20°~45°, for example, it can be 20°, 25°, 30°, 35°, 40° or 45°. By making the field of view of the image acquisition unit 10 larger than that of the coupling unit 33, the image acquisition unit 10 acquires a larger field of view of the environmental image P1, reserving processing space for subsequent preprocessing of the environmental image P1.

[0072] Specifically, the logic processing unit 21 preprocesses the environmental image P1 acquired by the image acquisition unit 10. This preprocessing includes operations such as cropping and viewing angle adjustment. Specifically, the environmental image P1 can be cropped according to the field of view of the coupling unit 33 to ensure that the field of view of the environmental image P1 matches that of the coupling unit 33, and that the size of the cropped environmental image P1 is the same as the size of the display area of ​​the augmented reality image P2. Furthermore, the logic processing unit 21 also adjusts the viewing angle of the environmental image P1 to match the user's viewing angle, while correcting distortions and parallax in the environmental image P1. After the above preprocessing, the environmental image P11 has the same viewing angle and image size as the real environment seen by the user from the display device 100.

[0073] Please refer to the following: Figure 4 and Figure 5 , Figure 5 This is a schematic diagram of a brightness adjustment background image provided in an embodiment of this application. In this embodiment, the logic processing unit 21 is used to extract each element of the environment image P1, identify the type of each element, and determine the corresponding ambient light brightening range according to the type of each element, so as to generate the corresponding brightness adjustment background image P3; wherein, the higher the correlation between the elements of the environment image P1 and the augmented reality image P2, the greater the corresponding ambient light brightening range.

[0074] After preprocessing the environmental image P11, the logic processing unit 21 performs element recognition, classifying each element in the environmental image P1 and determining the brightness enhancement level of the ambient light corresponding to the element based on the element type. Different element types correspond to different brightness enhancement levels to generate a brightness-adjusted background image P3 corresponding to each element in the environmental image P1. The element types can be classified according to element category, including background, scenery, and object types; or according to importance, including first, second, and third categories, with importance decreasing sequentially (first category is the most important, then second, and third); alternatively, the element types can be first classified according to element category, including background, scenery, and object types, and then the scenery and / or object categories can be second-classified according to element importance, including first, second, and third categories. The background category can be second-classified according to its original brightness, including first brightness category, second brightness category, and third brightness category.

[0075] Different types of elements correspond to a brightness enhancement level, i.e., a brightening level. The logic processing unit 21 determines the corresponding brightness enhancement level for each element in the environmental image P1 based on its type, thereby forming a complete brightness-adjusted background image P3. Specifically, significance weights can be intelligently assigned according to element type, and brightness adjustment is performed according to the weights. Among them, elements with a high degree of correlation with the augmented reality image P2 have higher weights, while those with low correlation have lower weights. The higher the weight, the higher the brightening level, and the lower the weight, the lower the brightening level. For example, in this embodiment, the weight of the already bright sky element is set lower to reduce its brightening level, while the weight of meaningful scene elements that need to be merged with the augmented reality image P2 is set higher to increase their brightening level, thereby improving the ambient light brightening effect and enhancing the fusion effect between the augmented reality image P2 and the real environment.

[0076] For example, such as Figure 4 and Figure 5 As shown, the environmental image P1 includes scene elements and sky elements. Scene elements can be categorized into a scene category, and sky elements into a background category. The sky elements can be further subdivided into a third category: brightness. Scene elements, being more important, have a higher correlation with the augmented reality image P2, and their corresponding areas receive higher brightness enhancement. Sky elements, as background elements, have relatively high brightness and a lower correlation with the augmented reality image P2. Excessive brightness in the sky would reduce the contrast of the augmented reality image P2; therefore, the brightness enhancement level for the sky element's corresponding area is set lower, or not enhanced at all. The logic processing unit 21 generates a brightness-adjusted background image P3 corresponding to the background image according to the above method. In this brightness-adjusted background image P3, the brightness of the areas corresponding to scene elements is higher, and the brightness of the areas corresponding to sky elements is lower. This allows for zoned brightening of the ambient light transmitted through the display device 100, maintaining good contrast in the AR element areas of the augmented reality image P2 while effectively brightening the ambient light, improving the integration of AR elements with the real environment, and enhancing the user experience. AR elements are graphics in augmented reality image P2 that are integrated with the real environment, such as directional arrows and text.

[0077] Please see Figures 6-8 , Figure 6 This is a schematic diagram of a brightness adjustment background image provided in another embodiment of this application. Figure 7 This is a schematic diagram of an augmented reality image provided in an embodiment of this application. Figure 8 This is a schematic diagram of an augmented reality overlay image provided in one embodiment of this application. In some embodiments, the logic processing unit 21 is further configured to reduce the brightness adjustment of the area in the background image P3 corresponding to the augmented reality image P2.

[0078] Specifically, in order to further improve the contrast of AR elements in augmented reality image P2, logic processing unit 21 reduces the brightness of the area corresponding to the augmented display image in brightness adjustment background image P3. As a result, after the augmented display image is superimposed on the corresponding area in brightness adjustment background image P3, the AR elements are displayed more clearly, thus achieving a higher contrast of AR elements.

[0079] Please continue to refer to this. Figure 1 In this embodiment, when the display device 100 is in the wearing state, the image acquisition unit 10 is located above the coupling unit 33 and coaxially arranged. Specifically, the display device 100 can be a near-eye display device 100. When the display device 100 is in the wearing state, the image acquisition unit 10 can be arranged on the frame of the light guide assembly 30 and located directly above the coupling unit 33, that is, the center of the image acquisition unit 10 and the center of the coupling unit 33 are on the same straight line, and the extension direction of this straight line is vertical. After such arrangement, the focal point of the environmental image P1 acquired by the image acquisition unit 10 and the focal point of the image projected by the coupling unit 33 have the same horizontal coordinate, so that when the logic processing unit 21 adjusts the viewing angle of the environmental image P1, it only needs to correct the viewing angle in the vertical direction, which can reduce the difficulty of image parallax correction.

[0080] Please see Figure 9 and Figure 10 , Figure 9 This is a schematic diagram of the structure of the display device provided in the second embodiment of this application. Figure 10 This is a schematic diagram of the structure of the display device provided in the third embodiment of this application. In this embodiment, the display device 100 is a near-eye display device 100, wherein a light guide component 30 and a dimming layer 40 constitute a display lens 101, and the display device 100 includes two display lenses 101 with fixed relative positions; each display lens 101 is equipped with an image acquisition unit 10, and the two image acquisition units 10 are respectively disposed on the side of the two display lenses 101 that are close to each other or on the side that are far from each other.

[0081] Specifically, the display device 100 includes two display lenses 101 with fixed relative positions. That is, the position of one display lens 101 relative to the position of the other display lens 101 is fixed. Alternatively, it can be understood that, with one display lens 101 as a reference point, the distance between the other display lens 101 and the first display lens 101 is fixed, and the orientation of the other display lens 101 relative to the first display lens 101 is also fixed. Each display lens 101 includes a light guide assembly 30 and a dimming layer 40 disposed on the surface of the light guide assembly 30. Each display lens 101 is equipped with an image acquisition unit 10.

[0082] like Figure 9As shown, the image acquisition unit 10 is disposed on the side of one display lens 101 closest to the other; that is, the image acquisition unit 10 is disposed on the side of the two display lenses 101 that are close to each other. Or, as... Figure 10 As shown, the image acquisition unit 10 can also be located on the side of the corresponding display lens 101 away from the other display lens 101, that is, at the turning point between the display lens 101 and the temple of the glasses. By placing the image acquisition unit 10 on the side where the two display lenses 101 are close to each other or on the side where they are far from each other, it is beneficial to the integrated design and concealed design of the image acquisition unit 10, which can reduce the size of the display device 100 and improve the user comfort and aesthetics of the display device 100.

[0083] Please see Figure 11 , Figure 11 This is a schematic diagram of the structure of the display device provided in the fourth embodiment of this application. In this embodiment, a light guide component 30 and a dimming layer 40 constitute a display lens 101. The display device 100 includes two display lenses 101 with fixed relative positions. Each display lens 101 is equipped with at least two image acquisition units 10, and the at least two image acquisition units 10 are arranged around the edge of the display lens 101.

[0084] Specifically, each display lens 101 is equipped with at least two image acquisition units 10, such as two, three, four, five, or six, and multiple cameras can be evenly spaced around the edge of the display lens 101. For example, each display lens 101 is equipped with four image acquisition units 10, which are respectively located on the upper, lower, left, and right sides of the display lens 101 and are evenly spaced along the circumference of the display lens 101. This arrangement can acquire environmental images P1 from multiple perspectives, making the parallax calibration of the environmental image P1 by the logic processing unit 21 more accurate and improving the image parallax correction accuracy of the logic processing unit 21.

[0085] Please see Figure 12 , Figure 12 This is a schematic diagram of the structure of a dimming layer provided in an embodiment of this application. Further, in some embodiments, the light transmittance of the dimming layer 40 and the low transmittance region 41 corresponding to the coupling unit 33 is 20% to 40%, and the light transmittance of the dimming layer 40 gradually increases to 80% to 95% along the direction away from the low transmittance region 41.

[0086] The projection area of ​​the coupling unit 33 onto the dimming layer 40 is a low-transmittance region 41, with a light transmittance of 20% to 40%, for example, 20%, 25%, 30%, 35%, or 40%. By reducing the light transmittance in this region, the contrast of the augmented reality image P2 can be effectively improved. Furthermore, along the direction away from the low-transmittance region 41, the light transmittance on the dimming layer 40 gradually increases to 80% to 95%, for example, to 80%, 85%, 90%, or 95%. This results in a gradual transition in light transmittance from the boundary of the coupling unit 33 outwards, thereby achieving a gradual blending of the background of the augmented reality image P2 display area with the real environment, thus improving the user's viewing experience.

[0087] Please refer to the following: Figures 1-8 as well as Figure 13 , Figure 13 This is a schematic flowchart illustrating a driving method for a display device according to an embodiment of this application. In this embodiment, a driving method for a display device 100 is provided, which drives the display device 100 provided in the above embodiment to perform display operations. The driving method includes:

[0088] S10: Real-time acquisition of environmental image P1;

[0089] S20: Identify each element of the environmental image P1 and determine the type of each element;

[0090] S30: Generate an augmented reality image P2 corresponding to the environment image P1 according to the type of each element;

[0091] S40: Determine the corresponding ambient light brightening level according to the type of each element, and generate the corresponding brightness adjustment background image P3;

[0092] S50: The augmented reality image is superimposed on the brightness-adjusted background image P3 to generate an augmented reality overlay image P4;

[0093] S60: Transmit the information of the augmented reality overlay P4 to the projection unit 22, so that the augmented reality overlay P4 is integrated with the environment in real time.

[0094] In step S10, the environment can be acquired in real time by the image acquisition unit 10 to obtain an environmental image P1. In step S20, the classification of each element and the type of element in the environmental image P1 is the same as or similar to that in the above embodiment. For details, please refer to the detailed description in the above embodiment, which will not be repeated here.

[0095] The brightness-adjusted background image P3 generated in steps S30 and S40 can brighten the ambient light transmitted through the display device 100 in different areas, thereby improving the contrast of the AR image (augmented reality image P2) and effectively increasing the brightness of the ambient light. Steps S50 and S60 enable the display device 100 to display the AR image and increase the brightness of the ambient light, thereby improving the integration of the AR image with the real environment and enhancing the user's viewing experience.

[0096] Please see Figure 14 , Figure 14 This is a schematic flowchart of a driving method for a display device provided in another embodiment of this application. In this embodiment, after step S10, the method further includes:

[0097] S11: Crop and adjust the viewing angle of the environmental image P1 to form an environmental background image of the same size as the display area of ​​the augmented reality image P2;

[0098] Step S40 specifically includes:

[0099] S41: Determine the brightness enhancement level of the ambient light corresponding to each element according to the type of each element, and generate a preliminary brightness adjustment background image P3;

[0100] S42: Determine the corresponding augmented reality element region on the initial brightness-adjusted background image P3 for augmented reality image P2;

[0101] S43: Reduce the brightness of the augmented reality element areas in the initial brightness adjustment background image P3 to form the brightness adjustment background image P3.

[0102] In this embodiment, after acquiring the environmental image P1, it is preprocessed, including cropping and viewing angle adjustment. Cropping and viewing angle adjustment make the environmental image P1 more realistic, and ensure that the image focus matches the focus of the user when observing the real environment while wearing the display device 100. This results in higher accuracy when the generated AR image is integrated with the real environment. Specifically, cropping parameters and viewing angle adjustment parameters can be calculated based on parameters such as the setting position of the image acquisition unit 10, the field of view, and the focus of the coupling unit 33. Then, the environmental image P1 is preprocessed based on these parameters.

[0103] In step S41, the logic processing unit 21 first determines the required brightening range of the ambient light corresponding to each element in the environment image P1 according to the type of each element, and generates a preliminary brightness adjustment background image P3 for use in partitioning and brightening the ambient light.

[0104] To further improve the contrast of the displayed AR image, in steps S42 and S43, the logic processing unit 21 determines the region corresponding to the AR element in the initial brightness adjustment background image P3, and then reduces the brightness of that region. This creates a dark area in the region corresponding to the AR element within the brightness adjustment background image P3, meaning the dark area is dimmer than its surrounding areas in the brightness adjustment background image P3. Therefore, in step S50, after superimposing the AR image with the brightness adjustment background image P3, the AR element graphics become clearer. Thus, when the augmented reality overlay image P4 is projected onto the user's field of vision and superimposed on the real environment, it can both increase the brightness of ambient light in different areas and maintain a high contrast in the AR image, improving the fusion effect of the AR element with the real environment and resulting in a better user experience.

[0105] Please see Figure 15 , Figure 15 This is a schematic diagram of the structure of a display system provided in one embodiment of this application. In this embodiment, a display system is provided, including a display device 100. The structure and function of the display device 100 are the same as or similar to those of the display device 100 provided in the above embodiments, and can achieve the same technical effects. For details, please refer to the detailed description above; it will not be repeated here. The driving method of the display device 100 is the same as or similar to the driving method of the display device 100 provided in the above embodiments; for details, please refer to the description above; it will not be repeated here.

[0106] The display system may also include other sensors and audiovisual units, thereby enabling more functions and making it suitable for more scenarios.

[0107] The above are merely embodiments of this application and do not limit the scope of patent protection of this application. Any equivalent structural or procedural changes made using the content of this application’s specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of this application.

Claims

1. A display device comprising: an image acquisition unit configured to acquire an environment image; an image generation unit comprising a logic processing unit and a projection unit; the logic processing unit is electrically connected with the image acquisition unit, configured to acquire the environment image and generate an associated augmented reality image; the projection unit is electrically connected with the logic processing unit, configured to emit light of the augmented reality image; a light guide assembly comprising a light guide plate, a coupling-in unit and a coupling-out unit; the coupling-in unit and the coupling-out unit are arranged on the same side of the light guide plate; the coupling-in unit is opposite to the projection unit, configured to couple the light emitted by the projection unit into the light guide plate; the coupling-out unit is configured to couple the light coupled into the light guide plate out of the light guide plate; a light modulation layer arranged on the other side of the light guide plate, configured to control the transmittance of ambient light; characterized in that the logic processing unit is further configured to identify each element of the environment image, generate a corresponding brightness adjustment background image according to the type of each element, and superimpose the augmented reality image to generate an augmented reality superimposed image; the logic processing unit is further configured to extract each element of the environment image and identify the type of each element, determine the brightening amplitude of the corresponding ambient light according to the type of each element, and generate the corresponding brightness adjustment background image; wherein different types of elements correspond to different brightening amplitudes, and the higher the correlation degree of the elements of the environment image and the augmented reality image, the greater the brightening amplitude of the corresponding ambient light.

2. The display device according to claim 1, wherein After generating the brightness adjustment background image, the logic processing unit is further configured to: determine the superimposition area of the augmented reality image on the brightness adjustment background image; reduce the brightening amplitude of the superimposition area.

3. The display device according to claim 1, wherein The field of view of the image acquisition unit is larger than the field of view of the coupling-out unit; the logic processing unit is further configured to crop and adjust the perspective of the environment image according to the field of view of the coupling-out unit, to form an environment background image with the same size as the display area of the augmented reality image.

4. The display device according to any one of claims 1 to 3, wherein When the display device is in a wearing state, the image acquisition unit is located above the coupling-out unit and coaxially arranged.

5. The display device according to any one of claims 1 to 3, wherein One of the light guide assemblies and one of the light modulation layers constitute a display lens, and the display device comprises two display lenses with fixed relative positions; wherein each of the display lenses is provided with one of the image acquisition units, and the two image acquisition units are arranged on the side close to each other or on the side away from each other of the two display lenses; or each of the display lenses is provided with at least two image acquisition units, and the at least two image acquisition units are arranged around the edge of the display lens.

6. The display device according to any one of claims 1 to 3, wherein The transmittance of the light of the low transmittance area corresponding to the light modulation layer and the coupling-out unit is 20% to 40%, and the transmittance of the light of the light modulation layer gradually increases to 80% to 95% in the direction away from the low transmittance area.

7. A driving method of a display device, the display device being the display device according to any one of claims 1 to 6, characterized by, The driving method comprises: acquiring an environment image in real time; identifying each element of the environment image and determining the type of each element; generating an associated augmented reality image according to the environment image; The brightness adjustment background image is generated according to the brightness adjustment range of the corresponding ambient light of each element; The augmented reality image is superimposed into the brightness adjustment background image to generate an augmented reality superimposition picture; Information of the augmented reality superimposition picture is transmitted to a projection unit, so that the augmented reality superimposition picture is fused with the environment in real time.

8. The driving method according to claim 7, wherein The step of acquiring the environment image in real time further comprises: The environment image is cropped and perspective adjusted to form an environment background image with the same perspective and image size as the real environment observed from the display device; The step of generating the brightness adjustment background image according to the brightness adjustment range of the corresponding ambient light of each element comprises: The brightness adjustment range of the corresponding ambient light of each element is determined according to the type of each element to generate a preliminary brightness adjustment background image; A corresponding augmented reality element region of the augmented reality image on the preliminary brightness adjustment background image is determined; The brightness of the augmented reality element region in the preliminary brightness adjustment background image is reduced to form the brightness adjustment background image.

9. A display system characterized by, The display device comprises any one of claims 1-6.

Images