Display device, driving method thereof and display system

Abstract

The invention provides a display device, a driving method thereof and a display system. The display device comprises an image acquisition unit, an image generation unit, a light guide assembly and a dimming layer. Wherein the image generation unit comprises a logic processing unit and a projection unit; the logic processing unit is electrically connected with the image acquisition unit and is used for acquiring an environment image and generating a related augmented reality image; and the projection unit is electrically connected with the logic processing unit and is used for emitting light for enhancing the reality image. And the logic processing unit is also used for identifying each element of the environment image, generating a corresponding brightness adjustment background image according to the type of each element, and superposing the augmented reality image to generate an augmented reality superposed picture. According to the display device, the brightness of ambient light penetrating through the display device can be improved, and the fusion effect of an AR picture and a real environment is improved.

Description

Technical Field

[0001] The present application relates to the field of display technology, and in particular to a display device and an image display method thereof. Background Art

[0002] With the rapid development of semiconductor technology, the interaction between people and computers is developing rapidly. Among them, augmented reality (AR) can provide users with more dimensional information, and AR glasses are one of the important media in the field of augmented reality. However, because AR glasses transmit ambient light, there is a problem of poor contrast when AR glasses are used in bright environments, and dark AR elements such as black characters cannot be displayed, which limits the promotion of AR glasses.

[0003] At present, in order to solve the above technical problems, devices such as electrochromic lenses or twisted nematic liquid crystal light valves are usually used to adjust the brightness. In order to achieve the ideal contrast of AR images, it is necessary to significantly reduce the penetration of ambient light. However, this will cause the ambient light to be darker in appearance, and the fusion effect of AR images and real environment will be worse. Summary of the invention

[0004] The present application provides a display device and a driving method thereof and a display system, aiming to solve the problem in the prior art that the ambient light transmittance of an AR display device is low, resulting in poor fusion effect between an AR image and a real environment.

[0005] In order to solve the above technical problems, the first technical solution provided by the present application is to provide a display device. The display device includes: An image acquisition unit, used for acquiring an environment image; The image generation unit includes a logic processing unit and a projection unit; the logic processing unit is electrically connected to the image acquisition unit, and is used to acquire an environment image and generate an associated augmented reality image; the projection unit is electrically connected to the logic processing unit, and is used to emit light of the augmented reality image; The light guide assembly comprises a light guide plate, an incoupling unit and an outcoupling unit; the incoupling unit and the outcoupling unit are arranged on the same side of the light guide plate; the incoupling unit is opposite to the projection unit and is used to couple the light emitted by the projection unit into the light guide plate; the outcoupling unit is used to couple the light coupled into the light guide plate out of the light guide plate; The dimming layer is arranged on the other side of the light guide plate and is used to control the transmittance of ambient light; Among them, the logic processing unit is also used to identify each element of the environmental image, generate a corresponding brightness-adjusted background image according to the type of each element, and superimpose the augmented reality image to generate an augmented reality superimposed screen.

[0006] In some embodiments, the logic processing unit is used to extract each element of the environmental image and identify the type of each element, and determine the brightening amplitude of the corresponding ambient light according to the type of each element to generate a corresponding brightness-adjusted background image; wherein, the higher the correlation between the elements of the environmental image and the augmented reality image, the greater the brightening amplitude of the corresponding ambient light.

[0007] In some embodiments, the logic processing unit is further configured to reduce the brightness of an area of ​​the background image corresponding to the augmented reality image.

[0008] In some embodiments, the field of view angle of the image acquisition unit is greater than the field of view angle of the coupling unit; the logic processing unit is also used to crop and adjust the viewing angle of the environmental image according to the field of view angle of the coupling unit to form an environmental background image of the same size as the display screen area of ​​the augmented reality image.

[0009] In some embodiments, when the display device is in a wearing state, the image acquisition unit is located above the outcoupling unit and is coaxially arranged.

[0010] In some embodiments, a light guide component and a dimming layer constitute a display lens, and the display device includes two display lenses fixed in relative positions; Wherein, each display lens is equipped with an image acquisition unit, and the two image acquisition units are respectively arranged on the side where the two display lenses are close to each other or on the side where the two display lenses are far away from each other; or, Each display lens is configured with at least two image acquisition units, and the at least two image acquisition units are arranged around the edge of the display lens.

[0011] In some embodiments, the transmittance of light in the low transmittance area corresponding to the dimming layer and the outcoupling unit is 20%-40%, and the transmittance of light in the dimming layer gradually increases to 80%-95% in a direction away from the low transmittance area.

[0012] In order to solve the above technical problem, the second technical solution provided by the present application is: to provide a driving method of a display device. The display device is the display device provided by the above technical solution, and the driving method of the display device includes: Acquire environmental images in real time; Identify the elements of the environmental image and determine the type of each element; Generate an augmented reality image corresponding to the environment image according to the type of each element; Determine the corresponding ambient light brightening range according to the type of each element, and generate a corresponding brightness-adjusted background image; Overlaying the augmented reality image onto the brightness-adjusted background image to generate an augmented reality overlay screen; The information of the augmented reality overlay image is transmitted to the projection unit, so that the augmented reality overlay image is integrated with the environment in real time.

[0013] In some embodiments, the step of acquiring the environment image in real time further includes: The environment image is cropped and the viewing angle is adjusted to form an environment background image of the same size as the display screen area of ​​the augmented reality image; The steps of determining the corresponding ambient light brightening range according to the type of each element and generating the corresponding brightness adjusted background image include: Determine the brightening range of the ambient light corresponding to each element according to the type of each element, and generate a preliminary brightness-adjusted background image; Determining a corresponding augmented reality element region of the augmented reality image on the preliminary brightness-adjusted background image; The brightness of the augmented reality element area in the preliminary brightness-adjusted background image is reduced to form a brightness-adjusted background image.

[0014] In order to solve the above technical problem, the third technical solution provided by the present application is: to provide a display system. The display system includes the display device provided by the above technical solution.

[0015] Beneficial effects of the present application: Different from the prior art, the present application provides a display device and a driving method thereof and a display system. The display device includes an image acquisition unit, an image generation unit, a light guide component and a dimming layer. Among them, the image generation unit includes a logic processing unit and a projection unit. The logic processing unit is connected to the image acquisition unit, and is used to obtain the environmental image acquired by the image acquisition unit, and generate an augmented reality image associated with the environmental image. The projection unit is used to receive information of the augmented reality image and emit light for the augmented display image. The light guide component includes a light guide plate and a coupling unit and a coupling unit arranged on the same side of the light guide plate. The coupling unit is used to couple the light emitted by the projection unit into the light guide plate, and the coupling unit is used to couple the light coupled into the light guide plate out of the light guide plate, so as to propagate the light of the augmented reality image to the user's eyes, so as to realize the fusion of the augmented reality image and the real environment. In the embodiment of the present application, the logic processing unit is also used to identify each element of the environmental image, and generate a corresponding brightness adjustment background image according to the type of each element, that is, the screen brightness of the area corresponding to each element in the brightness adjustment background image can appropriately brighten the ambient light of the corresponding area, thereby improving the problem of dark ambient light. At the same time, the logic unit is also used to superimpose the augmented reality image on the brightness-adjusted background image to generate an augmented reality overlay screen, thereby intelligently brightening the ambient light while integrating the augmented reality image into the real environment, thereby improving the fusion effect of the augmented reality image and the real environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0017] Figure 1 is a schematic structural diagram of a display device provided in the first embodiment of the present application; Figure 2 yes Figure 1 A schematic cross-sectional structure diagram of the display device in the embodiment along the AA direction; Figure 3 is a schematic diagram of an environment image captured by an image capture unit provided by an embodiment of the present application; Figure 4 is a schematic diagram of a preprocessed environment image provided by an embodiment of the present application; Figure 5 is a schematic diagram of a brightness-adjusted background image provided by an embodiment of the present application; Figure 6 is a schematic diagram of a brightness-adjusted background image provided by another embodiment of the present application; Figure 7 is a schematic diagram of an augmented reality image provided by an embodiment of the present application; Figure 8 is a schematic diagram of an augmented reality overlay screen provided by an embodiment of the present application; Fig. 9 is a schematic structural diagram of a display device provided in a second embodiment of the present application; Fig.10 is a schematic structural diagram of a display device provided in a third embodiment of the present application; Fig.11 is a schematic structural diagram of a display device provided in a fourth embodiment of the present application; Fig.12 is a schematic diagram of the structure of a dimming layer provided in an embodiment of the present application; Fig.13 A schematic diagram of a process of driving a display device provided in one embodiment of the present application; Fig.14 is a schematic flow chart of a method for driving a display device provided in another embodiment of the present application; Fig.15 It is a schematic diagram of the structure of a display system provided in one embodiment of the present application.

[0018] Reference numerals: 100, display device; 101, display lens; 10, image acquisition unit; 20, image generation unit; 21, logic processing unit; 22, projection unit; 30, light guide component; 31, light guide plate; 32, coupling unit; 33, coupling unit; 40, dimming layer; 41, low transmittance area; P1, environmental image; P11, pre-processed environmental image; P2, augmented reality image; P3, brightness adjusted background image; P4, augmented reality overlay screen. DETAILED DESCRIPTION

[0019] The scheme of the embodiment of the present application is described in detail below in conjunction with the drawings of the specification.

[0020] In the following description, for the purpose of explanation rather than limitation, specific details such as specific system structures, interfaces, and technologies are provided to facilitate a thorough understanding of the present application.

[0021] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0022] The terms "first", "second" and "third" in this application are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Thus, the features defined as "first", "second" and "third" can explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "multiple" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined. All directional indications (such as up, down, left, right, front, back...) in the embodiments of this application are only used to explain the relative position relationship, movement, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication also changes accordingly. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally also includes steps or units that are not listed, or optionally also includes other steps or units inherent to these processes, methods, products or devices.

[0023] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0024] The present application is described in detail below with reference to the accompanying drawings and embodiments.

[0025] See also Figure 1 and Figure 2 , Figure 1 is a schematic diagram of the structure of a display device provided in the first embodiment of the present application, Figure 2 yes Figure 1 A schematic diagram of a cross-sectional structure of a display device in the embodiment along the AA direction. In the embodiment, a display device 100 is provided, and the display device 100 includes: An image acquisition unit 10, used for acquiring an environment image P1; The image generation unit 20 includes a logic processing unit 21 and a projection unit 22; the logic processing unit 21 is electrically connected to the image acquisition unit 10, and is used to obtain the environment image P1 and generate the associated augmented reality image P2; the projection unit 22 is electrically connected to the logic processing unit 21, and is used to emit light of the augmented reality image P2; The light guide assembly 30 includes a light guide plate 31, a coupling unit 32 and a coupling unit 33; the coupling unit 32 and the coupling unit 33 are arranged on the same side of the light guide plate 31; the coupling unit 32 is opposite to the projection unit 22, and is used to couple the light emitted by the projection unit 22 into the light guide plate 31; the coupling unit 33 is used to couple the light coupled into the light guide plate 31 out of the light guide plate 31; The dimming layer 40 is disposed on the other side of the light guide plate 31 and is used to control the transmittance of ambient light.

[0026] The image acquisition unit 10 is disposed on one side of the light guide assembly 30, and is used to acquire the environment image P1 in real time to provide a display environment around the user for scene analysis and object recognition, that is, for environmental element recognition, and to provide a basis for generating AR elements associated with the environmental elements. The image acquisition unit 10 may be a camera.

[0027] The logic processing unit 21 is electrically connected to the image acquisition unit 10 and the projection unit 22. The logic processing unit 21 is used to obtain the environment image P1 acquired by the image acquisition unit 10, pre-process the environment image P1, generate an augmented reality image P2 associated with the environment image P1, and transmit the information of the associated augmented reality image P2 to the projection unit 22 in the form of an electrical signal. The projection unit 22 is used to receive the electrical signal of the augmented reality image P2, and emit light according to the electrical signal, so as to conduct the augmented reality image P2 to the light guide component 30 in the form of an optical signal.

[0028] The projection unit 22 may specifically include a micro display and an optical element. The micro display may specifically be a self-luminous display such as a micro organic light emitting diode (Micro Organic Light Emitting Diode, Micro OLED) display, a micro light emitting diode (Micro Light Emitting Diode, Micro LED) display, etc. The optical element may include elements such as lenses, prisms, and waveguides, which may be used to magnify, focus, and project light.

[0029] The light guide assembly 30 includes a light guide plate 31 and a coupling unit 32 and a coupling unit 33 disposed on the light guide plate 31. The coupling unit 32 is disposed opposite to the projection unit 22, that is, the coupling unit 32 is disposed on the optical path of the outgoing light of the projection unit 22, so as to receive the light of the projection unit 22, and guide the light emitted by the projection unit 22 into the light guide plate 31. The light is transmitted to the coupling unit 33 through the light guide plate 31, and the coupling unit 33 is used to couple the light out of the light guide plate 31, so that the light is transmitted to the user's eyes, so that the user can see the virtual image (augmented reality image P2). The coupling unit 32 and the coupling unit 33 can be specifically a grating structure or a semi-transparent and semi-reflective structure, so as to couple in and couple out the light. The light guide plate 31 can be an optical waveguide plate, specifically any one of a diffraction optical waveguide substrate, a volume holographic optical waveguide substrate, an array optical waveguide substrate, and a free-form surface prism waveguide substrate, so as to efficiently transmit the image light to the user's eyes through the waveguide structure and maintain transparency to facilitate the transmission of external ambient light and transmit it to the user's eyes, so that the user can simultaneously view the real environment, thereby realizing the superposition of the augmented reality image P2 with the real environment.

[0030] The dimming layer 40 is disposed on the surface of the light guide plate 31 facing the external environment, and is used to control the transmittance of the ambient light. The dimming layer 40 can be a film device that can be used to adjust the transmittance of light, such as an electrochromic film layer, a photochromic film layer, or a twisted nematic phase liquid crystal light valve. By setting the dimming layer 40, the transmittance of the dimming layer 40 can be controlled accordingly according to the brightness of the actual environment, so that the brightness of the ambient light after passing through the light guide component 30 can meet the contrast requirements of the AR image and the ambient picture.

[0031] In the embodiment of the present application, the logic processing unit 21 is further used to identify the environment image P1 (see Figure 3 ) of each element, and generates a corresponding brightness adjustment background image P3 according to the type of each element (see Figure 5 and Figure 6 ), and superimpose the augmented reality image P2 (see Figure 7 ), generate the augmented reality overlay screen P4 (see Figure 8 ).

[0032] Specifically, after the environmental image P1 is preprocessed, the logic processing unit 21 is also used to identify each element in the preprocessed environmental image P11, such as image elements such as scenery, sky, objects, and background, and then determine the corresponding brightening amplitude according to the type of each element in the image, thereby generating a corresponding brightness-adjusted background image P3, so as to brighten the ambient light corresponding to each element in the environmental image P1 by a corresponding amplitude. For example, the environmental image P1 includes scenery elements and sky elements. Since the brightness of the ambient light of the sky element itself is greater than the brightness of the ambient light of the scenery element, the brightening amplitude of the area corresponding to the sky can be made smaller than the brightening amplitude of the area corresponding to the scenery, or the area corresponding to the sky is not brightened. That is, the brightness-adjusted background image P3 can brighten the ambient light by partition, so as to meet the contrast between the augmented reality image P2 and the surrounding real environment, and can also brighten the ambient light of key elements in the display environment accordingly, i.e., more important elements and elements associated with the augmented reality image P2. Specifically, by superimposing the augmented reality image P2 onto the brightness-adjusted background image, an augmented reality superimposed screen P4 is generated, so that the ambient light corresponding to each element in the real environment can be brightened while projecting the augmented reality screen, thereby effectively improving the integration of the augmented reality image P2 and the real environment and improving the user experience.

[0033] See also Figure 3 and Figure 4 , Figure 3 is a schematic diagram of an environment image captured by an image capture unit provided in an embodiment of the present application, Figure 4 1 is a schematic diagram of a preprocessed environment image provided by an embodiment of the present application. In some embodiments, the field of view of the image acquisition unit 10 is greater than the field of view of the coupling unit 33; the logic processing unit 21 is also used to crop and adjust the viewing angle of the environment image P1 according to the field of view of the coupling unit 33 to form an environment background image of the same size as the display screen area of ​​the augmented reality image P2.

[0034] Specifically, the field of view (FOV) of the image acquisition unit 10 is 50° to 70°, for example, 50°, 55°, 60°, 65° or 70°; the field of view of the coupling unit 33 is 20° to 45°, for example, 20°, 25°, 30°, 35°, 40° or 45°. By making the field of view of the image acquisition unit 10 larger than the field of view of the coupling unit 33, the image acquisition unit 10 has a larger field of view of the environment image P1, thereby reserving processing space for subsequent preprocessing of the environment image P1.

[0035] Specifically, the logic processing unit 21 preprocesses the environmental image P1 acquired by the image acquisition unit 10, and the preprocessing includes operations such as cropping and viewing angle adjustment. Among them, the environmental image P1 can be cropped according to the field of view of the coupling unit 33, so that the field of view of the environmental image P1 is consistent with the field of view of the coupling unit 33, and the size of the cropped environmental image P1 is the same as the size of the display screen area of ​​the augmented reality image P2. Furthermore, the logic processing unit 21 also adjusts the viewing angle of the environmental image P1 so that the viewing angle of the environmental image P1 is consistent with the viewing angle of the user, and at the same time corrects the distortion in the environmental image P1 and corrects the parallax of the environmental image P1. The viewing angle and image size of the environmental image P11 after the above preprocessing are the same as the real environment seen by the user from the display device 100.

[0036] Please refer to Figure 4 and Figure 5 , Figure 5 : is a schematic diagram of a brightness adjustment background image provided by an embodiment of the present application. In this embodiment, the logic processing unit 21 is used to extract each element of the environment image P1, identify the type of each element, and determine the corresponding ambient light brightening amplitude according to the type of each element to generate the corresponding brightness adjustment background image P3; wherein, the higher the correlation between the elements of the environment image P1 and the augmented reality image P2, the greater the corresponding ambient light brightening amplitude.

[0037] After the preprocessed environment image P11, the logic processing unit 21 performs element recognition, classifies each element in the environment image P1, and determines the brightening amplitude of the environment light corresponding to the element according to the element type. Different element types correspond to different brightening amplitudes to generate a brightness-adjusted background image P3 corresponding to each element in the environment image P1. The element type can be classified according to the element category, including background category, scene category, object category and other types; or it can be classified according to importance, including the first category, the second category and the third category, and the importance decreases in sequence, that is, the first category has the highest importance, followed by the second category, and the third category has the lowest importance; or, the element type is first classified according to the element category, including background category, scene category, object category and other types, and then the scene category and / or object category are classified according to the element importance, including the first category, the second category and the third category, and the background category is classified according to the original brightness, including the first brightness category, the second brightness category and the third brightness category, and the first brightness category.

[0038] Different types of elements correspond to a brightness increase range, that is, a brightening range. The logic processing unit 21 determines the corresponding brightness increase range according to the type of each element in the environmental image P1, thereby forming a complete brightness-adjusted background image P3. Specifically, the significance weights can be intelligently assigned according to the element type, and the brightness can be adjusted according to the weights. Among them, elements with a high degree of correlation with the augmented reality image P2 have a higher weight, and elements with a low degree of correlation have a lower weight. The higher the weight, the higher the brightening range, and the lower the weight, the lower the brightening range. For example, in this embodiment, the weight of the already bright sky element is set to a lower value to reduce its brightening range, and the weight of the meaningful scenery element that needs to be fused with the augmented reality image P2 is set to a higher value to increase its brightening range, thereby improving the brightening effect of the ambient light and improving the fusion effect of the augmented reality image P2 with the real environment.

[0039] For example, Figure 4 and Figure 5As shown, the environment image P1 includes scene elements and sky elements. The scene elements can be classified into the scene class, and the sky elements can be classified into the background class. The sky elements can be further classified into the third brightness class. Among them, the scene elements, as a more important element, have a higher correlation with the augmented reality image P2, and the brightness of the corresponding area of ​​the element is higher; while the sky element, as a background element, has a relatively high brightness and a relatively low correlation with the augmented reality image P2. If the brightness is too high, the contrast of the augmented reality image P2 will be reduced. Therefore, the brightening amplitude of the area corresponding to the element is set to be low, or no brightening is performed. The logic processing unit 21 generates a brightness-adjusted background image P3 corresponding to the background image according to the above method. In the brightness-adjusted background image P3, the brightness of the area corresponding to the scene elements is higher, and the brightness of the area corresponding to the sky elements is lower, so that the ambient light passing through the display device 100 is brightened by partition, so that the AR element area in the augmented reality image P2 maintains a good contrast, and can effectively brighten the ambient light, improve the integration of the AR element and the real environment, and enhance the user experience. Among them, the AR elements are graphics in the augmented reality image P2 that are integrated with the real environment, such as graphic elements such as indicator arrows and texts.

[0040] See also Figure 6-Figure 8 , Figure 6 is a schematic diagram of a brightness adjustment background image provided by another embodiment of the present application, Figure 7 is a schematic diagram of an augmented reality image provided by an embodiment of the present application, Figure 8 2 is a schematic diagram of an augmented reality overlay screen provided by an embodiment of the present application. In some embodiments, further, the logic processing unit 21 is further configured to reduce the brightness adjustment of the area of ​​the background image P3 corresponding to the augmented reality image P2.

[0041] Specifically, in order to further improve the contrast of AR elements in the augmented reality image P2, the logic processing unit 21 reduces the brightness of the area corresponding to the additional item display image in the brightness adjustment background image P3, so that after the additional item display image is superimposed on the corresponding area in the brightness adjustment background image P3, the display of the AR element is more obvious, that is, the contrast of the AR element is higher.

[0042] Please continue to refer to Figure 1In this embodiment, when the display device 100 is in a wearing state, the image acquisition unit 10 is located above the coupling unit 33 and is coaxially arranged. Specifically, the display device 100 may be a near-eye display device 100. When the display device 100 is in a wearing state, the image acquisition unit 10 may be arranged on the frame of the light guide assembly 30 and is located directly above the coupling unit 33, that is, the center of the image acquisition unit 10 and the center of the coupling unit 33 are located in the same straight line, and the extension direction of the straight line is the vertical direction. After such an arrangement, the focus of the environmental image P1 acquired by the image acquisition unit 10 and the focus of the picture projected by the coupling unit 33 have the same horizontal coordinate in the horizontal direction, so that the logic processing unit 21 only needs to correct the viewing angle in the vertical direction when adjusting the viewing angle of the environmental image P1, which can reduce the difficulty of image parallax correction.

[0043] See also Fig. 9 and Fig.10 , Fig. 9 is a schematic diagram of the structure of a display device provided in the second embodiment of the present application, Fig.10 1 is a schematic diagram of the structure of a display device provided in the third embodiment of the present application. In this embodiment, the display device 100 is a near-eye display device 100, wherein a light guide component 30 and a dimming layer 40 constitute a display lens 101, and the display device 100 includes two display lenses 101 fixed in relative position; each display lens 101 is configured with an image acquisition unit 10, and the two image acquisition units 10 are respectively arranged on the side where the two display lenses 101 are close to each other or on the side where they are far away from each other.

[0044] Specifically, the display device 100 includes two display lenses 101 with fixed relative positions, that is, the position of one display lens 101 is fixed relative to the position of the other display lens 101, or it can be understood that, with one display lens 101 as a reference point, the distance between the other display lens 101 and the display lens 101 is fixed, and the orientation of the other display lens 101 is also fixed relative to the display lens 101. Each display lens 101 includes a light guide component 30 and a dimming layer 40 disposed on the surface of the light guide component 30, and each display lens 101 is configured with an image acquisition unit 10.

[0045] like Fig. 9 As shown, the image acquisition unit 10 is disposed on a side of the corresponding display lens 101 close to another display lens 101; that is, the image acquisition unit 10 is disposed on a side where the two display lenses 101 are close to each other. Fig.10As shown, the image acquisition unit 10 can also be arranged on the side of the corresponding display lens 101 away from the other display lens 101, that is, at the turning point between the display lens 101 and the temple of the glasses. By arranging the image acquisition unit 10 on the side where the two display lenses 101 are close to each other or on the side away from each other, it is beneficial to the integrated design and hidden design of the image acquisition unit 10, which can reduce the volume of the display device 100 and improve the comfort and aesthetics of the display device 100.

[0046] See also Fig.11 , Fig.11 1 is a schematic diagram of the structure of a display device provided in the fourth embodiment of the present application. In this embodiment, a light guide component 30 and a dimming layer 40 constitute a display lens 101, and the display device 100 includes two display lenses 101 fixed in relative positions; each display lens 101 is configured with at least two image acquisition units 10, and at least two image acquisition units 10 are arranged around the edge of the display lens 101.

[0047] Specifically, each display lens 101 is configured with at least two image acquisition units 10, for example, two, three, four, five, or six, and multiple cameras can be equally spaced around the edge of the display lens 101. For example, each display lens 101 is configured with four image acquisition units 10, and the four image acquisition units 10 are respectively arranged on the upper side, lower side, left side, and right side of the display lens 101, and are equally spaced along the circumference of the display lens 101. This arrangement can capture environmental images P1 of multiple perspectives, and can make the logic processing unit 21 more accurate in performing parallax calibration of the environmental image P1, thereby improving the image parallax correction accuracy of the logic processing unit 21.

[0048] See also Fig.12 , Fig.12 is a schematic diagram of the structure of the dimming layer provided in an embodiment of the present application. Further, in some embodiments, the transmittance of light in the low transmittance region 41 corresponding to the dimming layer 40 and the outcoupling unit 33 is 20% to 40%, and the transmittance of light in the dimming layer 40 gradually increases to 80% to 95% in the direction away from the low transmittance region 41.

[0049] Among them, the positive projection area of ​​the coupling unit 33 on the dimming layer 40 is a low-transmittance area 41, and the transmittance of light in this area is 20%~40%, for example, it can be 20%, 25%, 30%, 35% or 40%. By reducing the transmittance of light in this area, the contrast of the augmented reality image P2 can be effectively improved. Furthermore, along the direction away from the low-transmittance area 41 on the dimming layer 40, the transmittance of light gradually increases to 80%~95%, for example, it can increase to 80%, 85%, 90% or 95%, so that the light transmittance gradually transitions from the boundary of the coupling unit 33 to the outside, thereby realizing the gradual transition and fusion of the background of the display area of ​​the augmented reality image P2 and the real environment, so as to improve the user's viewing experience.

[0050] Please refer to Figure 1-Figure 8 as well as Fig.13 , Fig.13 1 is a flow chart of a method for driving a display device provided in one embodiment of the present application. In this embodiment, a method for driving a display device 100 is provided, and the method for driving the display device 100 provided in the above embodiment to perform display work. The method for driving includes: S10: Acquire the environment image P1 in real time; S20: identifying each element of the environment image P1 and determining the type of each element; S30: generating an augmented reality image P2 corresponding to the environment image P1 according to the type of each element; S40: determining the corresponding ambient light brightening range according to the type of each element, and generating a corresponding brightness-adjusted background image P3; S50: superimposing the augmented reality image onto the brightness adjusted background image P3 to generate an augmented reality superimposed screen P4; S60: Transmitting information of the augmented reality overlay screen P4 to the projection unit 22, so that the augmented reality overlay screen P4 is integrated with the environment in real time.

[0051] In step S10, the environment can be captured in real time by the image acquisition unit 10, thereby obtaining the environment image P1. In step S20, the classification of each element and the type of element in the environment image P1 is the same or similar to that in the above embodiment, and the details can be referred to in the above embodiment, which will not be repeated here.

[0052] The brightness-adjusted background image P3 generated by steps S30 and S40 can brighten the ambient light passing through the display device 100 by partitioning, thereby improving the contrast of the AR image in the AR image (augmented reality image P2) and effectively improving the brightness of the ambient light. Through steps S50 and S60, the display device 100 can display the AR image and improve the brightness of the ambient light, thereby improving the fusion effect of the AR image and the real environment and improving the user's viewing experience.

[0053] See also Fig.14 , Fig.14 1 is a flow chart of a method for driving a display device provided in another embodiment of the present application. In this embodiment, after step S10, the method further includes: S11: cropping and adjusting the viewing angle of the environment image P1 to form an environment background image having the same size as the display screen area of ​​the augmented reality image P2; Step S40 specifically includes: S41: determining the brightening amplitude of the ambient light corresponding to each element according to the type of each element, and generating a preliminary brightness-adjusted background image P3; S42: Determine an augmented reality element region corresponding to the augmented reality image P2 on the preliminary brightness-adjusted background image P3; S43: reducing the brightness of the augmented reality element area in the preliminary brightness-adjusted background image P3 to form the brightness-adjusted background image P3.

[0054] In this embodiment, after acquiring the environment image P1, the environment image P1 is first preprocessed, and the preprocessing includes cropping and viewing angle adjustment. After cropping and viewing angle adjustment, the environment image P1 is more restored, and the image focus is consistent with the focus of the user when observing the real environment after wearing the display device 100, so that the generated AR image is more accurate when fused with the real environment. Specifically, the cropping parameters and the adjustment parameters of the viewing angle adjustment can be calculated based on the setting position of the image acquisition unit 10, the field of view angle, and the focus of the coupling unit 33, and then the environment image P1 is preprocessed based on the cropping parameters and the adjustment parameters.

[0055] In step S41 , the logic processing unit 21 first determines the required brightening amplitude of the ambient light corresponding to each element in the ambient image P1 according to the type of each element, and generates a preliminary brightness-adjusted background image P3 for brightening the ambient light by different regions.

[0056] In order to further improve the contrast of the displayed graphics of the AR image, in step S42 and step S43, the logic processing unit 21 determines the area corresponding to the AR element in the preliminary brightness adjustment background image P3, and then reduces the brightness of the area, that is, the area corresponding to the AR element in the brightness adjustment background image P3 forms a dark area, and the dark area means that the brightness is darker than the brightness of the surrounding area of ​​the area in the brightness adjustment background image P3. Therefore, in step S50, after the AR image is superimposed with the brightness adjustment background image P3, the AR element graphics are clearer, so that after the augmented reality superimposed screen P4 is projected to the user's field of view and superimposed with the real environment, the brightness of the ambient light can be increased by partition, and the contrast of the AR image can be kept high, which improves the fusion effect of the AR element and the real environment and makes the user experience better.

[0057] See also Fig.15 , Fig.15 1 is a schematic diagram of the structure of a display system provided in an embodiment of the present application. In this embodiment, a display system is provided, and the display system includes a display device 100. The structure and function of the display device 100 are the same or similar to the structure and function of the display device 100 provided in the above embodiment, and the same technical effect can be achieved. For details, please refer to the detailed description above, and no further description is given here. The driving method of the display device 100 is the same or similar to the driving method of the display device 100 provided in the above embodiment, and the details can be referred to the above description, and no further description is given here.

[0058] The display system may also include other sensors and audio-visual units and other devices to achieve more functions and be suitable for more scenarios.

[0059] The above are only implementation methods of the present application, and are not intended to limit the patent protection scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly used in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A display device, comprising: An image acquisition unit, used for acquiring an environment image; An image generation unit, including a logic processing unit and a projection unit; The logic processing unit is electrically connected to the image acquisition unit, and is used to acquire the environment image and generate an associated augmented reality image; the projection unit is electrically connected to the logic processing unit, and is used to emit light of the augmented reality image; A light guide assembly, comprising a light guide plate, an incoupling unit and an outcoupling unit; the incoupling unit and the coupling-in unit are arranged on the same side of the light guide plate; the coupling-in unit is opposite to the projection unit and is used to couple the light emitted by the projection unit into the light guide plate; the outcoupling unit is used to couple the light coupled into the light guide plate out of the light guide plate; A dimming layer, disposed on the other side of the light guide plate, for controlling the transmittance of ambient light; It is characterized in that the logic processing unit is also used to identify each element of the environmental image, generate a corresponding brightness-adjusted background image according to the type of each element, and superimpose the augmented reality image to generate an augmented reality superimposed screen.

2. The display device according to claim 1, characterized in that The logic processing unit is used to extract each element of the environmental image and identify the type of each element, and determine the brightening amplitude of the corresponding ambient light according to the type of each element to generate the corresponding brightness-adjusted background image; wherein, the higher the correlation between the elements of the environmental image and the augmented reality image, the greater the brightening amplitude of the corresponding ambient light.

3. The display device according to claim 1, characterized in that The logic processing unit is further used to reduce the brightening amplitude of the area in the brightness adjustment background image corresponding to the augmented reality image.

4. The display device according to claim 1, characterized in that The field of view angle of the image acquisition unit is greater than the field of view angle of the coupling unit; the logic processing unit is also used to crop and adjust the viewing angle of the environmental image according to the field of view angle of the coupling unit to form an environmental background image of the same size as the display screen area of ​​the augmented reality image.

5. The display device according to any one of claims 1 to 4, characterized in that: When the display device is in a wearing state, the image acquisition unit is located above the coupling unit and is coaxially arranged.

6. The display device according to any one of claims 1 to 4, characterized in that: One of the light guide components and one of the dimming layer constitutes a display lens, and the display device comprises two display lenses fixed in relative positions; Wherein, each of the display lenses is equipped with one image acquisition unit, and the two image acquisition units are respectively arranged on the side where the two display lenses are close to each other or on the side where the two display lenses are far away from each other; or, Each of the display lenses is configured with at least two of the image acquisition units, and at least two of the image acquisition units are arranged around the edge of the display lens.

7. The display device according to any one of claims 1 to 4, characterized in that: The transmittance of light in the low transmittance area corresponding to the dimming layer and the outcoupling unit is 20%-40%, and the transmittance of light in the dimming layer gradually increases to 80%-95% in a direction away from the low transmittance area.

8. A method for driving a display device, wherein the display device is the display device according to any one of claims 1 to 7, characterized in that: The driving method comprises: Acquire environmental images in real time; Identify each element of the environment image and determine the type of each element; Generate an augmented reality image corresponding to the environment image according to the type of each element; Determine the corresponding ambient light brightening range according to the type of each element, and generate a corresponding brightness-adjusted background image; Overlaying the augmented reality image onto the brightness-adjusted background image to generate an augmented reality overlay screen; The information of the augmented reality overlay image is transmitted to the projection unit, so that the augmented reality overlay image is integrated with the environment in real time.

9. The driving method according to claim 8, characterized in that: The step of acquiring the environment image in real time also includes: The environment image is cropped and the viewing angle is adjusted to form an environment background image having the same size as the display screen area of ​​the augmented reality image; The step of determining the corresponding ambient light brightening amplitude according to the type of each element and generating the corresponding brightness adjusted background image comprises: Determine the brightening range of the ambient light corresponding to each element according to the type of each element, and generate a preliminary brightness-adjusted background image; Determine an augmented reality element area corresponding to the augmented reality image on the preliminary brightness-adjusted background image; The brightness of the augmented reality element area in the preliminary brightness-adjusted background image is reduced to form the brightness-adjusted background image.

10. A display system, characterized in that: The invention comprises a display device as claimed in any one of claims 1 to 7.

Images