AR assembly, camera shooting method and AR glasses

Abstract

The invention discloses an AR assembly, a camera shooting method and AR glasses, and relates to the technical field of AR, and the AR assembly comprises a light source, a waveguide assembly and a photosensitive element. The light source is used for generating first image light. The waveguide assembly is provided with a coupling-in area and is used for receiving the first image light and splitting the first image light into second image light incident to the waveguide assembly and reflected third image light at the position of the coupling-in area; the photosensitive element is used for receiving the third image light for imaging and receiving the ambient light passing through the coupling-in area of the waveguide assembly for imaging. According to the technical scheme provided by the invention, the effect of the combined image generated by the AR component can be improved.

Description

Technical Field

[0001] This invention relates to the field of AR technology, and in particular to an AR component, a camera method, and AR glasses. Background Art

[0002] AR devices display images while allowing users to observe their environment, creating a natural need to simultaneously capture images of the environment and map the AR device's displayed image onto that environment. This would essentially record the overall scene observed by the user. However, AR devices are often small and lack sufficient computing power. Directly mapping the original digital image projected by the AR device onto the captured external image using algorithms to obtain a composite image often yields poor results, and the resulting image may differ from what the user actually sees. Summary of the Invention

[0003] The main objective of this invention is to provide an AR component, a camera method, and AR glasses, which aim to improve the effect of combined images generated by the AR component.

[0004] To achieve the above objectives, the present invention proposes an AR component comprising a light source, a waveguide assembly, and a photosensitive element. The light source generates a first image light; the waveguide assembly has a coupling region; the waveguide assembly receives the first image light and splits it at the coupling region into a second image light incident on the waveguide assembly and a reflected third image light; the photosensitive element receives the third image light for imaging and receives ambient light passing through the coupling region of the waveguide assembly for imaging.

[0005] In some embodiments, the waveguide assembly includes an optical waveguide and an optical modulation element, and the coupling region is disposed on the optical waveguide; the optical modulation element has a first state and a second state; When the light modulation element is in the first state, it blocks ambient light from passing through the coupling area and / or forms the third image light through reflection; When the optical modulation element is in the second state, ambient light is allowed to pass through the coupling area.

[0006] In some embodiments, the optical modulation element includes a first modulation element; the optical waveguide has a coupling direction, and the second image light exits the optical waveguide in the coupling direction; the first modulation element is disposed on the side of the optical waveguide opposite to the coupling direction; When the optical modulation element is in the first state, the first modulation element is used to block ambient light to prevent ambient light from passing through the coupling area; when the optical modulation element is in the second state, the first modulation element is used to transmit ambient light to allow ambient light to pass through the coupling area.

[0007] In some embodiments, the optical waveguide has a transmission region for allowing ambient light to pass through the optical waveguide along the coupling direction; The orthogonal projection of the first modulation element onto the optical waveguide covers the transmission region and the coupling region.

[0008] In some embodiments, the optical modulation element includes a second modulation element; the second modulation element is disposed in the coupling region; When the light modulation element is in the first state, the second modulation element is used to reflect the first image light to form the third image light; when the light modulation element is in the second state, the second modulation element is used to transmit ambient light to allow ambient light to pass through the coupling area.

[0009] In some embodiments, the optical waveguide has an output direction and a transmission region, wherein the second image light exits the optical waveguide in the output direction, and the transmission region is used to allow ambient light to pass through the optical waveguide along the output direction; The orthographic projection of the second modulation element onto the optical waveguide is adjacent to or spaced from the transmission region and covers the coupling region.

[0010] In some embodiments, the light modulation element includes an electrochromic element.

[0011] In some embodiments, the AR component further includes a beam splitter and a projection lens assembly; The first image light is incident on the beam splitter from the light source; the first image light exits the beam splitter and then enters the projection lens group; the first image light exits the projection lens group and then enters the coupling region directly to form the third image light. The third image light propagates from the coupling region to the beam splitter in the opposite direction to the optical path of the first image light; the third image light is imaged onto the photosensitive element after exiting the beam splitter.

[0012] The present invention also proposes a camera method, which is applied to the aforementioned AR component; the camera method includes: The light modulation element is placed in the first state so that the third image light is imaged on the photosensitive element; The light modulation element is placed in the second state so that ambient light is imaged on the photosensitive element; or The light modulation element is placed in the second state so that ambient light is imaged on the photosensitive element; The light modulation element is placed in the first state so that the third image light is imaged on the photosensitive element.

[0013] The present invention also proposes an AR glasses, which includes the AR components described above or uses the imaging method described above for imaging.

[0014] In the technical solution of this invention, the same photosensitive element serves two functions: receiving the third image light and receiving ambient light from the coupling area of ​​the waveguide component. The third image light is a beam split from the first image light, so receiving the third image light can generate the image projected by the AR component; the ambient light generates the external image. Therefore, the same photosensitive element simultaneously displays both the projected image and the external image. Consequently, the digital images of the projected image and the external image acquired by the photosensitive element have a simple mapping relationship between their pixels. Thus, even with limited computing power, the digital image of the projected image can still be accurately mapped onto the digital image of the external image to obtain a better combined image. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the structures shown in these drawings without creative effort.

[0016] Figure 1 A schematic diagram of the structure of the first embodiment of the AR component provided by the present invention; Figure 2 A schematic diagram of the optical path of the first embodiment of the AR component provided by the present invention; Figure 3 A schematic diagram of the structure of the second embodiment of the AR component provided by the present invention; Figure 4 This is a schematic diagram of the first optical path of a second embodiment of the AR component provided by the present invention; Figure 5 This is a schematic diagram of the second optical path of the second embodiment of the AR component provided by the present invention; Figure 6 A schematic diagram of the structure of the third embodiment of the AR component provided by the present invention; Figure 7 This is a schematic diagram of the first optical path of the third embodiment of the AR component provided by the present invention; Figure 8 This is a schematic diagram of the second optical path of the third embodiment of the AR component provided by the present invention; Figure 9A schematic diagram of one embodiment of the projection component, beam splitter, photosensitive element, and light source of the AR component provided by the present invention; Figure 10 A flowchart of the first embodiment of the imaging method provided by the present invention; Figure 11 A flowchart of the second embodiment of the imaging method provided by the present invention; Figure 12 A flowchart of the third embodiment of the imaging method provided by the present invention.

[0017] Explanation of icon numbers: AR component 10; Light source 11; Waveguide assembly 12; optical waveguide 121; transmission region 121a; optical modulation element 122; first modulation element 1221; second modulation element 1222; coupling region 12a; Photosensitive element 13; Spectrometer 14; Projection lens group 15; aperture 151; lens 152; First image light a; Second image light b; Third image light c; Ambient light d; Human eye 20.

[0018] The realization of the objective, functional characteristics and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] It should be noted that if the embodiments of the present invention involve directional indicators (such as up, down, left, right, front, back, etc.), the directional indicators are only used to explain the relative positional relationship and movement of the components in a specific posture. If the specific posture changes, the directional indicators will also change accordingly.

[0021] Furthermore, if the embodiments of this invention involve descriptions such as "first" or "second," these descriptions are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined with "first" or "second" may explicitly or implicitly include at least one of those features. Additionally, the use of "and / or" or "and / or" throughout the text includes three parallel solutions; for example, "A and / or B" includes solution A, solution B, or a solution where both A and B are satisfied simultaneously. Furthermore, the technical solutions of various embodiments can be combined, but this must be based on the ability of those skilled in the art to implement them. When the combination of technical solutions is contradictory or impossible to implement, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed by this invention.

[0022] This invention proposes an AR component.

[0023] Please refer to Figure 1 and Figure 2 The AR component 10 proposed in this invention includes a light source 11, a waveguide component 12, and a photosensitive element 13. The light source 11 is used to generate a first image light a; the waveguide component 12 is provided with a coupling region 12a; the waveguide component 12 is used to receive the first image light a and split the first image light a at the position of the coupling region 12a into a second image light b incident on the waveguide component 12 and a reflected third image light c; the photosensitive element 13 is used to receive the third image light c for imaging and to receive ambient light d passing through the coupling region 12a of the waveguide component 12 for imaging.

[0024] The light source 11 is the light source 11 for projecting virtual images into the AR component 10. It can be a light source of type LBS (Laser Beam Scanning), Micro LED (Micro Light Emitting Diode), LCD (Liquid Crystal Display), etc. The light source 11 can constitute the optomechanical system of the AR component 10 on its own, or it can be combined with the imaging lens group in the following embodiments to form the optomechanical system of the AR component 10.

[0025] Light source 11 can be understood as a component that can form a complete projected image, and should not be considered as merely a light-providing element. For example, when light source 11 is a DLP (Digital Light Processing) type light source, light source 11 may include a lamp that provides backlight (such as a mercury lamp, a light-emitting diode, or a laser diode), a color wheel for filtering light, and a DMD (Digital Micromirror Device) for modulating light, and should not be considered as simply including a lamp.

[0026] In another example, when the light source 11 is an LCD type light source, the light source 11 may include a backlight assembly, a liquid crystal element, a polarizer, and a TFT (Thin-Film Transistor), and cannot be considered as the light source 11 only including a backlight assembly.

[0027] The first image light 'a' emitted by light source 11 contains information about the projected image required by AR component 10. For some types of light sources 11 (e.g., LBS-type light sources 11), which have appropriate VID (Virtual Image Distance), after propagation through waveguide component 12, the light is coupled out to human eye 20, allowing human eye 20 to perceive the image projected by AR component 10. For other types of light sources 11 (e.g., Micro LED or LCD-type light sources 11), they can be paired with the projection lens group 15 described below to obtain appropriate VID.

[0028] Waveguide assembly 12 is a component that deflects the image light projected by light source 11 and transmits ambient light d. In some embodiments, waveguide assembly 12 may only include an optical waveguide, while in other embodiments, waveguide assembly 12 may further include devices that can modulate the coupling and transmission of the optical waveguide (see the optical modulation element in the embodiments below for details).

[0029] Optical waveguides have a higher refractive index than the surrounding medium, allowing image light coupled into them to propagate within the waveguide. However, because the waveguide is essentially transparent, ambient light can also pass through. Please refer to [reference needed]. Figure 2 Because the waveguide component 12 deflects the light emitted from the light source 11, the light source 11 does not need to be positioned in front of the human eye 20, i.e., it does not obstruct the center of the human eye 20's field of view; at the same time, the optical waveguide is basically transparent, so ambient light d can directly pass through the waveguide component 12 and be received by the human eye 20. This allows the user to simultaneously observe the ambient scene and the projected image.

[0030] Waveguide component 12 has a coupling region 12a, please refer to Figure 2 and Figure 3 The coupling region 12a of the waveguide assembly 12 is the area where light is coupled into the optical waveguide 121. The surface of the optical waveguide 121 in the coupling region 12a can be used to receive the outgoing light from the light source 11, i.e., the first image light a. If it is necessary to change the propagation direction of the light entering the optical waveguide 121, a coupling grating can also be provided in the coupling region 12a so that the light entering the optical waveguide 121 can propagate in the optical waveguide 121 by total internal reflection.

[0031] The coupling region 12a of the waveguide assembly 12 is the area where light is coupled into the optical waveguide 121. Therefore, the first image light a will meet the surface of the optical waveguide 121 in the coupling region 12a. The refractive index of the optical waveguide 121 is different from that of the surrounding medium (often air), so partial reflection and partial transmission occur. The transmitted light enters the optical waveguide 121 and becomes the second image light b; the reflected light is reflected according to the law of reflection and becomes the third image light c. Of course, it is not necessary to reflect the light on the surface of the optical waveguide 121 to form the third image light c. The third image light c can also be formed by reflection on the optical modulation element in the following embodiment.

[0032] Since the second image light b and the third image light c are derived from the first image light a, they both carry the image information in the first image light a. If the second image light b and the third image light c can be imaged, a projected image of the AR component 10 can be formed.

[0033] The photosensitive element 13 is an element that receives light information, such as a CCD (Charge-coupled Device) or CMOS (Complementary Metal-Oxide Semiconductor) type photosensitive element. Typically, imaging on the photosensitive element 13 requires an imaging lens group. However, referring to the embodiments below, in some embodiments, in conjunction with a beam splitter and a projection lens group, the projection lens group can serve as the imaging lens group for the photosensitive element. In some embodiments, if the light source 11 is an LBS type light source, an imaging lens group may not be necessary; simply placing the photosensitive element 13 at a position conjugate to the virtual image formed by the LBS is sufficient to obtain a projected image.

[0034] In addition to receiving the third image light c to obtain image information of the projected image, the photosensitive element 13 can also receive ambient light d to form an image, thus obtaining image information of the real environment. Please refer to [reference needed]. Figure 2 In some embodiments, the ambient light d received by the photosensitive element 13 can be transmitted through the waveguide assembly 12, so that the environmental image obtained by the photosensitive element 13 has no difference in visual axis from the environmental image obtained by the human eye 20, and can better reflect the scene seen by the human eye 20; in other embodiments, the ambient light d received by the photosensitive element 13 can also pass directly through the waveguide assembly 12 (not shown in the figure) to avoid image distortion and improve imaging effect.

[0035] The digital information of the image to be projected by AR component 10 can be an image pre-stored by AR component 10, or an image generated by the controller of AR component 10 based on environmental information, user instructions, and / or pre-stored algorithms. However, in either case, the digital information of the image has already been generated in the controller before the image is projected, or before the first image light a is emitted. This digital information is the original digital image of the projected image.

[0036] If the photosensitive element 13 is only used to acquire ambient light d to obtain an environmental image, and then the original digital image is directly mapped onto the environmental image, the mapping relationship will change depending on factors such as the object distance and depth of field when the environmental image is captured. Complex algorithms are needed to correct the mapping relationship. Otherwise, the final combined image will be inconsistent with the image actually seen by the user. For example, the user may see that the numbers in the projected image are originally in the center of the field of vision, but in the combined image, the numbers are located near the edge.

[0037] The AR component 10 proposed in this application allows the photosensitive element 13, which is used to acquire ambient light d, to simultaneously acquire a third image light c. Since the third image light c is split from the first image light a, it has the image information in the first image light a. In this way, the ambient image and the projected image can be imaged on one photosensitive element 13. The digital image of the ambient image and the digital image of the projected image acquired by the photosensitive element 13 have a simpler correspondence between the pixels of the two images. Therefore, only a simple algorithm is needed to accurately combine the two images.

[0038] It is evident that the technical solution of this application reduces the risk of misalignment in the combined image, resulting in a better effect of the combined image.

[0039] Please refer to Figure 3 and Figure 6 In some embodiments, the waveguide assembly 12 includes an optical waveguide 121 and an optical modulation element 122, with a coupling region 12a disposed on the optical waveguide 121; the optical modulation element 122 has a first state and a second state. Please refer to Figure 4 and Figure 7 When the light modulation element 122 is in the first state, it blocks ambient light d from passing through the coupling region 12a and / or from forming the third image light c through reflection; Please refer to Figure 5 and Figure 8 When the optical modulation element 122 is in the second state, ambient light d is allowed to pass through the coupling region 12a.

[0040] The optical modulation element 122 is a component whose transmittance and / or reflectance can be controlled. Although for a typical homogeneous medium, transmittance and reflectance are often additive and not independent physical quantities, the optical modulation element 122 can absorb light in some embodiments.

[0041] In one example of an optical modulation element 122, the optical modulation element 122 can be a combination of a liquid crystal film and a polarizer. Polarizers with mutually perpendicular transmission directions are disposed on both sides of the liquid crystal film. In this way, when the optical rotation angle of the liquid crystal film is 0°, the optical modulation element 122 can have a large absorption rate of light (basically opaque) and at this time, it also reflects light almost nothing. When the optical rotation angle of the liquid crystal film is 90°, the absorption rate of light can be basically 0 (basically transparent).

[0042] In another example of a light modulation element 122, the light modulation element 122 can be an electrochromic element. Some types of electrochromic elements can form an effect similar to the liquid crystal type light modulation element 122 described above, that is, they can switch between a state of strong light absorption and a state of basic light transmission; other types of electrochromic elements can switch between a state of high reflectivity and a state of high transmittance.

[0043] In some embodiments, the optical modulation element 122 may also be a combination of an electrochromic element and a photonic crystal to improve the speed of state switching.

[0044] In the example of the optical modulation element 122 mentioned above, it can be seen that in some embodiments, the optical modulation element 122 can have a high absorption rate in the first state. Thus, when the optical modulation element 122 is placed in the optical path between the ambient light d from the outside to the optical waveguide 121, it can absorb the ambient light d, thereby blocking the ambient light d from passing through the coupling region 12a.

[0045] In one example, the optical modulation element 122 is disposed on the side of the optical waveguide 121 that receives external light, and its orthogonal projection onto the optical waveguide 121 at least partially covers the optical waveguide 121. This absorbs at least part of the external light entering the optical waveguide 121. Since external light entering the optical waveguide 121 from various points may pass through the coupling region 12a during transmission through the optical waveguide 121, this can block ambient light d from passing through the coupling region 12a.

[0046] In the example of the optical modulation element 122 mentioned above, it can be seen that in some other embodiments, the optical modulation element 122 can have a high reflectivity in the first state. In this way, as long as the optical modulation element 122 is placed in the optical path of the first image light a or the second image light b, the first image light a or the second image light b can be reflected to form the third image light c.

[0047] In one example, the optical modulation element 122 is positioned at the location where the second image light b is coupled out of the optical waveguide 121, so that the second image light b can be reflected back to the optical waveguide 121 and emitted from the coupling region 12a to form the third image light c.

[0048] In the example of the optical modulation element 122 mentioned above, it can be seen that in some embodiments, the optical modulation element 122 can have high transmittance in the second state. At this time, the optical modulation element 122 does not affect the operation of other elements of the AR component 10, so it can also transmit external light, so that at least part of the external light enters the optical waveguide 121. The external light entering the optical waveguide 121 can be directly transmitted, or it can be transmitted in the optical waveguide 121 and then coupled out from the coupling region 12a, that is, the ambient light d is allowed to pass through the coupling region 12a.

[0049] In the embodiment where the light modulation element 122 is provided, when the light modulation element 122 is in the first state, in an embodiment that obstructs ambient light d from passing through the coupling region 12a, the light intensity of the ambient light d on the photosensitive element 13 can be reduced; in an embodiment that forms the third image light c through reflection, the light intensity of the third image light c on the photosensitive element 13 can be increased. This allows the brightness of the projected image on the photosensitive element 13 to be greater than the brightness of the ambient image, resulting in a clearer projected image and making it more beneficial for post-processing.

[0050] In the embodiment where the light modulation element 122 is provided, when in the second state, ambient light d can be allowed to enter the optical waveguide 121 normally, thereby forming a clear image on the photosensitive element 13.

[0051] Please refer to the following implementation method. The imaging of the projected image by the photosensitive element 13 and the imaging of the external scene by the photosensitive element 13 can be performed alternately. Therefore, the projected image can be imaged in the first state and the environmental scene can be imaged in the second state, thereby improving the final imaging effect. However, since only the light intensity is modulated, the mapping relationship between the two images is not changed, thereby improving the effect of the combined image.

[0052] It should be noted that the photosensitive element 13 can also image the ambient image and the projected image simultaneously, because the transmittance (or reflectance) of the light modulation element 122 can, in principle, be any value between 0 and 1. This can appropriately reduce the light intensity of the ambient light or increase the light intensity of the third image light c, so that the brightness of the two images is close, and thus they can be imaged simultaneously.

[0053] Furthermore, appropriately adjusting the incident ambient light (d) facilitates the recognition of the relationship between the user's gestures and the projected image. The ambient image can also contain images of the user's gestures. Since both the ambient image and the projected image can be clearly imaged simultaneously on the photosensitive element 13, the relationship between the gestures and the projected image can be easily determined. This facilitates the system's recognition of the user's gestures, allowing the user to operate the AR device more accurately through gestures.

[0054] Please refer to Figure 3 In some embodiments, the optical modulation element 122 includes a first modulation element 1221; the optical waveguide 121 has a coupling direction, and the second image light b exits the optical waveguide 121 in the coupling direction; the first modulation element 1221 is disposed on the side of the optical waveguide 121 opposite to the coupling direction. When the optical modulation element 122 is in the first state, the first modulation element 1221 is used to block ambient light d to prevent ambient light d from passing through the coupling region 12a; when the optical modulation element 122 is in the second state, the first modulation element 1221 is used to transmit ambient light d to allow ambient light d to pass through the coupling region 12a.

[0055] The coupling direction of the optical waveguide 121, i.e., the direction in which the second image light b is coupled out of the optical waveguide 121 after propagation within it, is typically the direction in which the optical waveguide 121 faces the human eye 20 during use. The first modulation element 1221 is disposed on the side of the optical waveguide 121 opposite to the coupling direction, i.e., on the side of the optical waveguide 121 that receives ambient light. In this way, the optical modulation element 122 can absorb ambient light from the source, improving the efficiency of blocking ambient light from entering the optical waveguide 121.

[0056] Please refer to Figure 3 In some embodiments, the optical waveguide 121 has a transmission region 121a, which is used to allow ambient light d to pass through the optical waveguide 121 in the coupling direction. The orthographic projection of the first modulation element 1221 onto the optical waveguide 121 covers the transmission region 121a and the coupling region 12a.

[0057] The transmission zone 121a is the user's observation area, meaning the user can directly see the surrounding environment through the transmission zone. Often, the transmission zone also overlaps at least partially with the coupling zone of the optical waveguide 121, where the coupling zone is the area where the second image light b is coupled out. In order to change the propagation direction of the second image light b in the optical waveguide 121 and escape the total internal reflection state, the coupling zone may also be provided with structures such as coupling gratings to ensure that the second image light b is normally coupled out of the optical waveguide 121.

[0058] It can be seen that the direction of the ambient light d passing through the transmission region 121a is consistent with the direction of the second image light b coupled out of the waveguide 121, because both the ambient light d and the second image light b need to be imaged on the human eye 20.

[0059] The first modulation element 1221 covers the transmission region 121a and the coupling region 12a on the orthographic projection of the optical waveguide 121, thus fully covering the ambient light d from the optical waveguide 121 and improving the imaging effect of the third image light c. The reason for covering the coupling region 12a is that the optical waveguide 121 is basically transparent, and some ambient light d can also pass through the coupling region 12a.

[0060] Furthermore, since the first modulation element 1221 can adjust the transmittance of ambient light d, when the AR component 10 is applied to AR glasses, it can also create the effect of photochromic glasses.

[0061] Please refer to Figure 6 In some embodiments, the optical modulation element 122 includes a second modulation element 1222; the second modulation element 1222 is disposed in the coupling region 12a; When the light modulation element 122 is in the first state, the second modulation element 1222 is used to reflect the first image light a to form the third image light c; when the light modulation element 122 is in the second state, the second modulation element 1222 is used to transmit ambient light d to allow ambient light d to pass through the coupling area 12a.

[0062] The second modulation element 1222 is disposed in the coupling region 12a, meaning that the orthographic projection of the second modulation element 1222 onto the optical waveguide 121 is at least partially within the coupling region 12a. Since the first image light a is also coupled into the optical waveguide 121 in the coupling region 12a, the second modulation element 1222 can directly reflect the first image light a to form the third image light c.

[0063] Since the first image light a is not coupled into the optical waveguide 121, the transmission loss is small, which can greatly increase the light intensity of the third image light c, making the projected image on the photosensitive element 13 clearer or the exposure time shorter.

[0064] In addition, please refer to Figure 7 In fact, when the second modulation element 1222 has high reflectivity, it can also reflect the ambient light d coupled out from the coupling region 12a, further improving the imaging effect of the third image light c.

[0065] Please refer to Figure 6 In some embodiments, the optical waveguide 121 has a coupling direction and a transmission region 121a, the second image light b exits the optical waveguide 121 in the coupling direction, and the transmission region 121a is used to allow ambient light d to pass through the optical waveguide 121 in the coupling direction. The orthographic projection of the second modulation element 1222 onto the optical waveguide 121 is adjacent to or spaced from the transmission region 121a, and covers the coupling region 12a.

[0066] The second modulation element 1222 covers the coupling region 12a, thereby allowing all the first image light a to be reflected, increasing the light intensity of the third image light c. Simultaneously, the second modulation element 1222 is adjacent to or spaced apart from the transmission region 121a, ensuring that it does not obstruct the user's field of vision. Even when the second modulation element 1222 is in a high reflectivity state, the user can clearly observe the external scene.

[0067] The transmission region 121a may sometimes not have a defined range. In such cases, the second modulation element 1222 being adjacent to or spaced from the transmission region 121a is equivalent to being adjacent to or spaced from the coupling region.

[0068] It should be noted that in some embodiments, the first modulation element 1221 and the second modulation element 1222 can be configured simultaneously. In this way, the first modulation element 1221 and the second modulation element 1222 can collaboratively reduce the input of ambient light d. At this time, neither the first modulation element 1221 nor the second modulation element 1222 needs to switch to a completely light-blocking or completely reflective state to achieve sufficient reduction of external light so that the third image light c can be clearly imaged. Because the state transition is smaller, the first modulation element 1221 and the second modulation element 1222 can have a higher switching speed between the first and second states, thereby quickly completing the image capture.

[0069] Increasing the switching speed of the optical modulation element 122 can even enable continuous photography to create video recordings.

[0070] In some embodiments, the second modulation element 1222 may be a shutter, and a reflective structure (coated reflective film, attached reflective film, or reflector, etc.) may be provided on the side of the shutter curtain or blades facing the first image light a, so that when the shutter is open, ambient light d can be allowed to pass through; when the shutter is closed, the first image light a can be reflected to form the second image light b.

[0071] In some implementations, the light modulation element includes an electrochromic element.

[0072] Some electrochromic elements can have microsecond-level switching speeds and are simple to control. Often, only two conductive films need to be placed on the two sides of the electrochromic material to control the electrochromic material. Therefore, electrochromic elements can have sufficient switching speed between the first and second states and also have a simpler and easier-to-implement structure.

[0073] It should be noted that the optical modulation element includes an electrochromic element, such that both the first and second modulation elements can be electrochromic elements. Please refer to [reference needed]. Figure 3 and Figure 6 ,exist Figure 3 and Figure 6 In the embodiment shown, the first modulation element 1221 and the second modulation element 1222 are electrochromic elements.

[0074] Please refer to Figure 1 and Figure 9 In some embodiments, the AR component 10 further includes a beam splitter 14 and a projection lens group 15; The first image light a is incident on the beam splitter 14 from the light source 11; the first image light a is emitted from the beam splitter 14 and then incident on the projection lens group 15; the first image light a is emitted from the projection lens group 15 and then incident on the coupling region 12a to form the third image light c. The third image light c propagates from the self-coupled region 12a to the beam splitter 14 in the opposite direction to the optical path of the first image light a; after exiting the beam splitter 14, the third image light c is imaged onto the photosensitive element 13.

[0075] The beam splitter 14 is an element that can split an incident light beam into at least two outgoing beams. For example, it can be made of a glass sheet with a corresponding functional film. This functional film can be a thin film with a one-dimensional photonic crystal structure, thereby having specific transmittance and reflectance for a specific wavelength band (for AR component 10, it can be the visible light band). For example, the transmittance can be 40% and the reflectance can be 60%, or both the transmittance and reflectance can be 50%. The functional film can also be a metal coating. The transmittance and reflectance can also be changed by changing parameters such as the duty cycle of the coating on the glass surface or the film thickness.

[0076] The beam splitter 14 can also be a PBS (Polarizing Beam Splitter) type beam splitter. The PBS can be made of birefringent crystal or a beam splitter that uses alternating refractive index distribution to split the beam using Brewster's angle.

[0077] The projection lens group 15 is a lens group used to shape the first image light a. Shaping can involve magnifying, reducing, or appropriately deforming the image information carried by the first image light a, or it can be used to change the VID of the AR component 10. For an example, please refer to... Figure 9 The projection lens assembly 15 may include multiple lenses 152 and an aperture 151, wherein the lenses 152 may be used to shape the first image light a and / or eliminate aberrations of the AR component 10; the aperture 151 may be used to control the range through which the beam of the projection lens assembly 15 passes. In yet another example, the projection lens assembly 15 may have an exit pupil diameter of 3.3 mm, an F-number of 2.7, a diagonal FOV of 30°, and an overall length of 10 mm.

[0078] The first image light a can be incident from the light source 11 onto the beam splitter 14, and then be split on the beam splitter 14 before being emitted. One of the first image lights a split by the beam splitter 14 can be sent into the optical waveguide 121 for projection, while the other beams can be discarded or used for other purposes.

[0079] The first image light a is emitted from the beam splitter 14, i.e., incident on the projection lens group 15. After being shaped by the projection lens group 15, it is coupled into the coupling region 12a of the forward incident waveguide assembly 12. Forward incident means that the reflected light after incident can propagate in the opposite direction of the incident light. In one example, the forward incident coupling region 12a refers to the surface of the perpendicular incident waveguide 121. In another example, the forward incident coupling region 12a refers to the surface of the perpendicular incident second modulation element 1222.

[0080] As can be seen, after the normal incident coupling region 12a, the first image light a is reflected to form the third image light c. The third image light c propagates in the opposite direction to the propagation direction of the first image light a, and therefore will naturally pass through the projection lens group 15. In this way, the third image light c is equivalent to sharing a set of lenses 152 with the first image light a (in some embodiments, a reflector may also be included, not just lens 152). Since the first image light a only passes through the beam splitter 14 after exiting the light source 11 before entering the projection lens group 15, and the beam splitter 14 generally does not have a shaping effect on light, the first image light a and the third image light c will obtain "opposite" shaping effects.

[0081] After the third image light c passes through the projection lens group 15, it will naturally be incident on the beam splitter 14. At this time, the beam splitter 14 can split the third image light c into beams, and at least one beam of the third image light c will not be directed towards the light source 11. The photosensitive element 13 can be set at a position that can receive this beam of light, so as not to conflict with the position of the light source 11.

[0082] Similarly, since the beam splitter 14 does not typically shape the beam, and the third image light c is subjected to a shaping effect "opposite" to that of the first image light a, the photosensitive element 13 can be positioned at a position conjugate to the image formed by the light source 11, thereby obtaining the image information of the projected image.

[0083] Furthermore, the ambient light d propagating from the waveguide assembly 12 will pass through the coupling region 12a, and thus can also be imaged onto the photosensitive element 13 by the projection lens group 15, and the shaping of the ambient light d is the same as the shaping of the third image light c.

[0084] As can be seen, in the above embodiment, the first image light a, the third image light c, and the ambient light d share a set of lenses 152. This makes the AR component 10 structure more compact, and makes the correspondence between the pixels of the image formed by the ambient light d and the image formed by the third image light c clearer and simpler, thereby helping to further improve the effect of the combined image.

[0085] It should be noted that when the beam-splitting element 14 is a beam splitter (i.e., a lens with the aforementioned functional film on its surface), the first image light a used after exiting the beam splitter can be transmitted light, i.e., as shown in the figure. Figure 9 In the optical path shown, the transmitted first image light a is utilized; meanwhile, the reflected third image light c can be used for imaging. However, in other examples, such as in... Figure 9 In this process, by swapping the positions of the light source 11 and the photosensitive element 13, the first image light a, which is the reflected portion, is utilized, while the third image light c, which is the transmitted portion, is utilized.

[0086] Please refer to Figure 10 and Figure 11 The present invention also proposes a camera method, which is applied to an AR component, the specific structure of which refers to the above-described embodiments; the camera method includes: S10 puts the light modulation element in the first state so that the third image light is imaged on the photosensitive element; S20 puts the light modulation element in the second state so that ambient light is imaged on the photosensitive element; or S10* puts the light modulation element in a second state so that ambient light is imaged on the photosensitive element; S20* puts the light modulation element in a first state so that the third image light is imaged on the photosensitive element.

[0087] In the above imaging method, when the light modulation element is in the first state, the light intensity of the ambient light shining on the photosensitive element can be reduced, or the light intensity of the third image light on the photosensitive element can be increased, so that the third image light forms a clearer image on the photosensitive element. At this time, by making the third image light form an image on the photosensitive element, a clearer image formed by the third image light can be obtained.

[0088] When the light modulation element is in the second state, the intensity of ambient light shining on the photosensitive element can be increased, thereby making the image of ambient light clearer. At this time, by making the ambient light image on the photosensitive element, a clearer environmental image can be obtained.

[0089] It is evident that switching the state of the optical modulation element when video recording is required can result in a clearer image, thereby improving the final imaging effect.

[0090] It should be noted that the order of operations between steps S10 and S20 can be interchanged, resulting in steps S10* and S20*, without affecting the final imaging effect.

[0091] Furthermore, the aforementioned imaging method facilitates the separate processing of the image formed by the third image light and the image formed by the ambient light. Sometimes, the image formed by the third image light and the projected image observed by the user are inverted images of each other. In this case, the image formed by the third image light can be flipped and then superimposed on the image formed by the ambient light.

[0092] Images formed by ambient light or third-party light sometimes require enhancement. For example, when ambient light is weak, enhancement can be used to make the final image sharper. Similarly, when the third-party light itself is weak, its image can also be enhanced.

[0093] Specifically, in one example, the third-image light imaging can be used only for calibrating pixel relationship positions, while the image actually mapped onto the environment image is the original digital image of the projection image required by the AR component. In this case, the processing of the third-image light imaging can be such as extracting feature images, which facilitates position calibration.

[0094] To more clearly demonstrate the above-described imaging method, this application also provides a specific example, please refer to it. Figure 12 : Users will invoke the AR component's camera function. The system can ask the user whether to capture the screen's image, i.e., whether to capture the image projected by the AR component. If the user chooses not to capture the screen's image, the screen will light up (i.e., the light source will be turned on), displaying a frame to indicate the AR component's expected shooting range. The user can adjust this range, or the range can be a preset, non-adjustable range.

[0095] Once the user determines the frame position to be their desired location, the shooting function is triggered again. At this point, the display screen is turned off to prevent interference from third-party image light or other stray light directly transmitted from the display screen to the image sensor. After the display screen is turned off, the sensor (i.e., the image sensor) is turned on to read the image formed by the ambient light, and then the shooting ends.

[0096] If the user wishes to capture the screen's display, the screen is also illuminated, and a frame can be used to indicate the shooting area. After adjusting the shooting angle, the user can trigger the shooting function again, setting the electrochromic transmittance to its lowest setting (i.e., the transmittance of the first modulation element is set to its lowest) to block ambient light, thus capturing the screen image, i.e., the image formed by the third image light. Then, the electrochromic transmittance is set to its highest setting while the screen is turned off to capture external image information. After capturing the above image information, the image information can be processed separately, and the two processed images are superimposed to obtain the final combined image, after which the shooting ends.

[0097] This invention also proposes an AR glasses system, which includes an AR component or uses the aforementioned imaging method for imaging. The specific structure of the AR component and the specific process of the imaging method are described in the above embodiments. The beneficial effects of this AR glasses are also described in the above-described beneficial effects corresponding to the AR component and imaging method, and will not be repeated here.

[0098] AR glasses are devices worn directly on the user's head. Therefore, the optical waveguide of the AR component can form part of the lens of the AR glasses, while components such as photosensitive elements and light sources can be placed in the frame or temples.

[0099] Of course, the AR components of this application can also be applied to devices such as in-vehicle AR devices or indoor AR devices.

[0100] The above description is merely an exemplary embodiment of the present invention and does not limit the scope of protection of the present invention. Any equivalent structural transformations made based on the technical concept of the present invention and the contents of the specification and drawings of the present invention, or direct / indirect applications in other related technical fields, are included within the scope of protection of the present invention.

Claims

1. An AR component, characterized in that, include: A light source, used to generate the light for the first image; A waveguide assembly is provided with a coupling region; the waveguide assembly is used to receive the first image light and split the first image light at the position of the coupling region into a second image light incident on the waveguide assembly and a third image light reflected. A photosensitive element is used to receive the third image light for imaging and to receive ambient light passing through the coupling region of the waveguide assembly for imaging.

2. The AR component as described in claim 1, characterized in that, The waveguide assembly includes an optical waveguide and an optical modulation element, and the coupling region is disposed on the optical waveguide; the optical modulation element has a first state and a second state. When the light modulation element is in the first state, it blocks ambient light from passing through the coupling area and / or forms the third image light through reflection; When the optical modulation element is in the second state, ambient light is allowed to pass through the coupling area.

3. The AR component as described in claim 2, characterized in that, The optical modulation element includes a first modulation element; the optical waveguide has a coupling direction, and the second image light is emitted from the optical waveguide in the coupling direction; the first modulation element is disposed on the side of the optical waveguide opposite to the coupling direction; When the optical modulation element is in the first state, the first modulation element is used to block ambient light to prevent ambient light from passing through the coupling area; when the optical modulation element is in the second state, the first modulation element is used to transmit ambient light to allow ambient light to pass through the coupling area.

4. The AR component as described in claim 3, characterized in that, The optical waveguide has a transmission region, which is used to allow ambient light to pass through the optical waveguide along the coupling direction. The orthogonal projection of the first modulation element onto the optical waveguide covers the transmission region and the coupling region.

5. The AR component as described in claim 2, characterized in that, The optical modulation element includes a second modulation element; the second modulation element is disposed in the coupling region; When the light modulation element is in the first state, the second modulation element is used to reflect the first image light to form the third image light; when the light modulation element is in the second state, the second modulation element is used to transmit ambient light to allow ambient light to pass through the coupling area.

6. The AR component as described in claim 5, characterized in that, The optical waveguide has a coupling direction and a transmission region. The second image light is emitted from the optical waveguide in the coupling direction. The transmission region is used to allow ambient light to pass through the optical waveguide along the coupling direction. The orthographic projection of the second modulation element onto the optical waveguide is adjacent to or spaced from the transmission region and covers the coupling region.

7. The AR component as claimed in claim 2, characterized in that, The optical modulation element includes an electrochromic element.

8. The AR component as claimed in claim 1, characterized in that, The AR component also includes a beam splitter and a projection lens assembly; The first image light is incident on the beam splitter from the light source; the first image light exits the beam splitter and then enters the projection lens group; the first image light exits the projection lens group and then enters the coupling region directly to form the third image light. The third image light propagates from the coupling region to the beam splitter in the opposite direction to the optical path of the first image light; the third image light is imaged onto the photosensitive element after exiting the beam splitter.

9. A camera recording method, characterized in that, Applied to the AR component as described in any one of claims 2-7; the imaging method includes: The light modulation element is placed in the first state so that the third image light is imaged on the photosensitive element; The light modulation element is placed in the second state so that ambient light is imaged on the photosensitive element; or The light modulation element is placed in the second state so that ambient light is imaged on the photosensitive element; The light modulation element is placed in the first state so that the third image light is imaged on the photosensitive element.

10. An AR glasses, characterized in that, This includes the AR component as described in any one of claims 1-8 or the imaging method as described in claim 9.

Images