Ar component, camera method, and ar glasses

Abstract

The application discloses an AR assembly, a photographing method and AR glasses, and relates to the field of AR technology, wherein the AR assembly comprises a light source, a waveguide assembly and a photosensitive element. The light source is used for generating first image light; the waveguide assembly is provided with a coupling-in area and is used for receiving the first image light and splitting the first image light at the position of the coupling-in area into second image light and reflected third image light, which are incident on the waveguide assembly; and the photosensitive element is used for receiving the third image light to form an image and receiving ambient light passing through the coupling-in area of the waveguide assembly to form an image. The technical scheme provided by the application can improve the effect of the combined image generated by the AR assembly.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of AR technology, in particular to an AR assembly, a photographing method and AR glasses. BACKGROUND

[0002] AR devices allow users to observe the environment while displaying images, thus naturally requiring that the images displayed by the AR devices be mapped into the environment images while the environment images are taken, so as to record the overall view observed by the user. However, AR devices are usually small and lack computing power. If the original digital picture of the image projected by the AR device is directly mapped onto the taken external image by an algorithm to obtain a combined image, the effect is usually poor, and the formed image is easy to be different from the image observed by the user. SUMMARY

[0003] The main purpose of the present application is to provide an AR assembly, a photographing method and AR glasses, which aims to improve the effect of the combined image generated by the AR assembly.

[0004] To achieve the above-mentioned purpose, the AR assembly provided by the present application comprises a light source, a waveguide assembly and a photosensitive element. The light source is used to generate first image light; the waveguide assembly is provided with a coupling-in area; the waveguide assembly is used to receive the first image light and split the first image light at the position of the coupling-in area into second image light incident on the waveguide assembly and third image light reflected; and the photosensitive element is used to image the third image light and receive ambient light through the coupling-in area of the waveguide assembly to image.

[0005] In some embodiments, the waveguide assembly comprises an optical waveguide and an optical modulation element, and the coupling-in area is arranged on the optical waveguide; the optical modulation element has a first state and a second state;

[0006] When the optical modulation element is in the first state, the ambient light is prevented from passing through the coupling-in area, and / or the third image light is formed by reflection;

[0007] When the optical modulation element is in the second state, the ambient light is allowed to pass through the coupling-in area.

[0008] In some embodiments, the optical modulation element comprises a first modulation element; the optical waveguide has a coupling-out direction, and the second image light exits the optical waveguide in the coupling-out direction; and the first modulation element is arranged on the side of the optical waveguide opposite to the coupling-out direction;

[0009] The first modulation element is configured to block ambient light when the light modulation element is in the first state, and is configured to transmit ambient light when the light modulation element is in the second state.

[0010] In some embodiments, the optical waveguide has a transmission region configured to transmit ambient light along the out-coupling direction through the optical waveguide.

[0011] A projection of the first modulation element on the optical waveguide covers the transmission region and the in-coupling region.

[0012] In some embodiments, the light modulation element comprises a second modulation element; the second modulation element is disposed at the in-coupling region.

[0013] The second modulation element is configured to reflect the first image light to form the third image light when the light modulation element is in the first state, and is configured to transmit ambient light when the light modulation element is in the second state.

[0014] In some embodiments, the optical waveguide has an out-coupling direction and a transmission region; the second image light is out-coupled from the optical waveguide along the out-coupling direction; the transmission region is configured to transmit ambient light along the out-coupling direction through the optical waveguide.

[0015] A projection of the second modulation element on the optical waveguide is adjacent to or spaced apart from the transmission region, and covers the in-coupling region.

[0016] In some embodiments, the light modulation element comprises an electrochromic element.

[0017] In some embodiments, the AR assembly further comprises a light splitting element and a projection lens group.

[0018] The first image light is incident on the light splitting element from the light source; the first image light is incident on the projection lens group after being out-coupled from the light splitting element; the first image light is incident on the in-coupling region perpendicularly after being out-coupled from the projection lens group, to form the third image light.

[0019] The third image light propagates along a direction opposite to a light path of the first image light from the in-coupling region to the light splitting element; the third image light is imaged on the photosensitive element after being out-coupled from the light splitting element.

[0020] The present application further provides a photographing method applied to the AR assembly; the photographing method comprises:

[0021] putting the light modulation element in the first state to make the third image light image on the photosensitive element;

[0022] putting the light modulation element in the second state to make ambient light image on the photosensitive element;

[0023] or

[0024] putting the light modulation element in the second state to make ambient light image on the photosensitive element;

[0025] putting the light modulation element in the first state to make the third image light image on the photosensitive element.

[0026] The application further provides an AR glasses comprising the AR assembly or using the image capturing method.

[0027] In the technical scheme, the same photosensitive element plays two roles, one is to receive the third image light, and the other is to receive ambient light passing through the coupling-in area of the waveguide assembly; the third image light is split from the first image light, so that receiving the third image light can form the image projected by the AR assembly; the ambient light can form the image of the outside world. Therefore, the same photosensitive element simultaneously presents the projected image and the image of the outside world; therefore, the digital picture of the projected image and the digital picture of the image of the outside world obtained by the photosensitive element have a simple mapping relationship between the pixels, so that the digital picture of the projected image can be accurately mapped to the digital picture of the image of the outside world to obtain a combined image with better effect under the condition of weak computing power. BRIEF DESCRIPTION OF DRAWINGS

[0028] In order to more clearly illustrate the technical scheme in the embodiments of the application or the prior art, the drawings needed to be used in the embodiments or the prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative labor.

[0029] Figure 1 The structural schematic diagram of the first embodiment of the AR assembly provided by the application;

[0030] Figure 2 The optical path schematic diagram of the first embodiment of the AR assembly provided by the application;

[0031] Figure 3 The structural schematic diagram of the second embodiment of the AR assembly provided by the application;

[0032] Figure 4A first optical path schematic diagram of a second embodiment of the AR assembly provided by the present application;

[0033] Figure 5 A second optical path schematic diagram of the second embodiment of the AR assembly provided by the present application;

[0034] Figure 6 A structure schematic diagram of a third embodiment of the AR assembly provided by the present application;

[0035] Figure 7 A first optical path schematic diagram of the third embodiment of the AR assembly provided by the present application;

[0036] Figure 8 A second optical path schematic diagram of the third embodiment of the AR assembly provided by the present application;

[0037] Figure 9 A structure schematic diagram of an embodiment of the projection assembly, the light splitting element, the photosensitive element and the light source of the AR assembly provided by the present application;

[0038] Figure 10 A flow chart of a first embodiment of the image capturing method provided by the present application;

[0039] Figure 11 A flow chart of a second embodiment of the image capturing method provided by the present application;

[0040] Figure 12 A flow chart of a third embodiment of the image capturing method provided by the present application.

[0041] Explanation of reference numerals:

[0042] AR assembly 10;

[0043] Light source 11;

[0044] Waveguide assembly 12; optical waveguide 121; transmission area 121a; light modulation element 122; first modulation element 1221; second modulation element 1222; coupling-in area 12a;

[0045] Photosensitive element 13;

[0046] Light splitting element 14;

[0047] Projection lens group 15; diaphragm 151; lens 152;

[0048] First image light a; second image light b; third image light c; ambient light d;

[0049] Human eye 20.

[0050] The implementation, functional features and advantages of the present application will be further explained with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION

[0051] The technical solutions in the embodiments of the present application will be clearly and completely described in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the protection scope of the present application.

[0052] It should be noted that if the embodiments of the present application involve directional indications (such as up, down, left, right, front, back, etc.), the directional indications are only used to explain the relative positional relationship, movement condition, etc. between components in a certain posture, and if the certain posture changes, the directional indications also change accordingly.

[0053] In addition, if the embodiments of the present application involve descriptions such as “first”, “second”, etc., the descriptions of “first”, “second”, etc. are only for description purposes, and cannot be understood as indicating or implying the relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features limited by “first”, “second” can explicitly or implicitly include at least one of the features. In addition, “and / or” or “and / or” appearing throughout the text means that the three parallel schemes are included, for example, “A and / or B” includes A scheme, or B scheme, or A and B are satisfied at the same time. In addition, the technical solutions of each embodiment can be combined with each other, but it must be based on the realization of those skilled in the art, and when the combination of technical solutions contradicts each other or cannot be realized, it should be considered that the combination of technical solutions does not exist, and is not within the protection scope of the present application.

[0054] The present application provides an AR assembly.

[0055] Please refer to Figure 1 and Figure 2 The AR assembly 10 provided by the present application includes a light source 11, a waveguide assembly 12 and a photosensitive element 13. The light source 11 is used to generate first image light a; the waveguide assembly 12 is provided with a coupling-in area 12a; the waveguide assembly 12 is used to receive the first image light a, and to split the first image light a at the position of the coupling-in area 12a into second image light b incident on the waveguide assembly 12 and reflected third image light c; the photosensitive element 13 is used to receive the third image light c to form an image, and is used to receive ambient light d passing through the coupling-in area 12a of the waveguide assembly 12 to form an image.

[0056] The light source 11 is a light source 11 that projects a virtual image for the AR assembly 10, and can be a light source of a type such as LBS (Laser Beam Scanning), Micro LED, LCD (Liquid Crystal Display), and the like. The light source 11 can constitute a light engine of the AR assembly 10 alone, or can be combined with an imaging lens group in the following embodiments to form a light engine of the AR assembly 10.

[0057] The light source 11 can be understood as a component that can form a complete projected image, and should not be considered as merely an element that provides light. For example, when the light source 11 is a light source of a DLP (Digital Light Processing) type, the light source 11 can include a lamp (a lamp of a type such as a mercury lamp, a light-emitting diode, or a laser diode) that provides backlight, a color wheel that filters light, and a DMD (Digital Micromirror Device) that modulates light, and should not be considered as merely including the lamp.

[0058] In another example, when the light source 11 is a light source of an LCD type, the light source 11 can include a backlight assembly, a liquid crystal element, a polarizing plate, and a TFT (Thin-Film Transistor), and should not be considered as merely including the backlight assembly.

[0059] The first image light a emitted by the light source 11 is light that contains information of a projected image required by the AR assembly 10. For some types of light sources 11 (for example, a light source 11 of an LBS type), the light source 11 itself has a proper VID (Virtual Image Distance), and after propagation through the waveguide assembly 12, the light source 11 can be coupled out to the human eye 20 to form an image projected by the AR assembly 10. For other types of light sources 11 (for example, a light source 11 of a Micro LED or LCD type), the light source 11 can be combined with a projection lens group 15 proposed below to obtain a proper VID.

[0060] The waveguide assembly 12 is an assembly that bends the image light projected by the light source 11 and transmits ambient light d. In some embodiments, the waveguide assembly 12 can only include an optical waveguide, and in other embodiments, the waveguide assembly 12 can further include a device that can modulate the coupling-in and transmission of the optical waveguide (for details, please refer to the light modulation element in the following embodiments).

[0061] The optical waveguide has a refractive index greater than that of the medium surrounding the optical waveguide, so that the image light coupled into the optical waveguide can propagate in the optical waveguide, but at the same time the optical waveguide is substantially transparent, so that ambient light can be transmitted. For details, please refer to Figure 2Because of the turning of the light emitted by the light source 11 by the waveguide assembly 12, the light source 11 can be arranged not in front of the human eye 20, i.e. not blocking the central field of view of the human eye 20; meanwhile, the waveguide is substantially transparent, so that the ambient light d can directly pass through the waveguide assembly 12 to be received by the human eye 20. In this way, the user can simultaneously observe the ambient scene and the projected image.

[0062] The waveguide assembly 12 has a coupling-in region 12a, please refer to Figure 2 and Figure 3 The coupling-in region 12a of the waveguide assembly 12 is the region where light is coupled into the waveguide 121. The surface of the waveguide 121 in the coupling-in region 12a can be used to receive the light emitted by the light source 11, i.e. the first image light a; if it is necessary to change the propagation direction of the light entering the waveguide 121, a coupling-in grating can also be arranged in the coupling-in region 12a, so that the light entering the waveguide 121 can propagate in the waveguide 121 in a total reflection manner.

[0063] The coupling-in region 12a of the waveguide assembly 12 is the region where light is coupled into the waveguide 121, so the first image light a will meet the surface of the waveguide 121 in the coupling-in region 12a; because the refractive index of the waveguide 121 is different from the refractive index of the medium (usually air) around the waveguide 121, partial reflection and partial transmission will occur, the transmitted light will enter the waveguide 121 to become the second image light b; the reflected light will be reflected according to the law of reflection to become the third image light c. Of course, it is not necessary to reflect on the surface of the waveguide 121 to form the third image light c, it can also be reflected on the light modulation element in the following embodiments to form the third image light c.

[0064] Because the second image light b and the third image light c are split from the first image light a, they both carry the image information in the first image light a, if the second image light b and the third image light c can be imaged, the projected image of the AR assembly 10 can be formed.

[0065] The light sensing element 13 is an element for receiving light information, for example, it can be a CCD (Charge-coupled Device) or a CMOS (Complementary Metal-Oxide Semiconductor) type light sensing element. Generally, an imaging lens group needs to be arranged for the light sensing element 13 to image on the light sensing element 13, but please refer to the following embodiments, in some embodiments, the projection lens group can be used as the imaging lens group of the light sensing element in cooperation with the light splitting element and the projection lens group, in some embodiments, if the light source 11 is an LBS type light source, the imaging lens group can also not be arranged, only the light sensing element 13 needs to be arranged at a position conjugate to the virtual image formed by the LBS, and the projected image can be obtained.

[0066] The photosensitive element 13 can receive ambient light d to form an image, in addition to receiving the third image light c to obtain image information of the projected image. Please refer to Figure 2 In some embodiments, the ambient light d received by the photosensitive element 13 can be transmitted through the waveguide assembly 12, so that the ambient image obtained by the photosensitive element 13 has no difference in visual axis with the ambient image obtained by the human eye 20, and can better reflect the visual scene of the human eye 20; in other embodiments, the ambient light d received by the photosensitive element 13 can also directly pass through the waveguide assembly 12 (not shown in the figure) to avoid image distortion and improve imaging effect.

[0067] The digital information of the image to be projected by the AR assembly 10 can be an image pre-stored by the AR assembly 10, or an image generated by the controller of the AR assembly 10 according to environmental information, user instructions and / or pre-stored algorithms. However, in either case, the digital information of the image has been generated in the controller before the image is projected, or before the first image light a is emitted. The digital information is the original digital picture of the projected image.

[0068] If the photosensitive element 13 is only used to obtain the ambient light d to obtain an environmental image, and then the original digital picture described above is directly mapped onto the environmental image, the mapping relationship changes with factors such as object distance and depth of field when the environmental image is taken, and a complex algorithm is needed to correct the mapping relationship. Otherwise, the final combined image will not be consistent with the image actually seen by the user, for example, the user may see that the number in the projected image is originally in the center of the field of view, but in the combined image, the number is located near the edge.

[0069] The AR assembly 10 proposed in the present application uses the photosensitive element 13 used to obtain the ambient light d to also obtain the third image light c. Since the third image light c is split from the first image light a, it has the image information in the first image light a. In this way, the environmental image and the projected image can be imaged on one photosensitive element 13. The corresponding relationship between the pixels of the digital picture of the environmental image obtained by the photosensitive element 13 and the digital picture of the projected image is simpler, so that only a simple algorithm is needed to accurately combine the two pictures.

[0070] It can be seen that the technical solution of the present application reduces the risk of misplacement in the combined image, and makes the effect of the combined image better.

[0071] Please refer to Figure 3 and Figure 6 In some embodiments, the waveguide assembly 12 includes an optical waveguide 121 and an optical modulation element 122, and the coupling-in area 12a is arranged on the optical waveguide 121; the optical modulation element 122 has a first state and a second state;

[0072] Please refer to Figure 4 and Figure 7 When the light modulating element 122 is in the first state, the ambient light d is hindered from passing through the coupling-in region 12a, and / or a third image light c is formed by reflection.

[0073] Please refer to Figure 5 and Figure 8 When the light modulating element 122 is in the second state, the ambient light d is allowed to pass through the coupling-in region 12a.

[0074] The light modulating element 122 is an element that can control its transmittance and / or reflectance. Although for a general homogeneous medium, the transmittance and reflectance tend to add up to 1, and are not independent physical quantities. However, the light modulating element 122 in some embodiments can absorb light.

[0075] In one example of the light modulating element 122, the light modulating element 122 can be a combination of a liquid crystal film and polarizers, with the polarizers of mutually perpendicular vibration directions arranged on both sides of the liquid crystal film. In this way, when the optical rotation angle of the liquid crystal film is 0°, the light modulating element 122 can have a high absorption rate (substantially no light transmission), and at this time, the light is also substantially not reflected; when the optical rotation angle of the liquid crystal film is 90°, the absorption rate of light can be substantially 0 (substantially transparent).

[0076] In another example of the light modulating element 122, the light modulating element 122 can be an electrochromic element. Some types of electrochromic elements can form an effect similar to the above-mentioned liquid crystal type light modulating element 122, i.e., can switch between a state of strongly absorbing light and a state of substantially transmitting light; other types of electrochromic elements can switch between a state of high reflectivity and a state of high transmittance.

[0077] In some embodiments, the light modulating element 122 can also be a combination of an electrochromic element and a photonic crystal to improve the speed of state switching.

[0078] In the above-mentioned examples of the light modulating element 122, it can be seen that in some embodiments, the light modulating element 122 in the first state can have a high absorption rate. When the light modulating element 122 is arranged on the light path between the ambient light d from the outside and the optical waveguide 121, the ambient light d can be absorbed, i.e., the ambient light d is hindered from passing through the coupling-in region 12a.

[0079] In one example, the light modulating element 122 is arranged on the side of the optical waveguide 121 where ambient light enters, and the orthographic projection of the light modulating element 122 at least partially covers the optical waveguide 121, so that at least part of the ambient light entering the optical waveguide 121 is absorbed, and the ambient light d is prevented from passing through the coupling-in region 12a.

[0080] In the example mentioned above regarding the light modulating element 122, it can be seen that in some embodiments, the light modulating element 122, in the second state, can have high transmittance, and the light modulating element 122 does not affect the operation of other elements of the AR assembly 10, and can thus transmit ambient light, so that the ambient light at least partially enters the optical waveguide 121. The ambient light entering the optical waveguide 121 can be directly transmitted, or can be transmitted in the optical waveguide 121 and then coupled out from the coupling-in region 12a, i.e., the ambient light d is allowed to pass through the coupling-in region 12a.

[0081] In one example, the light modulating element 122 is arranged at a position where the second image light b is coupled out of the optical waveguide 121, so that the second image light b is reflected back into the optical waveguide 121 and exits from the coupling-in region 12a to form the third image light c.

[0082] In the example mentioned above regarding the light modulating element 122, it can be seen that in some embodiments, the light modulating element 122, in the second state, can have high transmittance, and the light modulating element 122 does not affect the operation of other elements of the AR assembly 10, and can thus transmit ambient light, so that the ambient light at least partially enters the optical waveguide 121. The ambient light entering the optical waveguide 121 can be directly transmitted, or can be transmitted in the optical waveguide 121 and then coupled out from the coupling-in region 12a, i.e., the ambient light d is allowed to pass through the coupling-in region 12a.

[0083] In the embodiments provided with the light modulating element 122, when the light modulating element 122 is in the first state, for the embodiments in which the ambient light d is prevented from passing through the coupling-in region 12a, the light intensity of the ambient light d on the photosensitive element 13 can be reduced; for the embodiments in which the third image light c is formed by reflection, the light intensity of the third image light c on the photosensitive element 13 can be increased. This can make the brightness of the projected image on the photosensitive element 13 greater than the brightness of the ambient image, so that the projected image is clearer and more conducive to post-processing.

[0084] In the embodiments provided with the light modulating element 122, when the light modulating element 122 is in the second state, the ambient light d can normally enter the optical waveguide 121 and be clearly imaged on the photosensitive element 13.

[0085] Please refer to the following embodiments, for the imaging of the photosensitive element 13 for the projection image, and the imaging of the photosensitive element 13 for the external scene can be alternately performed, so that the imaging of the projection image can be performed in the first state, and the imaging of the environment scene can be performed in the second state, thereby improving the final imaging effect, but since it is only the modulation of light intensity, the mapping relationship of the two images is not changed, thereby the effect of the combined image can be improved.

[0086] It should be noted that the imaging of the photosensitive element 13 for the environment image and the projection image can also be performed simultaneously, because the transmittance (or reflectance) of the light modulation element 122 can be any value between 0 and 1 in principle, so that the light intensity of the ambient light or the light intensity of the third image light c can be appropriately reduced or increased, so that the imaging brightness of the two is close, and then the two can be performed simultaneously.

[0087] In addition, appropriate adjustment of the incidence of the ambient light d is also beneficial to the identification of the relationship between the user's gesture and the projection image. The image of the user's gesture can also be contained in the environment image, and since the environment image and the projection image can be clearly imaged on the photosensitive element 13 at the same time, the relationship between the gesture and the projection image can be easily judged. This facilitates the system to identify the user's gesture, so that the user can more accurately operate the AR device through the gesture.

[0088] Please refer to Figure 3 In some embodiments, the light modulation element 122 includes a first modulation element 1221; the optical waveguide 121 has a coupling-out direction, and the second image light b exits the optical waveguide 121 in the coupling-out direction; the first modulation element 1221 is arranged on the side of the optical waveguide 121 away from the coupling-out direction.

[0089] When the light modulation element 122 is in the first state, the first modulation element 1221 is used to block the ambient light d to hinder the ambient light d from passing through the coupling-in area 12a; when the light modulation element 122 is in the second state, the first modulation element 1221 is used to transmit the ambient light d to allow the ambient light d to pass through the coupling-in area 12a.

[0090] The coupling-out direction of the optical waveguide 121 is the direction of the second image light b after being transmitted in the optical waveguide 121 and then coupled out of the optical waveguide 121, which is often the direction of the optical waveguide 121 facing the human eye 20 in use. The first modulation element 1221 is arranged on the side of the optical waveguide 121 away from the coupling-out direction, i.e., on the side of the optical waveguide 121 receiving the ambient light, so that the light modulation element 122 can absorb the ambient light from the source, improving the efficiency of hindering the ambient light from entering the optical waveguide 121.

[0091] Please refer to Figure 3In some embodiments, the optical waveguide 121 has a transmission region 121a for allowing ambient light rays d to pass through the optical waveguide 121 in a coupling-out direction;

[0092] The first modulating element 1221 is projected onto the transmission region 121a and the coupling-in region 12a on the optical waveguide 121.

[0093] The transmission region 121a is a viewing region for a user, i.e., the user can directly see the ambient scene through the transmission region. Often, the transmission region also at least partially overlaps with a coupling-out region of the optical waveguide 121, i.e., a region where the second image light b is coupled out of the optical waveguide 121. The coupling-out region is configured to change the propagation direction of the second image light b in the optical waveguide 121 from a total reflection state, and can also be provided with a coupling-out grating or the like to normally couple the second image light b out of the optical waveguide 121.

[0094] It can be seen that the direction of the ambient light rays d passing through the transmission region 121a is consistent with the direction of the second image light b coupled out of the optical waveguide 121, because both the ambient light rays d and the second image light b need to be imaged on the human eye 20.

[0095] The first modulating element 1221 is projected onto the transmission region 121a and the coupling-in region 12a on the optical waveguide 121, so as to comprehensively cover the ambient light rays d from the optical waveguide 121, thereby improving the imaging effect of the third image light c. The coupling-in region 12a needs to be covered because the optical waveguide 121 is substantially transparent, and part of the ambient light rays d can pass through the coupling-in region 12a.

[0096] In addition, since the first modulating element 1221 can adjust the transmittance of the ambient light rays d, when the AR assembly 10 is applied to AR glasses, the effect of variable color glasses can also be formed.

[0097] Please refer to Figure 6 In some embodiments, the light modulating element 122 includes a second modulating element 1222; the second modulating element 1222 is arranged at the coupling-in region 12a;

[0098] When the light modulating element 122 is in the first state, the second modulating element 1222 is configured to reflect the first image light a to form the third image light c; when the light modulating element 122 is in the second state, the second modulating element 1222 is configured to transmit the ambient light rays d to allow the ambient light rays d to pass through the coupling-in region 12a.

[0099] The second modulating element 1222 is arranged at the coupling-in region 12a, which means that the second modulating element 1222 is projected onto the coupling-in region 12a on the optical waveguide 121. Since the first image light a is also coupled into the optical waveguide 121 at the coupling-in region 12a, the second modulating element 1222 can directly reflect the first image light a to form the third image light c.

[0100] Since the first image light a is not coupled into the optical waveguide 121, the transmission loss is small, so the light intensity of the third image light c can be greatly improved, so that the imaging of the projected image on the photosensitive element 13 is clearer or the exposure time can be shorter.

[0101] In addition, please refer to Figure 7 In fact, when the second modulation element 1222 has high reflectivity, it can also reflect the ambient light d coupled out from the coupling-in area 12a, further improving the imaging effect of the third image light c.

[0102] Please refer to Figure 6 In some embodiments, the optical waveguide 121 has a coupling-out direction and a transmission area 121a, the second image light b exits the optical waveguide 121 in the coupling-out direction, and the transmission area 121a is used to make the ambient light d pass through the optical waveguide 121 along the coupling-out direction;

[0103] The orthographic projection of the second modulation element 1222 on the optical waveguide 121 is adjacent to or spaced from the transmission area 121a, and covers the coupling-in area 12a.

[0104] The second modulation element 1222 covers the coupling-in area 12a, so that all the first image light a can be reflected, improving the light intensity of the third image light c. At the same time, the second modulation element 1222 is adjacent to or spaced from the transmission area 121a, so as not to block the user's view. Even if the second modulation element 1222 is in a high reflectivity state, the user can clearly observe the outside scene.

[0105] The transmission area 121a may sometimes not have a clear range. For this case, it can be considered that the second modulation element 1222 adjacent to or spaced from the transmission area 121a is equivalent to adjacent to or spaced from the coupling-out area.

[0106] It should be noted that in some embodiments, the first modulation element 1221 and the second modulation element 1222 can be simultaneously provided. In this way, the first modulation element 1221 and the second modulation element 1222 can cooperate to weaken the input of the ambient light d, and at this time, the first modulation element 1221 and the second modulation element 1222 can not be switched to the completely light-shielding and completely reflecting states, that is, they can have sufficient weakening degree of external light to make the third image light c clearly imaged. Since the state transformation degree is smaller, the first modulation element 1221 and the second modulation element 1222 can have higher switching speed between the first state and the second state, so as to quickly complete the imaging.

[0107] Increasing the switching speed of the light modulation element 122 can even realize continuous shooting to form the effect of video recording.

[0108] In some embodiments, the second modulating element 1222 can be a shutter, and a reflective structure (e.g., a reflective film coated on the shutter, a reflective film attached to the shutter, or a mirror) can be arranged on the side of the curtain or leaf of the shutter facing the first image light a, so that when the shutter is open, ambient light d can pass through; when the shutter is closed, the first image light a can be reflected to form the second image light b.

[0109] In some embodiments, the light modulating element includes an electrochromic element.

[0110] Some electrochromic elements can have a subtle switching speed, and the control is simple, often only two conductive films need to be arranged on the two side surfaces of the electrochromic material, so that the electrochromic material can be controlled, and therefore the electrochromic element can have sufficient switching speed between the first state and the second state, and also has a simpler and easier to implement structure.

[0111] It should be noted that the light modulating element includes an electrochromic element, so that the first modulating element and the second modulating element can be electrochromic elements. Please refer to Figure 3 and Figure 6 In the embodiments shown in Figure 3 and Figure 6 , the first modulating element 1221 and the second modulating element 1222 are electrochromic elements.

[0112] Please refer to Figure 1 and Figure 9 In some embodiments, the AR assembly 10 further includes a light splitting element 14 and a projection lens group 15;

[0113] The first image light a is incident on the light splitting element 14 from the light source 11; the first image light a is incident on the projection lens group 15 after being emitted from the light splitting element 14; the first image light a is normally incident on the coupling-in region 12a after being emitted from the projection lens group 15, to form the third image light c;

[0114] The third image light c propagates in the direction opposite to the light path of the first image light a from the coupling-in region 12a to the light splitting element 14; the third image light c is imaged on the photosensitive element 13 after being emitted from the light splitting element 14.

[0115] The light splitting element 14 is an element that can split the incident light beam into at least two beams of emitted light, for example, a glass sheet provided with a functional film can be used, which can be a one-dimensional photonic crystal structure film, so as to have a specific transmittance and reflectance for a specific waveband (for the AR assembly 10, it can be a visible light waveband), for example, the transmittance can be 40%, the reflectance can be 60%, or both the transmittance and the reflectance can be 50%. The functional film can also be a metal coating film, and by changing the duty ratio or film thickness of the coating film on the glass surface, the transmittance and reflectance can also be changed.

[0116] The light splitting element 14 can also be a PBS (Polarizing Beam Splitter) type of light splitter. The PBS can be made of a birefringent crystal, or can be a light splitter that utilizes alternating refractive index distribution to split light at Brewster's angle.

[0117] The projection lens 15 is a lens group that shapes the first image light a. The shaping can be magnification, reduction, or appropriate deformation of the image information carried by the first image light a, or can be used to change the VID of the AR assembly 10. In one example, referring to FIG. 2, the projection lens 15 can include a plurality of lenses 152 and a stop 151, where the lenses 152 can be used to shape the first image light a and / or to correct aberrations of the AR assembly 10; the stop 151 can be used to control the range of the light beam passing through the projection lens 15. In another example, the projection lens 15 can have an exit pupil diameter of 3.3 mm, an F number of 2.7, a diagonal FOV of 30°, and a total length of 10 mm. Figure 9

[0118] The first image light a can be incident on the light splitting element 14 from the light source 11, and can be split by the light splitting element 14 before being emitted. The first image light a split by the light splitting element 14 can cause one of the beams to enter the waveguide 121 to be projected, and the other beam can be discarded directly, or can be used for other purposes.

[0119] The emitted beam of the first image light a from the light splitting element 14 is incident on the projection lens 15, and is shaped by the projection lens 15 before being incident on the in-coupling region 12a of the waveguide assembly 12. The in-coupling region 12a is a region where the incident light is in-coupled into the waveguide 121. In one example, the in-coupling region 12a is a region where the incident light is in-coupled into the waveguide 121 perpendicularly. In another example, the in-coupling region 12a is a region where the incident light is in-coupled into the second modulating element 1222 perpendicularly.

[0120] It can be seen that the first image light a is reflected by the in-coupling region 12a to form the third image light c, which propagates in the opposite direction of the first image light a. Therefore, the third image light c naturally passes through the projection lens 15. In this way, the third image light c and the first image light a share a set of lenses 152 (in some embodiments, a mirror can also be included instead of the lenses 152). Since the first image light a is emitted from the light source 11 and is incident on the projection lens 15 only after passing through the light splitting element 14, which generally does not shape the light, the first image light a and the third image light c can obtain "opposite" shaping effects.

[0121] ​After the third image light c passes through the projection lens group 15, it will naturally be incident on the light splitting element 14, at which time the light splitting element 14 can split the third image light c, and at least one of the split third image light c is not emitted in the direction of the light source 11, and the photosensitive element 13 can be arranged at a position that can receive this emitted light, so as not to conflict with the position of the light source 11.

[0122] Similarly, since the light splitting element 14 generally does not shape the light beam, and the third image light c is subjected to shaping that is "opposite" to that of the first image light a, the photosensitive element 13 can be arranged at a position that is image conjugate with the light source 11, i.e. the image information of the projected image can be obtained.

[0123] In addition, the ambient light d propagating from the waveguide assembly 12 can also be imaged on the photosensitive element 13 by the projection lens group 15, since it passes through the coupling-in region 12a, and the ambient light d is subjected to the same shaping as the third image light c.

[0124] It can be seen that in the above embodiment, the first image light a, the third image light c and the ambient light d share a set of lenses 152, which on the one hand makes the AR assembly 10 more compact, and on the other hand makes the correspondence between the pixels of the image formed by the ambient light d and the image formed by the third image light c more explicit and simple, thereby facilitating further improvement of the combined image.

[0125] It should be noted that when the light splitting element 14 is a light splitting mirror (i.e. a mirror with the above functional film on its surface), the first image light a that is utilized after the light splitting mirror can be transmitted light, i.e. in the optical path shown in Figure 9 , the transmitted first image light a is utilized, and at this time, the reflected third image light c can be utilized for imaging. However, in other examples, for example, in Figure 9 , the positions of the light source 11 and the photosensitive element 13 are exchanged, so that the first image light a is reflected and the third image light c is transmitted.

[0126] Please refer to Figure 10 and Figure 11 , the present application also proposes a photographing method, which is applied to an AR assembly, and the specific structure of the AR assembly is referred to the above embodiments; the photographing method comprises:

[0127] S10, causing the light modulating element to be in a first state, so as to cause the third image light to be imaged on the photosensitive element;

[0128] S20, causing the light modulating element to be in a second state, so as to cause the ambient light to be imaged on the photosensitive element;

[0129] or

[0130] S10* makes the light modulation element in the second state to make the ambient light image on the photosensitive element;

[0131] S20* makes the light modulation element in the first state to make the third image light image on the photosensitive element.

[0132] In the above imaging method, when the light modulation element is in the first state, the light intensity of the ambient light irradiated on the photosensitive element can be reduced, or the light intensity of the third image light on the photosensitive element is increased, so that the third image light is imaged more clearly on the photosensitive element. At this time, the third image light is imaged on the photosensitive element, and a clearer image formed by the third image light is obtained.

[0133] When the light modulation element is in the second state, the light intensity of the ambient light irradiated on the photosensitive element can be larger, so that the ambient light is imaged more clearly. At this time, the ambient light is imaged on the photosensitive element to obtain a clearer ambient image.

[0134] It can be seen that when imaging is needed, the state of the light modulation element is switched to obtain clearer imaging to improve the final imaging effect.

[0135] It should be noted that the operation sequence between the above S10 step and S20 step can be exchanged, that is, the forming steps S10* and S20* are formed, and the final imaging effect is not affected.

[0136] In addition, the above imaging method can also facilitate the separate processing of the image formed by the third image light and the image formed by the ambient light. Sometimes the image formed by the third image light and the projection image observed by the user are mutually inverted, at which time the image formed by the third image light is flipped and then superimposed with the image formed by the ambient light.

[0137] The image formed by the ambient light or the image formed by the third image light sometimes also needs to be enhanced. For example, when the ambient light is weak, it can be enhanced to make the finally obtained image clearer. When the light of the third image light itself is weak, the image formed by it can also be enhanced.

[0138] In particular, in one example, the image formed by the third image light can be used only for calibrating the position of the pixel relationship, and the image actually mapped to the ambient image is the original digital picture of the projection image required by the AR component. At this time, the processing of the image formed by the third image light can be extracting a feature image, etc., which is beneficial to the processing of calibrating the position.

[0139] To more clearly show the above imaging method, the present application also shows a specific example, please refer to Figure 12 :

[0140] The user can call the shooting function of the AR component, and the system can ask the user whether to shoot the screen or not, that is, whether to shoot the picture projected by the AR component. If the user chooses not to shoot the screen, the screen is turned on (that is, the light source is turned on), and a frame is displayed to prompt the user of the shooting range expected by the AR component. The user can adjust the range, or the range is a preset range that cannot be adjusted.

[0141] When the user determines that the frame position is the desired position, the shooting function is continued to be triggered, and the screen is turned off to avoid the third image light or other stray light directly propagating from the screen to the imaging element. After the screen is turned off, the sensor (that is, the imaging element) is turned on to read the image formed by the ambient light, and the shooting is ended.

[0142] If the user wants to shoot the display screen, the screen is also turned on, and the frame can be used to prompt the user of the shooting range. After the user adjusts the shooting angle, the shooting function is continued to be triggered, and the electrochromic transmittance is adjusted to the minimum (that is, the transmittance of the first modulation element is adjusted to the minimum) to hinder the ambient light from entering. At this time, the screen picture, that is, the picture formed by the third image light, can be collected. Then, the electrochromic transmittance is adjusted to the maximum, and the screen is turned off to collect the image information of the outside. After the above image information is collected, the image information can be processed respectively, and the processed two images are superimposed to obtain the final combined image, and the shooting is ended.

[0143] The application also provides an AR glasses, which comprises the AR component or uses the above-mentioned image shooting method. The specific structure of the AR component is referred to the above-mentioned embodiments, and the specific process of the image shooting method is referred to the above-mentioned embodiments. The beneficial effects of the AR glasses are also referred to the beneficial effects of the AR component and the image shooting method, which will not be repeated here.

[0144] The AR glasses are directly worn on the head of the user, so that the optical waveguide of the AR component can form part of the lens of the AR glasses, and the imaging element and the light source and other components can be arranged at the position of the frame or the leg.

[0145] Of course, the AR component of the application can also be applied to devices such as vehicle-mounted AR devices or indoor AR devices.

[0146] The above-mentioned is only an exemplary embodiment of the application, and does not limit the protection scope of the application. Any equivalent structural transformation made by referring to the content of the specification and drawings, or direct / indirect application in other related technical fields is included in the protection scope of the application.

Claims

1. An AR component, characterized in that, For use in AR glasses, the AR component includes: A light source, used to generate the light for the first image; A waveguide assembly includes an optical waveguide and an optical modulation element, wherein a coupling region is provided on the optical waveguide; the waveguide assembly is used to receive the first image light and to split the first image light at the position of the coupling region into a second image light incident on the waveguide assembly and a third image light reflected; the optical waveguide is used to form a part of the lens of the AR glasses; A photosensitive element is used to receive the third image light for imaging and to receive ambient light passing through the coupling region of the waveguide assembly for imaging. The optical modulation element includes a first modulation element; the optical waveguide has a coupling direction, and the second image light is emitted from the optical waveguide in the coupling direction; the first modulation element is disposed on the side of the optical waveguide opposite to the coupling direction; the optical modulation element has a first state and a second state. When the optical modulation element is in the first state, the first modulation element is used to block ambient light to prevent ambient light from passing through the coupling area; when the optical modulation element is in the second state, the first modulation element is used to transmit ambient light to allow ambient light to pass through the coupling area.

2. The AR component as described in claim 1, characterized in that, When the optical modulation element is in the first state, the waveguide component blocks ambient light from passing through the coupling area and / or forms the third image light through reflection. When the optical modulation element is in the second state, ambient light is allowed to pass through the coupling area.

3. The AR component as described in claim 2, characterized in that, The optical waveguide has a transmission region, which is used to allow ambient light to pass through the optical waveguide along the coupling direction. The orthogonal projection of the first modulation element onto the optical waveguide covers the transmission region and the coupling region.

4. The AR component as described in claim 2, characterized in that, The optical modulation element includes a second modulation element; the second modulation element is disposed in the coupling region; When the light modulation element is in the first state, the second modulation element is used to reflect the first image light to form the third image light; when the light modulation element is in the second state, the second modulation element is used to transmit ambient light to allow ambient light to pass through the coupling area.

5. The AR component as described in claim 4, characterized in that, The optical waveguide has a coupling direction and a transmission region. The second image light is emitted from the optical waveguide in the coupling direction. The transmission region is used to allow ambient light to pass through the optical waveguide along the coupling direction. The orthographic projection of the second modulation element onto the optical waveguide is adjacent to or spaced from the transmission region and covers the coupling region.

6. The AR component as described in claim 2, characterized in that, The optical modulation element includes an electrochromic element.

7. The AR component as claimed in claim 1, characterized in that, The AR component also includes a beam splitter and a projection lens assembly; The first image light is incident on the beam splitter from the light source; the first image light exits the beam splitter and then enters the projection lens group; the first image light exits the projection lens group and then enters the coupling region directly to form the third image light. The third image light propagates from the coupling region to the beam splitter in the opposite direction to the optical path of the first image light; the third image light is imaged onto the photosensitive element after exiting the beam splitter.

8. A camera recording method, characterized in that, Applied to the AR component as described in any one of claims 2-6; the imaging method includes: The light modulation element is placed in the first state so that the third image light is imaged on the photosensitive element; The light modulation element is placed in the second state so that ambient light is imaged on the photosensitive element; or The light modulation element is placed in the second state so that ambient light is imaged on the photosensitive element; The light modulation element is placed in the first state so that the third image light is imaged on the photosensitive element.

9. An AR glasses, characterized in that, This includes the AR component as described in any one of claims 1-7 or the imaging method as described in claim 8.

Images