AR device control methods and AR devices

Abstract

This application discloses an AR device control method and an AR device. The AR device has a dimming module on the outer side of its optical module for adjusting light transmittance, dividing the dimming module into multiple independently controllable areas. During use, the AR device can control the light transmittance of corresponding areas in the dimming module based on detected ambient light data. This reduces the impact of ambient light on the virtual image transmitted by the AR device, ensuring that the ratio between the brightness of the external environment perceived by the human eye and the brightness of the virtual image is within a preset suitable range. Consequently, when ambient light and light from the display module used to transmit the virtual image enter the eye, the blended image perceived by the human eye has distinct brightness levels, thus improving the clarity of the blended image, ensuring that the user obtains more detailed information, and enhancing the user experience.

Description

Technical Field

[0001] This application relates to the field of terminals, and more particularly to an AR device control method and an AR device. Background Technology

[0002] Augmented Reality (AR) devices can transmit light to ensure that users can see the real environment outside, while also projecting light from virtual images into the user's eyes. This allows users to see virtual images while viewing the real environment. The mutual supplementation and enhancement of these two types of information creates a fused image in the user's mind, where the virtual image and the real environment are superimposed.

[0003] Ensuring that users can clearly see both the real environment and the virtual images while using AR devices, and thus providing a clearer and more integrated visual experience, is a problem that urgently needs to be solved. Summary of the Invention

[0004] This application provides an AR device control method and an AR device. The AR device has a dimming module on the outer side of its optical module for adjusting transmittance, dividing the dimming module into multiple independently controllable areas. The AR device can control the transmittance of corresponding areas in the dimming module based on detected ambient light data, thereby reducing the impact of external ambient light on the virtual image transmitted by the AR device. This results in clearer brightness and darkness levels in the blended image perceived by the human eye, improving the clarity of the blended image, ensuring users obtain more detailed information, and enhancing the user experience.

[0005] In a first aspect, this application provides an AR device control method. The method is applied to an AR device, which includes a dimming module and an optical module. The dimming module covers the side of the optical module furthest from the human eye. The optical module is used to project ambient light passing through the dimming module onto the human eye. The method includes: the AR device setting the transmittance of a first region in the dimming module to a first transmittance (the first transmittance may be a default value set by the AR device); the AR device detecting a first ambient light brightness; the first region completely covering the area of ​​the optical module used to project virtual images; the AR device detecting a second ambient light brightness; adjusting the transmittance of the first region to a second transmittance, where the second ambient light brightness is greater than the first ambient light brightness and the second transmittance is less than the first transmittance; and the AR device projecting light from the virtual image not affected by the dimming module onto the human eye through the optical module.

[0006] After implementing the method provided in the first aspect, the AR device can reduce the transmittance of the first area, which affects the presentation of the virtual image, in the dimming module when it detects that the ambient light brightness has increased. This reduces the impact of the ambient light passing through this area on the virtual image, making the virtual image in the fused image perceived by the human eye clearer and with distinct light and dark levels. This improves the clarity of the fused image, ensures that users can obtain more detailed information, and enhances the user experience.

[0007] In conjunction with the method described in the first aspect, the greater the brightness of the second ambient light, the smaller the second transmittance.

[0008] Thus, the greater the increase in ambient light brightness, the smaller the corresponding adjusted transmittance, which can block ambient light to a greater extent and effectively avoid the influence of ambient light transmitted through the first area of ​​the dimming module on the virtual image.

[0009] In conjunction with the method described in the first aspect, the dimming module further includes a second region, which is a region in the dimming module that is different from the first region; after the AR device detects the second ambient light brightness, the method further includes: adjusting the transmittance of the second region from a third transmittance (the third transmittance can be the default value of the AR device, which can be equal to or not equal to the first transmittance) to a fourth transmittance, wherein the fourth transmittance is less than the third transmittance.

[0010] In this way, when the AR device detects that the ambient light is getting brighter, in addition to reducing the light transmittance of the first area, which affects the virtual image presentation effect in the dimming module, it can also reduce all or part of the areas other than the first area. This not only avoids strong light from the external environment from hurting the user's eyes, but also makes the brightness of the external environment and the virtual image more uniform and the image more blended in the user's perception of the blended image.

[0011] In conjunction with the method described in the first aspect, the fourth transmittance is greater than or equal to the second transmittance.

[0012] This is because the second region is not used to transmit virtual images like the first region. Therefore, the transmittance of the second region does not need to be lower than that of the first region; it only needs to be greater than or equal to the second transmittance. When it is greater than, the user can see the external environment clearly. When it is equal to, the brightness of the first and second regions in the blended image is more uniform.

[0013] In conjunction with the method described in the first aspect, after adjusting the light transmittance of the second region from the third light transmittance to the fourth light transmittance, the method further includes: the AR device detects the third ambient light brightness, adjusts the light transmittance of the second region to the fifth light transmittance, and maintains the light transmittance of the first region at the second light transmittance, wherein the third ambient light brightness is greater than the second ambient light brightness, and the fifth light transmittance is less than the fourth light transmittance.

[0014] In this way, when the ambient light brightens for the first time, the transmittance of the first area can be adjusted to a lower value in one step. Then, when the AR device detects that the ambient light becomes stronger, it does not need to adjust the transmittance of the first area, but only reduces the transmittance of the second area. This not only reduces the number of transmittance adjustments and the power consumption of the AR device, but also prevents users from being hurt by ambient light when looking at the outside environment.

[0015] In conjunction with the method described in the first aspect, the first ambient light brightness and the second ambient light brightness are the ambient light brightness of the first region. The dimming module also includes a second region, which is a region in the dimming module that is different from the first region. The method further includes: the AR device setting the transmittance of the second region to a third transmittance, and the AR device detecting the third ambient light brightness of the second region; the AR device detecting the fourth ambient light brightness of the second region, and adjusting the transmittance of the second region to the fourth transmittance; the fourth ambient light brightness is greater than the third ambient light brightness, and the fourth transmittance is less than the third transmittance.

[0016] In this way, when the dimming module includes a first area and a second area, the AR device can adjust the transmittance of the first area based on the change in ambient light in the first area, and adjust the transmittance of the second area based on the change in ambient light in the second area. By adjusting the transmittance of the corresponding area according to the ambient light of each area, the purpose of precise zonal dimming can be achieved.

[0017] In conjunction with the method described in the first aspect, when the fourth ambient light brightness and the second ambient light brightness are the same, the fourth transmittance is greater than or equal to the second transmittance.

[0018] This is because the second region is not used to transmit virtual images like the first region. Therefore, the transmittance of the second region does not need to be lower than that of the first region; it only needs to be greater than or equal to the second transmittance. When it is greater than, the user can see the external environment clearly. When it is equal to, the brightness of the first and second regions in the blended image is more uniform.

[0019] In conjunction with the method described in the first aspect, the dimming module further includes a third region, which is different from the first region and the second region. The method further includes: the AR device setting the transmittance of the third region to a fifth transmittance, and the AR device detecting a fifth ambient light brightness at the second region; the AR device detecting a sixth ambient light brightness at the second region, and adjusting the transmittance of the third region to a sixth transmittance; the sixth ambient light brightness is greater than the fifth ambient light brightness, and the sixth transmittance is less than the fifth transmittance.

[0020] In this way, the dimming module can be divided into more areas as needed, and the transmittance of the corresponding area can be adjusted according to the ambient light of each area, so as to achieve the purpose of precise zone dimming.

[0021] In conjunction with the method described in the first aspect, at the same time, the brightness of the sixth ambient light is greater than that of the fourth ambient light, and the transmittance of the sixth light is less than that of the fourth light.

[0022] In this way, when the illumination of different areas in the dimming module varies due to sunlight / light, by dividing the module into more areas and adjusting the transmittance of the corresponding areas according to the ambient light in each area, the goal of precise zone dimming can be achieved, reducing dimming errors.

[0023] In conjunction with the method described in the first aspect, the method further includes: after the AR device detects that a video application is running, adjusting the transmittance of the first region to a seventh transmittance.

[0024] In this way, in addition to adjusting the transmittance of the dimming module according to changes in ambient light, AR devices can also adjust the transmittance according to the scene of the AR device. For example, when the AR device is running a video application and is in a movie-watching scene, the transmittance of the first area can be reduced to bring users a more immersive movie-watching experience.

[0025] In conjunction with the method described in the first aspect, after the AR device detects the second ambient light brightness, the method further includes: the AR device increasing the brightness of the light projecting the virtual image through the optical module.

[0026] In this way, in addition to adjusting the transmittance of the dimming module according to changes in ambient light, AR devices can also improve the display brightness of virtual images. These multi-faceted adjustments can further ensure that the ratio of the brightness of the virtual image to the brightness of the ambient light is within an appropriate range, making the blended image perceived by the user clearer.

[0027] In conjunction with the method described in the first aspect, after adjusting the transmittance of the first region to the second transmittance, the method further includes: the AR device detecting a seventh ambient light brightness, adjusting the transmittance of the first region to an eighth transmittance, wherein the seventh ambient light brightness is less than the second ambient light brightness, and the eighth transmittance is greater than the second transmittance.

[0028] In this way, in addition to reducing the transmittance when the ambient light brightness increases, the AR device can also increase the transmittance when the ambient light brightness decreases, ensuring that the user can see the external environment clearly.

[0029] In conjunction with the method described in the first aspect, before adjusting the light transmittance of the first region to the second light transmittance, the method further includes: the AR device receiving a first operation.

[0030] In this way, users can choose whether to enable the automatic dimming function according to their personal needs. When the AR device receives the corresponding user operation, the automatic dimming function will be enabled.

[0031] Secondly, this application provides an AR device, comprising: a processor, an ambient light sensor, a dimming module, and an optical module; the optical module is used to project ambient light passing through the dimming module onto a human eye; the processor is used to set the transmittance of a first region in the dimming module to a first transmittance, the first region completely covering the area in the optical module used for projecting virtual images; the ambient light sensor is used to detect a first ambient light brightness; the ambient light sensor is also used to detect a second ambient light brightness; the processor is also used to adjust the transmittance of the first region to a second transmittance, the second ambient light brightness being greater than the first ambient light brightness, and the second transmittance being less than the first transmittance; the optical module is also used to project light from a virtual image not processed by the dimming module onto a human eye.

[0032] When the AR device provided in the second aspect is used, the AR device can reduce the transmittance of the first area, which affects the presentation effect of the virtual image, in the dimming module when it detects that the ambient light brightness has increased. This reduces the impact of the ambient light passing through this area on the virtual image, making the virtual image in the fused image perceived by the human eye clearer and with distinct light and dark levels. This improves the clarity of the fused image, ensures that the user can obtain more detailed information, and enhances the user experience.

[0033] Thirdly, this application provides an AR device comprising: a processor, one or more memories, a dimming module, and an optical module; the dimming module covers the side of the optical module away from the human eye; the one or more memories are coupled to the one or more processors; the one or more memories are used to store computer program code, the computer program code including computer instructions; when the one or more processors execute the computer instructions, the electronic device performs the method as described in any of the first aspects.

[0034] Fourthly, this application provides a computer storage medium including computer instructions that, when executed on an AR device, cause the AR device to perform the method described in any of the first aspects.

[0035] Fifthly, this application provides a computer program product that, when run on a computer, causes the computer to perform the method described in any of the first aspects. Attached Figure Description

[0036] Figure 1 A schematic diagram of an AR device structure provided in an embodiment of this application;

[0037] Figure 2 This is a schematic diagram of another AR device structure provided in an embodiment of this application;

[0038] Figure 3 A schematic diagram of a set of dimming module components provided in the embodiments of this application;

[0039] Figure 4 This is a schematic flowchart of an AR device control method provided in an embodiment of this application. Detailed Implementation

[0040] The technical solutions in the embodiments of this application will be clearly and thoroughly described below with reference to the accompanying drawings. In the description of the embodiments of this application, unless otherwise stated, " / " means "or," for example, A / B can mean A or B; the word "and / or" in the text is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone.

[0041] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as implying or suggesting relative importance or implicitly indicating the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature, and in the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more.

[0042] In this application, the reference to "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a mutually exclusive, independent, or alternative embodiment. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described in this application can be combined with other embodiments.

[0043] The term "user interface (UI)" used in the following embodiments of this application refers to the medium interface through which an application or operating system interacts and exchanges information with the user. It realizes the conversion between the internal form of information and the form that the user can accept. The user interface is source code written in a specific computer language such as Java or Extensible Markup Language (XML). The interface source code is parsed and rendered on the electronic device, ultimately presenting content that the user can recognize. A common form of user interface is the graphical user interface (GUI), which refers to a user interface related to computer operation displayed graphically. It can be visible interface elements such as text, icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, and widgets displayed on the screen of an electronic device.

[0044] First, combined Figure 1 The basic structure of the AR device shown is used to briefly introduce the basic functions of the AR device.

[0045] like Figure 1 As shown, AR devices mainly include: a camera module, a control module, a display module, and an optical module. The display module and the optical module together form an optical display system. Furthermore, an AR device's optical display system typically contains two sets, one for each of the user's eyes. Figure 1 Only one set is shown as an example, but since the two sets of optical display systems are similar, the other set of optical display systems will not be introduced for the time being.

[0046] The camera module is mainly used to capture images so that the control module can identify the current environment and reconstruct the three-dimensional world, or to identify user-inputted interactive gestures, or for everyday photography and video recording.

[0047] The control module is also used to control the optical display system to transmit relevant information about the virtual image to the user. Specifically, this includes controlling the display module to display the virtual image and emitting the light emitted when displaying the virtual image to the optical module, so that the optical module can refract this light to the user's eyes, thereby allowing the user to perceive the virtual image without having to see the display module, but only by looking at the optical module.

[0048] AR devices also have see-through capabilities. Specifically, the optical module can transmit ambient light to the user's eyes, allowing them to see the real environment. Because the optical module needs to transmit ambient light to ensure the user can see the real environment, the display module needs to be placed in a location that does not interfere with the user's reception of ambient light, such as placing it to the side of the optical module.

[0049] When both ambient light and light from the display module enter the human eye simultaneously, the user's mind will perceive a fused image created by superimposing the real environment and the virtual image, thus enabling the AR device to provide an enhanced display effect on top of the real environment.

[0050] Because AR devices need to deliver virtual images to users while ensuring that users see the real environment, the final effect of the blended image perceived by the user is affected by both the brightness of the virtual image and the brightness of the ambient light. The brightness of the virtual image is mainly determined by the light reflected from the display module by the optical module, while the ambient light brightness is mainly determined by the light from the external environment (e.g., indoor or outdoor environment) transmitted through the optical module. Since the ambient light and the light from the display module overlap at the optical module before entering the human eye, the superposition of ambient light with the light from the display module will affect the final virtual image presented to the human eye. Specifically, when the ambient light is too bright, the brightness of the virtual image perceived by the user is too low compared to the brightness of the real environment, making it difficult for the user to see the virtual image clearly, thus resulting in a poor blended image perceived in the user's mind.

[0051] To improve the imaging effect of AR devices, one approach is to overlay a light shield on the outside of the optical module. The light shield typically has a constant transmittance. When users use AR devices outdoors or in bright ambient light, the light shield blocks external light, ensuring that the brightness of the virtual image perceived by the user is not too low relative to the ambient light, thus allowing the virtual image to be clearly seen in the resulting fused image. However, when users use AR devices indoors or in low ambient light, a large portion of the light entering the light shield is absorbed, resulting in a lower perceived brightness. This can make it difficult for users to see the real environment clearly, potentially leading to safety issues such as bumps and knocks. On the other hand, when the ambient light is too bright, the brightness of the virtual image can be increased. Although this method can ensure that the brightness of the virtual image perceived by the user is not too low compared to the brightness of the external environment, thus making the virtual image visible in the superimposed fused image, it will increase the power consumption of the AR device. Furthermore, reducing the impact of ambient light by increasing the brightness of the virtual image is limited. When too much ambient light passes through the optical module, this ambient light, after being superimposed on the light reflected by the optical module and the display module, makes the dark areas in the fused image formed by the superimposed virtual image and the real scene brighter, while the bright areas are limited by the maximum grayscale level and will not become brighter. In other words, the fused image lacks brightness and darkness levels and still appears blurry.

[0052] To address the aforementioned issues, this application provides an AR device control method and an AR device. The method includes: setting a dimming module for adjusting transmittance outside the optical module in the AR device, dividing the dimming module into multiple independently controllable areas. During AR device use, the AR device can control the transmittance of corresponding areas in the dimming module based on detected ambient light data, thereby reducing the impact of external ambient light on the virtual image transmitted by the AR device. This ensures that the ratio between the brightness of the external environment perceived by the human eye and the brightness of the virtual image is within a suitable range, resulting in clear distinction between light and dark levels in the blended image perceived by the human eye after the ambient light and the light from the display module used to transmit the virtual image enter the eye. This improves the clarity of the blended image, ensuring that the user obtains detailed information from the virtual image and the real environment, thus enhancing the user experience.

[0053] Next, we will introduce the form, structure, and control method of the AR device involved in this application one by one.

[0054] In this embodiment of the application, the AR device may be equipped with Alternatively, portable terminal devices with other operating systems, such as AR devices, can take the form of glasses, helmets, etc. The embodiments of this application do not limit the specific form of AR devices.

[0055] refer to Figure 2 , Figure 2 An exemplary schematic diagram of the AR device structure provided in this application is shown.

[0056] like Figure 2 As shown, the top view of the AR device and the front view of the optical module, an important component of the AR device, are shown respectively.

[0057] AR devices mainly include, but are not limited to: control modules, camera modules, display modules, optical modules, dimming modules, and ambient light sensors. The display module, optical module, and dimming module can constitute an optical display system. Furthermore, the optical display system typically includes two sets, each corresponding to one of the two eyes. Since the two sets of optical display systems are similar, therefore... Figure 2 Only one set is shown as an example; the other set of optical display systems will not be introduced here.

[0058] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the AR device. In other embodiments of this application, the AR device may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. Figure 2 The components presented can be implemented in hardware, software, or a combination of both.

[0059] The control module may include one or more processing units, such as an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural network processing unit (NPU). Different processing units may be independent devices or integrated into one or more processors. The controller may serve as the neural center and command center of the AR device. The controller can generate operation control signals based on instruction opcodes and timing signals to control instruction fetching and execution. The control module may also include a memory for storing instructions and data. In some embodiments, the memory in the control module is a cache memory. This memory can store instructions or data that the control module has just used or that is used repeatedly. If the control module needs to reuse an instruction or data, it can directly retrieve it from the memory. This avoids repeated access, reduces the waiting time of the control module, and thus improves system efficiency.

[0060] In some embodiments, the control module may include one or more interfaces. Interfaces may include inter-integrated circuit (I2C) interfaces, inter-integrated circuit sound (I2S) interfaces, pulse code modulation (PCM) interfaces, universal asynchronous receiver / transmitter (UART) interfaces, mobile industry processor interfaces (MIPI), general-purpose input / output (GPIO) interfaces, subscriber identity module (SIM) interfaces, and / or universal serial bus (USB) interfaces, etc. The USB interface is a USB standard-compliant interface, specifically a Mini USB interface, Micro USB interface, USB Type-C interface, etc. The USB interface can be used to connect a charger to charge the AR device, and can also be used for data transfer between the AR device and peripheral devices. It can also be used to connect headphones for audio playback. This interface can also be used to connect other electronic devices, such as mobile phones and computers.

[0061] In this embodiment, the control module can be used to receive ambient light data sent by the ambient light sensor and read dimming rules from the memory. Then, based on the ambient light brightness, it adaptively adjusts the transmittance of each area of ​​the dimming module according to the dimming rules to reduce the impact of ambient light brightness on the virtual image. This makes the fused image that the optical module finally transmits to the human eye clearer and more distinct in brightness and darkness.

[0062] Camera modules can provide shooting capabilities for AR devices. Specifically, camera modules include, but are not limited to: one or more cameras, an Image Signal Processor (ISP), a digital signal processor, a video codec, and an NPU (neural-network, NN) computing processor.

[0063] An ISP (Image Signal Processor) processes data fed back from the camera. For example, when taking a picture, the shutter is opened, and light is transmitted through the lens to the camera's image sensor. The light signal is converted into an electrical signal, which is then transmitted to the ISP for processing, transforming it into a visible image. The ISP can also perform algorithmic optimizations on image noise, brightness, and skin tone. It can also optimize parameters such as exposure and color temperature for the shooting scene. In some embodiments, the ISP can be integrated into the camera itself.

[0064] A camera is used to capture still images or videos. An object is projected onto a photosensitive element through a lens, generating an optical image. The photosensitive element can be a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, which is then passed to an ISP (Internet Service Provider) for conversion into a digital image signal. The ISP outputs the digital image signal to a DSP (Digital Signal Processor) for processing. The DSP converts the digital image signal into image signals in standard formats such as RGB and YUV. In some embodiments, an AR device may include one or N cameras, where N is a positive integer greater than 1.

[0065] Digital signal processors (DSPs) are used to process digital signals. Besides digital image signals, they can also process other digital signals. For example, when an AR device selects a frequency, a DSP can perform a Fourier transform on the frequency energy.

[0066] Video codecs are used to compress or decompress digital video. AR devices can support one or more video codecs. This allows AR devices to play or record videos in various encoding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0067] NPU stands for Neural Network (NN) Computing Processor. By borrowing the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, it can rapidly process input information and continuously learn on its own. NPUs can enable intelligent cognitive applications in AR devices, such as interactive gesture recognition, image recognition, facial recognition, speech recognition, and text understanding.

[0068] The display module can also be a miniature display screen, an optical engine, a projector, etc. The miniature display screen is used to display miniature virtual images. When the miniature virtual image is displayed on the miniature display screen, the light emitted can be refracted by the optical module from the area of ​​the virtual image within the optical module and directed to the user's eye, allowing the user to see the virtual image within that area of ​​the optical module. In other words, the miniature display screen can project a miniature virtual image onto the area of ​​the virtual image within the optical module, allowing the user to see an enlarged virtual image superimposed on the real environment.

[0069] The microdisplay includes a display panel. The display panel can be manufactured using liquid crystal display (LCD), liquid crystal on silicon (LCOS), digital light processing (DLP), light emitting diode (LED), laser beam scanning (LBS) display, organic light-emitting diode (OLED), active-matrix organic light-emitting diode (AMOLED), flexible light-emitting diode (FLED), miniled, microled, micro-oled, quantum dot light-emitting diodes (QLED), etc. In some embodiments, the AR device may include one or N displays, where N is a positive integer greater than 1.

[0070] The optical module is positioned directly in front of the human eye. The optical module can be, for example, a prism, a curved mirror, or a reflective grating. The entire area of ​​the optical module (including...) Figure 2 The virtual image area shown can be used to transmit ambient light, allowing the user to see the real environment. The virtual image area refers to the region within the optical module that refracts light from the display module to the user's eye. The user receives the refracted light from this area and perceives the virtual image; in other words, this area can display a virtual image. The virtual image area is typically smaller than or equal to the entire area of ​​the optical module. In short, the optical module allows the user to receive both transmitted ambient light and refracted light from the display module. The superposition of these two types of light, transmitted to the eye, allows the user to perceive a fused image of the real environment and the virtual image.

[0071] The dimming module is positioned outside the optical module, on the side furthest from the viewer's eye. The dimming module is a component that completely or partially covers the outside of the optical module; that is, the size of the dimming module is smaller than or equal to the size of the optical module, and the dimming module at least covers the area of ​​the virtual image within the optical module. The dimming module can be pre-configured according to the following... Figure 3 The mentioned zoning rules divide the area into multiple zones, and the transmittance of each zone can be controlled independently.

[0072] In one specific embodiment, the dimming module mainly includes a transmittance adjustment layer, and may also include a protective film / lens. The transmittance adjustment layer covers the outer side of the optical module (the side furthest from the eye). This layer can be a polymer liquid crystal material, such as an electrochromic (EC) film or a liquid crystal film. The transmittance adjustment layer can be pre-divided into multiple zones according to the zoning rules mentioned later. Each zone has an independent control electrode. By inputting different voltages to each control motor, each zone can be adjusted to a different transmittance. The protective film / lens covers the outer side of the transmittance adjustment layer (the side furthest from the eye) and serves to protect it from scratches.

[0073] An ambient light sensor is located on the outside of the AR device. This sensor detects ambient light data, specifically the numerical value of the ambient light brightness at the dimming module, or information about changes in ambient light brightness. Based on the detected ambient light data, the AR device can adaptively adjust the transmittance of different zones within the dimming module according to preset dimming rules.

[0074] In some implementations, the ambient light sensor can be used to detect ambient light data received by the entire dimming module. In other implementations, the ambient light sensor can also be used to detect ambient light data received by different areas within the dimming module. The ambient light sensor used to detect ambient light data in different areas can be a single ambient light sensor or multiple ambient light sensors respectively disposed on different zones.

[0075] The camera module, control module, and display module are all positioned in locations that do not interfere with the user's reception of ambient light from the optical module or the light from the display module, such as at the edge of the optical module. For further information on the functions of the camera module, control module, display module, and optical module, please refer to the previous section. Figure 1 This section mainly introduces the functions of the newly added modules in AR devices.

[0076] Figure 2The structure of the AR device shown is merely an example. The structure illustrated in the embodiments of this application does not constitute a specific limitation on the AR device. In other examples of this application, the AR device may also include more modules, such as memory, audio module, touch sensor, buttons, wireless communication module, etc. These modules can be implemented in hardware, software, or a combination of software and hardware.

[0077] The memory can be coupled to the control module, allowing the control module to read necessary content from the memory to provide relevant information to the user. For example, the memory can store dimming rules, which the control module can read and combine with data detected by the ambient light sensor to control the transmittance of the dimming module in different zones.

[0078] Specifically, the memory can include internal memory and external storage interfaces.

[0079] Internal memory may include one or more random access memory (RAM) and one or more non-volatile memory (NVM).

[0080] Random access memory (RAM) can include static random-access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), and double data rate synchronous dynamic random access memory (DDR SDRAM, such as fifth-generation DDR SDRAM, which is generally called DDR5 SDRAM). RAM can be directly read and written by the control module and can be used to store executable programs (such as machine instructions) of the operating system or other running programs, as well as user and application data.

[0081] Non-volatile memory can include disk storage devices and flash memory. Flash memory can be classified according to its operating principle, such as NOR flash, NAND flash, and 3D NAND flash; according to the level of its storage cells, such as single-level cell (SLC), multi-level cell (MLC), triple-level cell (TLC), and quad-level cell (QLC); and according to its storage specification, such as universal flash storage (UFS) and embedded multimedia card (eMMC). Non-volatile memory can also store executable programs and user and application data, which can be pre-loaded into random access memory for control modules to read and write directly.

[0082] An external memory interface can be used to connect to external non-volatile memory, expanding the storage capacity of AR devices. The external non-volatile memory communicates with the control module through the external memory interface to achieve data storage functionality. For example, music, video, and other files can be saved on the external non-volatile memory.

[0083] The audio module is used to convert digital audio information into analog audio signals for output, and also to convert analog audio input into digital audio signals. The audio module can also be used for encoding and decoding audio signals. In some embodiments, the audio module can be located within the control module, or some functional modules of the audio module can be located within the control module.

[0084] AR devices can achieve audio functions through audio modules, speakers, receivers, microphones, headphone jacks, and control modules, such as music playback and recording.

[0085] A loudspeaker, also known as a "speaker," is used to convert audio electrical signals into sound signals. AR devices can listen to music or make hands-free calls through loudspeakers.

[0086] The receiver, also known as the "earpiece," is used to convert audio electrical signals into sound signals. When an AR device answers a phone call or voice message, it can do so by bringing the receiver close to the user's ear.

[0087] A microphone, also known as a "voice transducer," is used to convert sound signals into electrical signals. When making a phone call or sending a voice message, the user speaks by bringing their mouth close to the microphone, inputting the sound signal into the microphone. AR devices can have at least one microphone. In some embodiments, AR devices can have two microphones, which, in addition to collecting sound signals, can also perform noise reduction. In other embodiments, AR devices can have three, four, or more microphones, enabling sound signal collection, noise reduction, sound source identification, and directional recording, among other functions.

[0088] The headphone jack is used to connect wired headphones. The headphone jack can be a USB 130 interface or a 3.5mm Open Mobile Terminal Platform (OMTP) standard interface, or a CTIA (Cellular Telecommunications Industry Association of the USA) standard interface.

[0089] A touch sensor, also known as a "touch panel," can be placed in a location easily accessible to the user, such as the side of an AR device. Touch sensors detect touch actions applied to or near them. They then transmit the detected touch actions to a control module to determine the type of touch event. For example, a display module can provide visual output related to the touch action, an audio module can provide auditory output, or a dimming module can provide dimming functionality.

[0090] Pressure sensors are used to sense pressure signals and convert them into electrical signals. In some embodiments, the pressure sensor can be placed in a location easily accessible to the user, such as the side of an AR device, or integrated into a button. There are many types of pressure sensors, such as resistive pressure sensors, inductive pressure sensors, and capacitive pressure sensors. A capacitive pressure sensor may consist of at least two parallel plates with conductive material. When force is applied to the pressure sensor, the capacitance between the electrodes changes. The AR device determines the pressure intensity based on the change in capacitance. When a press / operation is applied to a button, the AR device detects the intensity of the press / touch operation based on the pressure sensor. The AR device can also calculate the touch position based on the detection signal from the pressure sensor. In some embodiments, touch operations applied to the same press / touch position but with different press / touch operation intensities can correspond to different operation commands. For example, when a press / touch operation with an intensity less than a first pressure threshold is applied to a video application icon, a command to play the video is executed. When a touch operation with an intensity greater than or equal to the first pressure threshold is applied to a video application icon, a command to play the video at double speed is executed.

[0091] The buttons include a power button, volume buttons, and transmittance adjustment buttons. These buttons can be mechanical or touch-sensitive. The AR device can receive button input and generate key signal inputs related to preset settings and function control. The transmittance adjustment buttons, in particular, can be used to receive user input to control the transmittance of the dimming module in different zones.

[0092] The wireless communication function of AR devices can be achieved through antennas, wireless communication modules, modem processors, and baseband processors.

[0093] Antennas are used to transmit and receive electromagnetic wave signals. Each antenna in an AR device can be used to cover one or more communication frequency bands. Different antennas can also be reused to improve antenna utilization. For example, antennas can be reused as diversity antennas for a wireless local area network. In some other embodiments, antennas can be used in conjunction with tuning switches.

[0094] A modem processor may include a modulator and a demodulator. The modulator modulates a low-frequency baseband signal to be transmitted into a mid-to-high frequency signal. The demodulator demodulates a received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to a baseband processor for processing. After processing by the baseband processor, the low-frequency baseband signal is transmitted to an application processor. The application processor outputs sound signals through audio devices (not limited to speakers, receivers, etc.) or provides image or video information through a display module. In some embodiments, the modem processor may be a separate device. In other embodiments, the modem processor may be independent of the control module and housed within the same device as other functional modules.

[0095] Wireless communication modules can provide solutions for AR devices using various wireless communication technologies, including wireless local area networks (WLANs) (such as Wi-Fi), Bluetooth (BT), Global Navigation Satellite System (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR). A wireless communication module can be one or more devices integrating at least one communication processing module. The wireless communication module receives electromagnetic waves via an antenna, demodulates and filters the electromagnetic wave signals, and sends the processed signal to the control module. The wireless communication module can also receive signals to be transmitted from the control module, frequency-modulate and amplify them, and then convert them into electromagnetic waves for radiation via the antenna.

[0096] In some embodiments, the AR device's antenna and wireless communication module are coupled, enabling the AR device to communicate with networks and other devices via wireless communication technologies. These wireless communication technologies may include Global System for Mobile Communications (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Time-Division Code Division Multiple Access (TD-SCDMA), Long Term Evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technologies. The GNSS may include Global Positioning System (GPS), Global Navigation Satellite System (GLONASS), BeiDou Navigation Satellite System (BDS), Quasi-Zenith Satellite System (QZSS), and / or Satellite Based Augmentation Systems (SBAS).

[0097] Based on the form and structure of AR devices introduced above, the following section will introduce several dimming module partitioning rules.

[0098] As previously explained, a dimming module is a component that completely or partially covers the outside of an optical module, and the dimming module at least covers the area of ​​the virtual image within the optical module. To more comprehensively improve the imaging effect of the entire blended image and ensure clear brightness and contrast in the final blended image perceived by the user, this embodiment uses an example where the size of the dimming module completely covers the entire size of the optical module. Furthermore, an AR device typically has two dimming modules, each covering the outside of the two optical modules corresponding to the user's eyes. Since the structures of the two dimming modules are similar, the following description uses only one dimming module as an example to illustrate the partitioning rules. The partitioning rules for the other dimming module can also be found in the following description and will not be repeated here.

[0099] refer to Figure 3 , Figure 3 An exemplary schematic diagram of the three dimming mode components provided in this application is shown.

[0100] like Figure 3 As shown, dimming mode group partitioning rule 1, dimming mode group partitioning rule 2 and dimming mode group partitioning rule 3 are illustrated respectively.

[0101] (1) The dimming module group zoning rule 1 specifically includes: dividing the entire dimming module group used to present the external environment into region 1 and region 2. Among them, region 1 is greater than or equal to the region of the virtual image in the optical module corresponding to the region in the dimming module; and region 2 refers to all regions in the entire dimming module except for region 1.

[0102] As can be seen, in the entire blended image perceived by the user, since the user's main focus is usually on the virtual image, as long as the imaging effect of the virtual image is guaranteed, the imaging effect of the blended image can be greatly improved. Therefore, the dimming module can be divided into region 1 corresponding to the virtual image and other regions 2. In other words, the dimming module zoning rule 1 only considers the influence of ambient light from the outside world passing through the dimming module on the imaging effect of the virtual image in the optical module.

[0103] (2) The dimming mode group zoning rule 2 specifically includes: based on zoning rule 1, region 2 is further divided into upper and lower regions. That is, the entire dimming mode group is divided into regions corresponding to the virtual image (see reference). Figure 3 Region 1 in the middle) and the upper region (reference) Figure 3 Region 2-1 in the middle and the lower region (reference) Figure 3 (Region 2-2 in the text). This division is because when the user is wearing the AR device and is not looking up, the sunlight from the sky / the lights from the roof are above the AR device. This results in the upper part of the dimming module receiving more ambient light than the lower part of the dimming module. Therefore, further dividing the above-mentioned region 2 into upper region 2-1 and lower region 2-2 allows for more precise control of the light transmittance of the dimming module.

[0104] (3) The dimming mode group zoning rule 3 specifically includes: based on zoning rule 2, the upper region (region 2-1) and the lower region (region 2-2) are further divided into left and right regions respectively. That is, the entire dimming mode group is divided into regions corresponding to the virtual image (see reference). Figure 3 Area 1 in the middle), upper left area (reference) Figure 3 Areas 2-3 in the middle), and the upper right area (for reference). Figure 3 Areas 2-4 in the middle), lower left area (reference) Figure 3 Areas 2-5 in the middle and the lower right area (see reference) Figure 3(Regions 2-6 in the text). This division is because when a user is wearing the AR device and is not looking up, the sunlight from the sky / the lights from the roof may be located to the upper left / upper right of the AR device. This will cause the ambient light received by the dimming module to be different in the upper left, upper right, lower left, and lower right. Therefore, dividing the above-mentioned regions 2-1 and 2-2 into two smaller regions, left and right, allows for more precise control of the light transmittance of the dimming module.

[0105] As can be seen, the dimming module zoning rules 2 and 3 not only consider the impact of ambient light passing through the dimming module on the imaging effect of the virtual image, but also take into account the imaging effect of ambient light passing through the dimming module on the real scene when the user is wearing the AR device in different usage states. Specifically, in the overall blended image perceived by the user, in addition to primarily focusing on the virtual image, the user will occasionally pay attention to the real environment. Therefore, by ensuring the imaging effect of the virtual image while further ensuring the imaging effect of the external environment, the overall imaging effect of the blended image can be comprehensively improved.

[0106] In the embodiments of this application, region 1 may also be referred to as the first region, region 2 or any one or more of region 2-1, region 2-2, region 2-3... region 2-6 may also be referred to as the second region, and any one or more of region 2-1, region 2-2, region 2-3... region 2-6 may also be referred to as the third region.

[0107] Figure 3 The three dimming module grouping rules shown are merely examples and do not constitute a specific limitation on the dimming module grouping rules provided in this application. In other examples of this application, other usage states / application scenarios can also be considered, and the dimming module can be divided into more regions according to the corresponding rules. For example, the region 2 mentioned above can be divided into more smaller regions.

[0108] Next, combine Figure 4 The method flow shown will be used to describe in detail the AR control method provided in this application.

[0109] like Figure 4 As shown, the method includes the following steps:

[0110] S401 divides the dimming module into multiple zones according to preset zoning rules.

[0111] Specifically, before the AR device leaves the factory, developers need to pre-set zoning rules and then divide the dimming module in the AR device into multiple independent and controllable areas according to the preset zoning rules. This application embodiment does not limit the specific number of divided areas; the final number of areas may differ when different zoning rules are used. For details regarding the zoning rules used in this application, please refer to the zoning rules described above. Figure 3 The three partitioning rules shown in the example will not be elaborated here.

[0112] In one specific implementation, the process of dividing the dimming module into multiple regions includes, for example:

[0113] When the dimming module uses the electrochromic (EC) film described earlier, a conductive layer is incorporated within the EC. A commonly used conductive layer is indium tin oxide (ITO). By controlling the voltage applied to the ITO layers, different transmittances of the EC can be controlled. Therefore, to divide the EC into multiple independently controllable regions, a complete ITO layer needs to be cut (e.g., using laser cutting) into multiple ITO sub-regions. Each independent ITO sub-region is controlled by an independent circuit, typically leading to the periphery of the dimming module. Then, EC material is added to the ITO for encapsulation, ultimately forming a complete EC. However, this complete EC actually contains multiple independently controllable ITO sub-regions, meaning it has multiple independently controllable EC sub-regions. By inputting different voltages to the independent circuits of each ITO sub-region, the transmittance of the corresponding EC sub-region can be adjusted. It can be understood that the cut ITO sub-regions correspond to their respective EC sub-regions, which is equivalent to the previously described... Figure 3 The examples shown are region 1 and region 2.

[0114] S402, detects ambient light data.

[0115] Specifically, after the AR device is powered on, it can directly trigger the detection of ambient light data and then perform subsequent zone dimming operations. Alternatively, the AR device can trigger the detection of ambient light data and then perform subsequent zone dimming operations based on the user's first operation. Or, the AR device can trigger the detection of ambient light data and then perform subsequent zone dimming operations after displaying a virtual image. This application embodiment does not limit the conditions for triggering the detection of ambient light data.

[0116] Specifically, the ambient light data detection can be achieved using an ambient light sensor in the AR device. In one feasible implementation, the ambient light sensor can detect the ambient light data of the entire dimming module. In another feasible implementation, the ambient light sensor can detect the ambient light data of each zone within the dimming module separately. For example, when the dimming module is divided using the aforementioned zoning rule 1, the ambient light sensor can detect the ambient light data of the entire dimming module, or it can detect the ambient light data of only zone 1 within the dimming module. When the dimming module is divided using the aforementioned zoning rule 2 or zoning rule 3, the ambient light sensor can detect the ambient light data of each zone separately.

[0117] The ambient light data can take the form of information about changes in ambient light brightness or specific numerical values ​​of ambient light brightness. The ambient light data detected by the ambient light sensor can be reported to the control module of the AR device, allowing the control module to adaptively adjust the transmittance of the dimming module in real time based on the ambient light data.

[0118] Specifically, the change information of ambient light brightness can refer to the switching information between different ranges of ambient light brightness. For example, it can refer to the switching information between a high-brightness ambient light range, a low-brightness ambient light range, or a medium-brightness ambient light range. In this embodiment, the range of different brightness values ​​is not specifically limited.

[0119] S403 adjusts the transmittance of the dimming module in zones according to preset dimming rules based on the detected ambient light data.

[0120] Specifically, the AR device pre-stores dimming rules, which indicate the relationship between different ambient light data and the transmittance of each zone in the dimming module. After the AR device's control module receives the ambient light data reported by the ambient light sensor, it adjusts the transmittance of each zone in the dimming module according to the preset dimming rules based on the ambient light data.

[0121] The dimming rules could be, for example, as follows: when the ambient light brightness indicated by the ambient light data is detected to increase, the transmittance of the dimming module is reduced; when the ambient light brightness indicated by the ambient light data is detected to decrease, the transmittance of the dimming module is increased.

[0122] The ambient light data indicates that the ambient light brightness has increased or decreased. Specifically, this includes either an increase or decrease in the specific numerical value of the ambient light brightness, or an increase or decrease in the brightness range to which the ambient light brightness belongs.

[0123] Adjusting the light transmittance of the dimming module by lowering / raising it specifically involves independently adjusting each area within the dimming module.

[0124] (1) When the AR device adjusts the transmittance of each area based on the ambient light data of the entire dimming module, the specific dimming rules are as follows:

[0125] For example, when the ambient light of the entire dimming module is detected to increase from the first ambient light brightness to the second ambient light brightness, the transmittance of region 1 is reduced from the first transmittance to the second transmittance. At the same time, the transmittance of other regions (such as region 2, region 2-1... region 2-6) can also be reduced from the third transmittance to the fourth transmittance.

[0126] Specifically, the higher the second ambient light brightness, the lower the second transmittance. This allows for greater occlusion of ambient light even in brighter conditions, reducing its impact on the virtual image. The fourth transmittance is greater than or equal to the second transmittance. This is because regions 2, 2-1… 2-6 do not display virtual images like region 1. Therefore, the transmittance of these regions does not need to be lower than that of region 1, ensuring that users can clearly see the external environment.

[0127] For example, when the ambient light of the entire dimming module is detected to increase from the second ambient light brightness to the third ambient light brightness, the transmittance of other areas outside of area 1 (e.g., area 2, area 2-1…area 2-6, any one or more of these areas) is reduced from the fourth transmittance to the fifth transmittance. Alternatively, area 1 can maintain its second transmittance, or its transmittance can be further reduced. The reason for maintaining the ambient light brightness of area 1 is that area 1 has already been adjusted to a lower value (i.e., the second transmittance) in the first step. This one-step operation ensures that the virtual image is not affected by external ambient light. Therefore, when the ambient light brightens a second or subsequent time, there is no need to further reduce the transmittance of area 1; only the transmittance of area 2 needs to be reduced to ensure that the user's eyes are not hurt when looking at the external environment.

[0128] (2) When the AR device adjusts the transmittance of each area based on the ambient light data of each area in the dimming module, the specific dimming rules are as follows:

[0129] For example, when the ambient light in region 1 is detected to increase from a first ambient light brightness to a second ambient light brightness, the transmittance of region 1 is reduced from the first transmittance to the second transmittance. When the ambient light in other regions (e.g., region 2, region 2-1… region 2-6) is detected to increase from a third ambient light brightness to a fourth ambient light brightness, the transmittance of the corresponding region is reduced from the third transmittance to the fourth transmittance. When the ambient light in other regions (e.g., region 2-1… region 2-6) is detected to increase from a fifth ambient light brightness to a sixth ambient light brightness, the transmittance of the corresponding region is reduced from the fifth transmittance to the sixth transmittance.

[0130] Specifically, the greater the brightness of the second ambient light, the smaller the second transmittance. This allows for greater occlusion of ambient light even when the ambient light brightness is high, thus reducing the impact of ambient light on the virtual image.

[0131] Specifically, when the fourth ambient light brightness is the same as the second ambient light brightness, the fourth transmittance is greater than or equal to the second transmittance. This is because regions 2, 2-1... 2-6 do not display virtual images like region 1, so the transmittance of these regions does not need to be lower than that of region 1, thus ensuring that users can clearly see the external environment.

[0132] The above only describes the dimming rules when the ambient light brightness increases. When the ambient light brightness decreases, for example, when the ambient light brightness decreases from the second ambient light brightness to the seventh ambient light brightness, the transmittance of area 1 can be increased, for example, adjusted to the eighth transmittance, which is greater than the second transmittance. Alternatively, the transmittance of other areas outside area 1 can also be increased, but these will not be elaborated here.

[0133] Next, we will use a specific example to introduce the dimming rules for AR devices that adjust the transmittance of each area according to the ambient light when using different zoning rules.

[0134] (1) When the dimming module is divided into region 1 and region 2 according to the partitioning rule 1 described above.

[0135] If the ambient light data indicates the ambient light brightness received by the entire dimming module, the dimming rules can be as follows: when the ambient light data indicates that the ambient light received by the entire dimming module is in a low brightness range, the transmittance of the entire dimming module is the first value (e.g., 75%); when the ambient light data indicates that the ambient light received by the entire dimming module is in a medium brightness range, the transmittance of region 1 is the second value (e.g., 15%), and the transmittance of region 2 remains unchanged at the first value; when the ambient light data indicates that the entire dimming module is in a high brightness range, the transmittance of region 1 is the second value (e.g., 15%), and the transmittance of region 2 is the third value (e.g., 30%).

[0136] If the ambient light data indicates the ambient light brightness received by region 1 and region 2 respectively, then the dimming rule can be as follows: When the ambient light data indicates that the ambient light received by region 1 is within the range of low brightness, medium brightness, and high brightness, then the transmittance of region 1 corresponds to the fourth, fifth, and sixth values ​​respectively; when the ambient light data indicates that the ambient light received by region 2 is within the range of low brightness, medium brightness, and high brightness, then the transmittance of region 2 corresponds to the seventh, eighth, and ninth values ​​respectively. Specifically, the fourth value is greater than the fifth value, the fifth value is greater than the sixth value, the seventh value is greater than the eighth value, and the eighth value is greater than the ninth value; furthermore, the fourth value is less than the seventh value, the fifth value is less than the eighth value, and the sixth value is less than the ninth value.

[0137] In this way, even when ambient light levels are high, the transmittance of the dimming module can be adjusted. Specifically, the transmittance of areas 1 and 2 corresponding to the virtual image can be reduced to varying degrees. This reduces the impact of ambient light on the virtual image while ensuring that the user sees a suitable level of brightness in the real environment to avoid eye strain. For example, this ensures that AR devices can provide users with a clearer and more immersive viewing experience.

[0138] (2) When the dimming module is divided into region 1, region 2-1 and region 2-2 according to the partitioning rule 2 described above, the dimming rule can be: when the ambient light data indicates that the ambient light brightness received by region 1, region 2-1 or region 2-2 is higher, the transmittance of these regions is lower. Furthermore, when the ambient light brightness of region 1 is at the same brightness value or within the same range as the ambient light brightness of region 2-1 / region 2-2, the transmittance of region 1 is less than the transmittance of region 2-1 / region 2-2.

[0139] In this way, precise zoning control of the light transmittance of different areas can solve the problem that when users use AR devices in different usage states (as mentioned above, whether they are looking up at the sunlight or lamplight), the ambient light brightness received by different zones in the dimming module is different, resulting in uneven distribution of light projected from the dimming module and optical module to the human eye, and thus the problem of uneven distribution of brightness and darkness in the final blended image perceived by the user.

[0140] (3) When the dimming module is divided into region 1, region 2-3, region 2-4, region 2-5 and region 2-6 according to the partitioning rule 2 described above, the dimming rule can be: when the ambient light data indicates the ambient light brightness received by region 1, region 2-3, region 2-4, region 2-5 or region 2-6, the transmittance of these regions is lower. Furthermore, when the ambient light brightness of region 1 is at the same brightness value or range as that of region 2-3, region 2-4, region 2-5 or region 2-6, the transmittance of region 1 is less than that of region 2-3, region 2-4, region 2-5 or region 2-6.

[0141] In this way, more precise zoning control of the light transmittance of different areas can solve the problem that when users use AR devices in different usage states (as mentioned above, whether they are tilting their heads to the left or right to face sunlight or lights), the ambient light brightness received by different zones in the dimming module is different, resulting in uneven distribution of light projected from the dimming module and optical module to the human eye, and thus the problem of uneven distribution of brightness and darkness in the final blended image perceived by the user.

[0142] In this embodiment, S402-S403 are continuously executed steps, that is, the AR device can control the ambient light sensor to periodically detect ambient light data. Whenever the ambient light data changes and meets the change conditions, the AR device will adjust the transmittance of the dimming module according to the preset dimming rules based on the changed ambient light data.

[0143] In this embodiment, besides adjusting the transmittance based on ambient light data, the AR device can also adjust the transmittance according to the application scenario, which includes, but is not limited to, movie-watching scenarios, game-watching scenarios, etc. Specifically, when the AR device detects that a video application is running, i.e., it is in a movie-watching scenario, it can reduce the transmittance of region 1 (for example, reduce the transmittance of the first region to a seventh transmittance, which is a lower value, such as less than the second transmittance), or it can also reduce the transmittance of other regions besides region 1. Alternatively, it can also adjust the transmittance of the dimming module in sections based on the dimming operation received on the touchpad or buttons.

[0144] During the execution of S401-S403 by the AR device, the AR device may also execute S404 in any step after S401, or continue to execute S404 after S401.

[0145] S404 controls the display module to display virtual images.

[0146] Specifically, after the AR device is powered on, it can control the display module to display virtual images according to user operations. As described above regarding the functions of the AR device, when the display module displays virtual images, the light emitted by the display module is projected into the optical module. The area of ​​the virtual image in the optical module can refract this light into the user's eyes, allowing the user to see the virtual image by looking at the optical module.

[0147] In one possible implementation of this application, the AR device can not only adjust the transmittance of the dimming module according to ambient light data, but also adjust the display brightness of the virtual image. For example, when the ambient light becomes brighter, the display brightness of the virtual image is increased, and vice versa.

[0148] As can be seen, by adopting the dimming rules provided in this application, the contradiction between the light transmittance of AR devices and the brightness of virtual image displays can be resolved to a certain extent during the use of AR devices. Specifically, the light transmittance of the dimming module is precisely adjusted by partitioning, which ensures that users can see the real environment outside, while reducing the impact of ambient light on the brightness of virtual images, thereby ensuring that the user ultimately perceives a clear and distinct blended image with distinct light and dark layers.

[0149] It should be understood that the steps in the above-described method embodiments provided in this application can be implemented by integrated logic circuits in the processor hardware or by instructions in software form. The method steps disclosed in the embodiments of this application can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules in the processor.

[0150] This application also provides an electronic device that may include a memory and a processor. The memory may be used to store a computer program; the processor may be used to invoke the computer program in the memory to cause the electronic device to perform the method in any of the above embodiments.

[0151] This application also provides a chip system including at least one processor for implementing the functions involved in the methods performed by the electronic device in any of the above embodiments.

[0152] In one possible design, the chip system also includes a memory for storing program instructions and data, which may be located within or outside the processor.

[0153] The chip system can consist of chips or include chips and other discrete components.

[0154] Optionally, the chip system may contain one or more processors. These processors can be implemented in hardware or software. When implemented in hardware, the processor can be a logic circuit, an integrated circuit, etc. When implemented in software, the processor can be a general-purpose processor, implemented by reading software code stored in memory.

[0155] Optionally, the chip system may contain one or more memories. The memory may be integrated with the processor or disposed separately from it; this application embodiment does not limit this. For example, the memory may be a non-transient processor, such as a read-only memory (ROM), which may be integrated with the processor on the same chip or disposed separately on different chips. This application embodiment does not specifically limit the type of memory or the arrangement of the memory and processor.

[0156] For example, the chip system may be a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on chip (SoC), a central processor unit (CPU), a network processor (NP), a digital signal processor (DSP), a micro controller unit (MCU), a programmable logic device (PLD), or other integrated chips.

[0157] This application also provides a computer program product comprising: a computer program (also referred to as code or instructions) that, when run, causes a computer to perform the method executed by the electronic device in any of the above embodiments.

[0158] This application also provides a computer-readable storage medium storing a computer program (also referred to as code or instructions). When the computer program is run, it causes the computer to perform the method executed by the electronic device in any of the above embodiments.

[0159] The various embodiments of this application can be combined arbitrarily to achieve different technical effects.

[0160] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive).

[0161] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This program can be stored in a computer-readable storage medium, and when executed, it can include the processes described in the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM or random access memory (RAM), magnetic disks, or optical disks.

[0162] In summary, the above description is merely an embodiment of the technical solution of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made according to the disclosure of the present invention should be included within the scope of protection of the present invention.

Claims

1. An AR device control method, characterized in that, The method is applied to an AR device, which includes a dimming module and an optical module. The dimming module covers the side of the optical module away from the human eye. The optical module is used to project ambient light passing through the dimming module onto the human eye. The dimming module includes a first region, a fourth region, and a fifth region. The area corresponding to the first region in the optical module is used to project a virtual image, and the area of ​​the first region is larger than the area of ​​the area in the optical module used to project the virtual image. The areas corresponding to the fourth and fifth regions in the optical module are not used to project virtual images. The areas corresponding to the first, fourth, and fifth regions in the optical module are all used to present a real image. The fourth and fifth regions are arranged around the first region, with the fourth region above the first region and the fifth region below the first region. The fourth and fifth regions do not overlap with the first region. Each of the first, fourth, and fifth regions of the dimming module contains an independent conductive layer. The magnitude of the voltage received by the conductive layer determines the light transmittance of the corresponding area of ​​the conductive layer. The method includes: The AR device detects the first ambient light brightness of the first area and sets the transmittance of the first area in the dimming module to the first transmittance. The AR device detects the third ambient light brightness of the fourth area and sets the transmittance of the fourth area to the third transmittance. The AR device detects the fifth ambient light brightness of the fifth area and sets the transmittance of the fifth area to the fifth transmittance. The third ambient light brightness is different from the fifth ambient light brightness. The AR device detects the second ambient light brightness of the first area and adjusts the transmittance of the first area to the second transmittance. The AR device detects the fourth ambient light brightness of the fourth area and adjusts the transmittance of the fourth area to the fourth transmittance. The AR device detects the sixth ambient light brightness of the fifth area and adjusts the transmittance of the fifth area to the sixth transmittance. The second ambient light brightness is greater than the first ambient light brightness, and the second transmittance is less than the first transmittance. The fourth ambient light brightness is different from the sixth ambient light brightness. The fourth ambient light brightness is greater than the third ambient light brightness, and the fourth transmittance is less than the third transmittance. The sixth ambient light brightness is greater than the fifth ambient light brightness, and the sixth transmittance is less than the fifth transmittance. The AR device projects light from a virtual image that has not been dimmed onto the human eye through the optical module.

2. The method according to claim 1, characterized in that, The greater the ambient light intensity, the lower the light transmittance.

3. The method according to claim 1 or 2, characterized in that, After adjusting the transmittance of the first region to the second transmittance, the method further includes: The AR device detects the third ambient light brightness of the first area and maintains the light transmittance of the first area at the second light transmittance, wherein the third ambient light brightness is greater than the second ambient light brightness.

4. The method according to claim 1 or 2, characterized in that, When the fourth ambient light brightness is the same as the second ambient light brightness, the fourth transmittance is greater than or equal to the second transmittance.

5. The method according to claim 1 or 2, characterized in that, At the same time, the sixth ambient light brightness is greater than the fourth ambient light brightness, and the sixth transmittance is less than the fourth transmittance.

6. The method according to claim 1 or 2, characterized in that, The method further includes: After the AR device detects that a video application is running, it adjusts the light transmittance of the first area to a seventh light transmittance, which is less than the second light transmittance.

7. The method according to claim 1 or 2, characterized in that, After the AR device detects the second ambient light intensity of the first area, the method further includes: The AR device increases the brightness of the light projected onto the virtual image through the optical module.

8. The method according to claim 1 or 2, characterized in that, After adjusting the transmittance of the first region to the second transmittance, the method further includes: The AR device detects the seventh ambient light brightness of the first area and adjusts the light transmittance of the first area to the eighth light transmittance. The seventh ambient light brightness is less than the second ambient light brightness, and the eighth light transmittance is greater than the second light transmittance.

9. The method according to claim 1 or 2, characterized in that, Before adjusting the transmittance of the first region to the second transmittance, the method further includes: The AR device receives the first operation.

10. An AR device, characterized in that, The AR device includes a processor, an ambient light sensor, a dimming module, and an optical module. The dimming module covers the side of the optical module away from the human eye. The dimming module includes a first region, a fourth region, and a fifth region. The area corresponding to the first region in the optical module is used to project a virtual image, and the area of ​​the first region is larger than the area of ​​the region in the optical module used to project a virtual image. The areas corresponding to the fourth and fifth regions in the optical module are not used to project virtual images. The areas corresponding to the first, fourth, and fifth regions in the optical module are all used to present real images. The fourth and fifth regions are arranged around the first region, with the fourth region located above the first region and the fifth region located below the first region. The fourth and fifth regions do not overlap with the first region. Each of the first, fourth, and fifth regions of the dimming module contains an independent conductive layer. The magnitude of the voltage received by the conductive layer determines the light transmittance of the corresponding area of ​​the conductive layer. The optical module is used to project ambient light that has passed through the dimming module onto the human eye; The processor is configured to set the transmittance of the first region to a first transmittance when the ambient light sensor detects a first ambient light brightness in the first region, set the transmittance of the fourth region to a third transmittance when the ambient light sensor detects a third ambient light brightness in the fourth region, and set the transmittance of the fifth region to a fifth transmittance when the ambient light sensor detects a fifth ambient light brightness in the fifth region, wherein the third ambient light brightness is different from the fifth ambient light brightness. The processor is further configured to adjust the transmittance of the first region to a second transmittance when the ambient light sensor detects a second ambient light luminance in the first region, adjust the transmittance of the fourth region to a fourth transmittance when the ambient light sensor detects a fourth ambient light luminance in the fourth region, and adjust the transmittance of the fifth region to a sixth transmittance when the ambient light sensor detects a sixth ambient light luminance in the fifth region. The second ambient light brightness is greater than the first ambient light brightness, the second transmittance is less than the first transmittance, the fourth ambient light brightness is different from the sixth ambient light brightness, the fourth ambient light brightness is greater than the third ambient light brightness, the fourth transmittance is less than the third transmittance, the sixth ambient light brightness is greater than the fifth ambient light brightness, and the sixth transmittance is less than the fifth transmittance. The optical module is also used to project light from a virtual image that is not controlled by the dimming module onto the human eye.

11. An AR device, characterized in that, The AR device includes: a processor, one or more memories, a dimming module, and an optical module; the dimming module covers the side of the optical module away from the human eye; the one or more memories are coupled to the one or more processors; the one or more memories are used to store computer program code, the computer program code including computer instructions; when the one or more processors execute the computer instructions, the AR device performs the method as described in any one of claims 1-9.

12. A computer storage medium, characterized in that, Includes computer instructions that, when executed on an AR device, cause the AR device to perform the method as described in any one of claims 1-9.

13. A computer program product, characterized in that, When the computer program product is run on a computer, it causes the computer to perform the method as described in any one of claims 1-9.

Images