A control method, device and electronic equipment

Abstract

The embodiments of the present application disclose a control method, device and electronic device, which relate to electronic devices, so that according to the determined dominant eye, the working mechanism and rendering processing mechanism of the eye tracking module are adjusted, thereby reducing the ineffective work of the insensitive auxiliary eye, achieving the effect of reducing the power consumption of VR glasses, and then improving the working efficiency of VR glasses. The specific scheme is: determine the dominant eye. In response to the determination result, turn off the eye tracking module corresponding to the non-dominant eye. In response to the determination result, adjust the rendering parameters of the image corresponding to the non-dominant eye from the first rendering parameter to the second rendering parameter. Among them, the rendering parameters include the output image frame rate, and the output image frame rate included in the first rendering parameter is greater than the output image frame rate included in the second rendering parameter. The image corresponding to the non-dominant eye is rendered according to the second rendering parameter.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of electronic devices, and in particular, to a control method, device and electronic device. Background Art

[0002] At present, augmented reality (AR) and virtual reality (VR) technologies are gradually emerging. In some application scenarios of AR / VR technologies, AR / VR devices are required to be able to achieve eye tracking through eye tracking modules and provide corresponding functions accordingly.

[0003] Take VR glasses as an example of AR / VR devices. VR glasses can be equipped with two eye tracking modules, each of which is used to track one eye. When a user wears VR glasses, the VR glasses can obtain eye tracking data of both eyes of the user through the eye tracking module, and provide gaze point rendering, identity authentication based on iris recognition, payment, personal information customization and other functions.

[0004] It can be seen that in order to provide the above functions, VR glasses need to continuously perform eye tracking, which requires the two eye tracking modules to work for a long time. This will lead to the problem of high power consumption of VR glasses. Summary of the invention

[0005] The embodiments of the present application provide a control method, device and electronic device, which can adjust the working mechanism and / or rendering processing mechanism of the eye tracking module according to the determined dominant eye, thereby reducing the ineffective work of the insensitive auxiliary eye, achieving the effect of reducing the power consumption of VR glasses, and then improving the working efficiency of VR glasses.

[0006] In order to achieve the above purpose, the embodiment of the present application adopts the following technical solution:

[0007] In a first aspect, a control method is provided, which is applied to an electronic device, wherein the electronic device includes two eye tracking modules, and the two eye tracking modules correspond to different human eyes. The method includes: determining a dominant eye. In response to the determination result, turning off the eye tracking module corresponding to the non-dominant eye. In response to the determination result, adjusting the rendering parameters of the image corresponding to the non-dominant eye from a first rendering parameter to a second rendering parameter. The rendering parameters include an output image frame rate, and the output image frame rate included in the first rendering parameter is greater than the output image frame rate included in the second rendering parameter. The image corresponding to the non-dominant eye is rendered according to the second rendering parameter.

[0008] Based on this solution, a power consumption reduction solution that can be applied to current electronic devices is provided. In this example, the electronic device can be an AR / VR device. In this example, by turning off the eye tracking module corresponding to the auxiliary eye, the power consumption generated by the long-term operation of the eye tracking module is saved. In addition, the image rendering parameters of the non-dominant eye (i.e., the auxiliary eye) can be adjusted to further reduce power consumption without affecting the user's visual perception. It can be understood that since the auxiliary eye is not sensitive to image quality, reducing the image quality corresponding to the auxiliary eye will not affect the user's visual perception. Therefore, the eye tracking used to improve the image quality of the auxiliary eye can be turned off, and the auxiliary eye is not tracked, so as to reduce the power consumption of the device without affecting the user's visual perception. In addition, since the auxiliary eye is not tracked, the computing power overhead in the process of eye tracking the auxiliary eye can also be saved, thereby improving the working efficiency of the electronic device. It should be noted that, in this example, since a solution of reducing the output image frame rate is adopted to render the image corresponding to the auxiliary eye, the processing pressure of rendering the image corresponding to the auxiliary eye can be saved. At the same time, since the number of frame images that need to be displayed is reduced, the instruction overhead in the process of submitting the rendering results for display can also be reduced. It should be understood that the above effects can enable electronic devices to save unnecessary power consumption, thereby improving work performance. Among them, for different users, the dominant eye can be the left eye or the right eye, and the embodiment of the present application does not limit whether the dominant eye is the left eye or the right eye.

[0009] In a possible design, determining the dominant eye includes: displaying a light-transmitting hole and prompt information. The image plane where the light-transmitting hole is located is between the image plane where the prompt information is located and the electronic device. In response to detecting a first operation, adjusting the position of the light-transmitting hole on the image plane where the light-transmitting hole is located. In response to detecting a second operation, according to the display content on the first display screen and the second display screen, the human eye corresponding to the display screen displaying the prompt information is used as the dominant eye. Among them, the first display screen is a display screen in the electronic device that displays an image to the first human eye. The second display screen is a display screen in the electronic device that displays an image to the second human eye. Based on this scheme, a specific scheme for determining the dominant eye of the user is provided. In this example, the electronic device can determine the dominant eye by showing a virtual three-dimensional scene to the user and combining the user's operation. For example, the user can hold a piece of paper with a light-transmitting hole with an aperture of about 1.5 inches in the virtual scene. The user can adjust the position of the paper in the virtual scene through the first operation so that the prompt information (such as 1-inch high letters) in the three-dimensional scene can be observed through the light-transmitting hole by both eyes. In this example, the user can input the second operation to the electronic device when the prompt information can be clearly seen through the light-transmitting hole by both eyes. In response to the second operation, the electronic device can determine the dominant eye according to the content displayed on the two display screens. Among them, the second operation can be a user pressing or touching the corresponding button or icon, or input through voice or body language. It should be noted that in some other implementations of the present application, the electronic device can also determine that the user can see the prompt information through the light-transmitting hole when the user does not input the operation of moving the light-transmitting hole / paper within a certain time (such as a preset duration). It can be understood that when the images seen by the dominant eye and the auxiliary eye are different, the human brain will take the image seen by the dominant eye as the main one, and due to the difference in the position of the two eyes in space, the graphics seen by the two eyes through the small holes are not consistent. In other words, when the prompt information can be seen by both eyes, the human eye corresponding to one of the display screens displaying the prompt information on the two display screens of the electronic device can be the dominant eye. Thus, the electronic device can determine the dominant eye according to the content displayed on the two display screens. In this way, through the method in this example, the electronic device can accurately obtain the distribution of the user's dominant eye and auxiliary eye with the slight participation of the user.

[0010] In a possible design, the image plane where the light-transmitting hole is located is the first virtual image plane, and the image plane where the prompt information is located is the second virtual image plane. The distance between the first virtual image plane and the electronic device is the first preset distance. The distance between the first virtual image plane and the second virtual image plane is the second preset distance. Based on this scheme, a specific implementation scheme for an electronic device to display light-transmitting holes and prompt information is provided. In this example, the electronic device can display the light-transmitting holes and prompt information in a virtual three-dimensional space according to the preset first preset distance and the second preset distance. Among them, the first preset distance can be the user's arm length, and the second preset distance can be about 10 feet. In some implementations of the present application, the electronic device can obtain the user's arm length information when it is used by the user for the first time, for example, prompting the user to enter the arm length information through the corresponding application, and the electronic device can use the obtained arm length information as the first preset distance. In some other implementations of the present application, the electronic device can also preset the first preset distance and the second preset distance therein, for example, the first preset distance can be the arm length of an ordinary user determined by survey sampling.

[0011] In a possible design, the electronic device corresponds to a handheld device. Displaying a light-transmitting hole and prompt information includes: determining the position of the light-transmitting hole according to the position of the handheld device, and displaying the light-transmitting hole at the position of the light-transmitting hole. The distance between the image plane where the prompt information is located and the image plane where the light-transmitting hole is located is a third preset distance. The position of the handheld device is detected and obtained by the electronic device, or the position of the handheld device is obtained by the electronic device from the handheld device. Based on this scheme, another specific implementation scheme for displaying light-transmitting holes and prompt information by an electronic device is provided. In this example, the electronic device can display the light-transmitting hole and prompt information in combination with the corresponding handheld device. Take the handheld device as a remote control as an example. The electronic device can prompt the user to hold the remote control and stretch out his arm to simulate stretching his arm in a virtual space to pick up a piece of paper with a 1.5-inch aperture light-transmitting hole. The electronic device can collect relevant information through a sensor or an optoelectronic device set on the remote control to determine the position of the remote control in space. For example, an electromagnetic transmitting component can be set on the remote control, and the electronic device can receive electromagnetic waves emitted by the electromagnetic transmitting component, and spatially locate the remote control according to the electromagnetic wave, thereby determining the position of the remote control. For another example, a light-emitting component may be provided on the remote control, and the remote control may start to emit light at the moment when it is aligned with the electronic device. Correspondingly, the electronic device may determine the distance between the remote control and the electronic device according to the time difference between the moment when the light is received and the moment when the remote control emits light, and then determine the spatial position of the remote control in combination with the incident angle of the received light. For another example, the electronic device may take a photo of the remote control, and determine the spatial position of the remote control based on the result of analyzing the acquired image. In some other implementations of the present application, the remote control may also determine the current spatial position through a sensor provided therein, and send information carrying the spatial position to the electronic device so that the electronic device knows the spatial position of the remote control. It is understandable that since the image plane and the viewing plane (or the corresponding optical axis when the electronic device is worn to view the virtual three-dimensional space) are in a vertical relationship in space, after determining the spatial position of the handheld device, the electronic device may determine the image plane where the light-transmitting hole is located in the virtual three-dimensional space according to the current viewing plane, or line of sight, or other information that can be used to identify the optical axis, combined with the spatial position of the handheld device, and then display the light-transmitting hole at the position corresponding to the remote control on the image plane. It should be noted that in another implementation of this example, the electronic device can also display the light hole at the corresponding position according to the preset position relationship between the handheld device and the light hole. Based on the position of the handheld device, or the image plane where the light hole is located, the electronic device can display the corresponding prompt information at the third preset distance. It should be noted that in this example, the third preset distance can be 10 feet, the same as the second preset distance, or other preset values.

[0012] In one possible design, the eye tracking module includes an LED light group and an IR camera, and turning off the eye tracking module corresponding to the non-dominant eye includes: turning off the LED light group and / or the IR camera in the eye tracking module corresponding to the non-dominant eye. Based on this solution, a specific solution for turning off the eye tracking module corresponding to the non-dominant eye is provided. For example, the LED light group corresponding to the non-dominant eye can be turned off to save the power consumption generated by the long-term operation of the LED light group. For another example, the IR camera corresponding to the non-dominant eye can be turned off to save the power consumption generated by the continuous taking pictures of the IR camera. For another example, the LED light group and IR camera corresponding to the non-dominant eye can also be turned off at the same time. The selection of the components to be turned off can be flexibly selected according to the internal architecture of the electronic device that implements the solution.

[0013] In one possible design, the rendering parameters also include resolution, and / or a gaze point rendering range. Based on this solution, a specific composition of rendering parameters is provided. In this example, the rendering parameters may also include resolution, and / or a gaze point rendering range. This allows the electronic device to render the auxiliary eye image with lower rendering parameters, thereby saving the power consumption and computing power of the electronic device without affecting the user's visual perception.

[0014] In a second aspect, a control device is provided, which is applied to an electronic device, wherein the electronic device includes two eye tracking modules, and the two eye tracking modules correspond to different human eyes. The device includes: a determination unit, which is used to determine the dominant eye. A tracking control unit, which is used to turn off the eye tracking module corresponding to the non-dominant eye in response to the determination result. An adjustment unit, which is used to adjust the rendering parameters of the image corresponding to the non-dominant eye from the first rendering parameters to the second rendering parameters in response to the determination result. The rendering parameters include an output image frame rate, and the output image frame rate included in the first rendering parameter is greater than the output image frame rate included in the second rendering parameter. The image corresponding to the non-dominant eye is rendered according to the second rendering parameter.

[0015] In a possible design, the determination unit is specifically used to: display a light-transmitting hole and prompt information. The image plane where the light-transmitting hole is located is located between the image plane where the prompt information is located and the electronic device. In response to detecting the first operation, the position of the light-transmitting hole on the image plane where the light-transmitting hole is located is adjusted. In response to detecting the second operation, according to the display content on the first display screen and the second display screen, the human eye corresponding to the display screen displaying the prompt information is used as the dominant eye. Among them, the first display screen is a display screen in the electronic device that displays an image to the first human eye. The second display screen is a display screen in the electronic device that displays an image to the second human eye.

[0016] In a possible design, the image plane where the light-transmitting hole is located is the first virtual image plane, and the image plane where the prompt information is located is the second virtual image plane. The distance between the first virtual image plane and the electronic device is the first preset distance. The distance between the first virtual image plane and the second virtual image plane is the second preset distance.

[0017] In a possible design, the electronic device corresponds to a handheld device. The determination unit is specifically used to: determine the position of the light-transmitting hole according to the position of the handheld device, and display the light-transmitting hole at the position of the light-transmitting hole. The distance between the image plane where the prompt information is located and the image plane where the light-transmitting hole is located is a third preset distance. The device also includes an acquisition unit, which is used to detect and acquire the position of the handheld device, or to acquire the position of the handheld device from the handheld device.

[0018] In one possible design, the tracking control unit is specifically used to turn off the LED light group and / or IR camera in the eye tracking module corresponding to the non-dominant eye.

[0019] In one possible design, the rendering parameters also include resolution, and / or foveated rendering range.

[0020] In a third aspect, an electronic device is provided, wherein a first eye tracking module and a second eye tracking module are provided in the electronic device, wherein the first eye tracking module is used to perform eye tracking on a first person's eye, and the second eye tracking module is used to perform eye tracking on a second person's eye. The electronic device includes one or more processors and one or more memories. The one or more memories are coupled to the one or more processors, and the one or more memories store computer instructions. When the one or more processors execute the computer instructions, the electronic device executes a control method as described in any one of the first aspect and possible designs thereof.

[0021] In a fourth aspect, a computer-readable storage medium is provided, the computer-readable storage medium comprising computer instructions, and when the computer instructions are executed, a control method as described in any one of the first aspect and possible designs thereof is executed.

[0022] In a fifth aspect, a chip system is provided, which includes an interface circuit and a processor; the interface circuit and the processor are interconnected through lines; the interface circuit is used to receive signals from a memory and send signals to the processor, and the signals include computer instructions stored in the memory; when the processor executes the computer instructions, the chip system executes a control method as described in the first aspect above and any one of various possible designs.

[0023] In a sixth aspect, a computer program product is provided, which includes instructions. When the computer program product runs on a computer, the computer can execute a control method as described in the first aspect and any of various possible designs according to the instructions.

[0024] It should be understood that the technical solutions provided in the second, third, fourth, fifth and sixth aspects above and their technical features can all correspond to the control methods provided in the first aspect and its possible designs, and therefore the beneficial effects that can be achieved are similar and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 A schematic diagram of the composition of an eye tracking module;

[0026] Figure 2 A schematic diagram of the composition of another eye tracking module;

[0027] Figure 3 A schematic diagram of the composition of an AR / VR device provided in an embodiment of the present application;

[0028] Figure 4 A schematic diagram of the composition of a VR glasses provided in an embodiment of the present application;

[0029] Figure 5 A schematic diagram of a method for VR glasses to determine a user's dominant eye and / or auxiliary eye provided by an embodiment of the present application;

[0030] Figure 6 A schematic diagram of a method for determining a user's primary viewing eye and / or auxiliary viewing eye using VR glasses provided in an embodiment of the present application;

[0031] Figure 7 A schematic diagram of the operation of VR glasses provided in an embodiment of the present application;

[0032] Figure 8 A schematic diagram for comparing the playback timing of a display screen provided in an embodiment of the present application;

[0033] Fig. 9 A schematic diagram for comparing the playback timing of another display screen provided in an embodiment of the present application;

[0034] Fig.10 A schematic diagram for comparing the playback timing of another display screen provided in an embodiment of the present application;

[0035] Fig.11 A schematic diagram of the composition of a control device provided in an embodiment of the present application;

[0036] Fig.12 A schematic diagram of the composition of an electronic device provided in an embodiment of the present application;

[0037] Fig.13 A schematic diagram of the composition of a chip system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0038] In current AR / VR devices, eye tracking can be achieved through an eye tracking module, thereby determining the gaze point of the human eye and / or information such as pupil characteristics, and then providing users with various corresponding functions. Exemplarily, the AR / VR device can determine the gaze point coordinates corresponding to the user's eyes through the eye tracking module, and perform gaze point rendering according to the gaze point coordinates. For example, the AR / VR device can finely render the pixels near the gaze point coordinates on the image according to the gaze point coordinates of the human eye, so that the user can see a high-quality image near the gaze point. Since the human eye is not sensitive to the image quality far away from the gaze point, the AR / VR device can appropriately reduce the rendering quality for pixels far away from the gaze point. It can be understood that through gaze point rendering, the AR / VR device can improve the user's visual perception without fine rendering of the full-frame image. This can effectively reduce the computing pressure of the AR / VR device.

[0039] To enable eye tracking, please refer to Figure 1 , shows the composition of an eye tracking module. Figure 1 As shown, the eye tracking module may include a light-emitting diode (LED) light group and an infrared (IR) camera, wherein the LED light group may include a plurality of LED lights. Figure 1 In the description, an LED light group including 4 LED lights is used as an example. An eye tracking module can correspond to one of the user's eyes and is used to track the eye movement of the eye.

[0040] For example, taking the determination of gaze point coordinates through an eye tracking module as an example, AR / VR devices can establish a correspondence between the image features of the human eye's response to light and the gaze point coordinates, and then determine the gaze point coordinates in real time based on the image features of the human eye's response to light.

[0041] For example, when the eye tracking module is working, the AR / VR device can display an observation point on the display screen to prompt the user to look at the observation point. In other words, the coordinates of the gaze point corresponding to the human eye at the current moment (i.e., the position of the observation point) are known.

[0042] The LED light group can project the generated light (such as infrared light) toward the human eye along path 1. The IR camera can collect the response image of the human eye when the light is projected onto the human eye through path 2.

[0043] It should be noted that different areas of the human eye (such as the pupil area and other areas of the human eye) respond differently to light. In this example, the feature of the pupil area's response to light expressed in the image (i.e., the pupil feature) is recorded as feature 1. Feature 1 can be included in the response image acquired by the IR camera. When the human eye looks at different positions, the position of the pupil area in the response image is different, so the position of feature 1 in the response image is also different. Therefore, by guiding the user to look at a known observation point, the correspondence between the human eye's gaze point coordinates (the same as the observation point position) and the response image including feature 1 at the corresponding position can be established.

[0044] The AR / VR device can adjust the observation point position multiple times and perform the above measurements respectively. After completing multiple measurements, the AR / VR device can obtain the coordinates of the gaze point corresponding to feature 1 at different positions on the image. In this way, when the user subsequently uses the AR / VR device, the coordinates of the gaze point of the human eye at the corresponding moment can be quickly determined based on the position of feature 1 in the response image obtained by the IR camera at each moment.

[0045] After obtaining the gaze point coordinates of the human eye, the AR / VR device can perform gaze point rendering. It can be understood that through the above scheme, the IR camera can obtain the response image of different areas of the human eye, including the pupil area, to light. Based on the response image, the AR / VR device can determine the identity of the current user based on iris recognition technology, and then realize functions such as identity authentication, payment, and personal information customization.

[0046] Combination pair Figure 1 In the AR / VR device, eye tracking modules can be set near the user's eyes to achieve eye tracking for both eyes. For example, please refer to Figure 2 , taking the AR / VR device as VR glasses as an example, the composition of VR glasses including an eye tracking module is shown. Figure 2 As shown, an eye tracking module 1 including an LED light group 1 and an IR camera 1, and an eye tracking module 2 including an LED light group 2 and an IR camera 2 can be provided in the VR glasses. The eye tracking module 1 can be used to track the left eye, and the eye tracking module 2 can be used to track the right eye. Figure 2 The composition shown can realize real-time eye tracking of both eyes through two eye tracking modules.

[0047] It should be understood that when the eye tracking module is working, the LED light group needs to continuously project light to the human eye, and the IR camera also needs to continuously take pictures of the human eye to obtain the image of the human eye's response to light at the corresponding moment. This will put a great pressure on the power consumption of VR glasses. The resulting equipment heating and other problems will also affect the use of VR glasses.

[0048] In order to solve the above problems, the technical solution provided in the embodiment of the present application enables the AR / VR device to achieve a visual perception effect similar to that when binocular eye tracking is performed, based on the characteristics of the human eye (such as the distribution of the primary eye and the auxiliary eye), with only one eye tracking module. The solution provided in the embodiment of the present application can significantly save the power consumption of the AR / VR device without affecting the user experience.

[0049] The solution provided in the embodiments of the present application is described in detail below with reference to the accompanying drawings.

[0050] Please refer to Figure 3 , which is a schematic diagram of the composition of an AR / VR device provided in an embodiment of the present application. The AR / VR device may include a processor, two display modules (such as display module 1 and display module 2), and two eye tracking modules (such as eye tracking module 1 and eye tracking module 2).

[0051] It should be understood that the structure illustrated in this embodiment does not constitute a specific limitation on the AR / VR device. In other embodiments, the AR / VR device may include more or fewer components than shown in the figure, or combine certain components, or split certain components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0052] In such Figure 3In the composition shown, the processor may include one or more processing units, for example: the processor may include an application processor (application processor, AP), a modem processor, a graphics processor (graphics processing unit, GPU), an image signal processor (image signal processor, ISP), a controller, a memory, a video codec, a digital signal processor (digital signal processor, DSP), a baseband processor, and / or a neural-network processing unit (neural-network processing unit, NPU), etc. Among them, different processing units can be independent devices or integrated in one or more processors. In different implementations of the present application, the rendering process of the image can be performed by the GPU or by a processor with computing functions such as the CPU. In addition, the acquisition, processing and issuance of commands in the electronic device can be performed by the CPU or by other components with similar functions. For the sake of convenience, the following is an example of the GPU performing image rendering and the CPU performing the acquisition, processing and issuance of other commands.

[0053] A memory may also be provided in the processor for storing instructions and data. In some embodiments, the memory in the processor is a cache memory. The memory may store instructions or data that the processor has just used or is cyclically used. If the processor needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor, and thus improves the efficiency of the system.

[0054] The two display modules may correspond to one of the user's eyes respectively. For example, display module 1 may correspond to the user's left eye, and display module 2 may correspond to the user's right eye. Each display module may be used to display corresponding content to the corresponding eye.

[0055] As an example, the display module may include a display screen and a lens. For example, the display module 1 may include a display screen 1 and a lens 1, and the display module 2 may include a display screen 2 and a lens 2. Taking the display module 1 as an example, when the display module 1 is working, an image corresponding to the left eye may be displayed on the display screen 1 under the control of the processor. The left eye may see the image on the display screen 1 through the lens 1. Due to the refraction of light by the lens 1, the lens 1 may provide a larger field of view when the human eye is close to the screen, thereby allowing the user to see a wider range of images through the lens 1. It should be noted that in different implementation scenarios, the functions of the lens 1 or the lens 2 may be realized by one lens respectively, or by a lens group including multiple optical elements. In addition, in the actual implementation process, the display screen 1 and the display screen 2 may correspond to different hardware display components, and in other implementations, the display screen 1 and the display screen 2 may also correspond to different display areas on a display component.

[0056] Similarly, the two eye tracking modules also correspond to one of the user's eyes. For example, eye tracking module 1 can correspond to the user's left eye, and eye tracking module 2 can correspond to the user's right eye. Each eye tracking module can be used to track the corresponding eye. For example, the eye tracking module can work under the control of the processor according to the following example: Figure 2 Similar methods are implemented as shown and will not be described again here.

[0057] In this example, the processor can be used to process various data and information, and control each module to implement its corresponding function. For example, the GPU can perform rendering processing according to the image features corresponding to each coordinate in the three-dimensional space in the virtual scene that needs to be displayed provided by the application, so as to obtain the data stream corresponding to the two-dimensional images corresponding to the left eye and the right eye respectively. Among them, in the process of performing the rendering processing, the GPU can perform rendering processing according to some rendering parameters. For example, these rendering parameters may include the frame rate, resolution and gaze point rendering range of the output image. After obtaining the data streams corresponding to the two-dimensional images corresponding to the left eye and the right eye respectively, the GPU can transmit these data streams to display screen 1 and display screen 2 respectively, so that each display screen displays the corresponding images to the left eye and the right eye respectively.

[0058] Taking AR / VR devices as VR glasses as an example, Figure 4 It shows that Figure 3 A VR glasses composed of the following components. Figure 4As shown, the display screen 1 and the lens 1 can provide a display function for the left eye. The LED light group 1 and the IR camera 1 can be used to implement eye tracking for the left eye. Similarly, the display screen 2 and the lens 2 can provide a display function for the right eye. The LED light group 2 and the IR camera 2 can be used to implement eye tracking for the right eye.

[0059] In some implementations, the LEDs in the LED light group may be arranged along the lens. Figure 4 As shown, the LED lamp group may include 4 LED lamps, and the 4 LED lamps may be evenly arranged along the outer extension of the lens.

[0060] When using the current solution in Figure 4 In the VR glasses shown, the processor can control the eye tracking module 1 to perform eye tracking on the left eye, thereby obtaining information including the left eye gaze point, pupil characteristics, etc. The processor can perform image rendering (such as gaze point rendering) in combination with the above information to obtain a data stream of the image corresponding to the left eye, and display it to the left eye through the display screen 1. At the same time, the processor can also control the eye tracking module 2 to perform eye tracking on the right eye, thereby obtaining information including the right eye gaze point, pupil characteristics, etc., and based on this, obtain a data stream of the image corresponding to the right eye, and display it to the right eye through the display screen 2. This requires LED light group 1 and LED light group 2, IR camera 1 and IR camera 2 to work for a long time at the same time. The processor (such as GPU) also needs to synchronously process the data corresponding to both eyes and perform rendering and other processing. As a result, the power consumption of VR glasses is often too high, affecting their use.

[0061] It should be noted that, based on the study of human eyes, human eyes are divided into dominant eyes and auxiliary eyes. Among them, the dominant eye is also called the fixation eye or the dominant eye. From a physiological point of view, everyone has a dominant eye, which may be the left eye or the right eye. What the dominant eye sees will be accepted by the brain first. The human brain habitually uses the imaging of the dominant eye to analyze and locate objects in the image. In other words, in visual perception, the dominant eye is dominant. Correspondingly, the image seen by the auxiliary eye can generally be used as a supplement to the image seen by the dominant eye. When the images seen by the dominant eye and the auxiliary eye are different, the human brain tends to determine the current image based on the image seen by the dominant eye. Therefore, it can be understood that the dominant eye is more sensitive to changes in image quality, while the auxiliary eye is not sensitive to changes in image quality. On the premise of ensuring the image quality of the dominant eye, the slightly poor image quality of the auxiliary eye will not affect the user's visual perception.

[0062] Combining the above Figure 4As shown in the figure, the working mechanism of VR glasses is described. Since VR glasses provide images of the same precision to both eyes (i.e., the main eye and the auxiliary eye), and the auxiliary eye is not sensitive to the image quality, the eye tracking process for the auxiliary eye will have little benefit. At the same time, the fine image rendering for the auxiliary eye is also unnecessary. It should be understood that these will lead to the waste of power consumption of VR glasses, thereby reducing the working efficiency of VR glasses.

[0063] To address this problem, an embodiment of the present application provides a control method, which allows AR / VR devices including VR glasses to pre-determine the user's dominant eye and / or auxiliary eye, and adjust the working mechanism and / or rendering processing mechanism of the eye tracking module accordingly, thereby reducing ineffective work on the insensitive auxiliary eye, achieving the effect of reducing the power consumption of VR glasses, and thereby improving the working efficiency of VR glasses.

[0064] The solution provided in the embodiment of the present application can be applied to Figure 3 In the AR / VR device composed as shown. For example, this solution can be applied to Figure 4 In the VR glasses shown in FIG. For the sake of convenience, the following application of this solution is as follows Figure 4 The VR glasses shown are used as an example for explanation.

[0065] Different from the prior art, the solution provided in the embodiment of the present application can detect the user's dominant eye and / or auxiliary eye. As an example, VR glasses can determine the user's dominant eye and / or auxiliary eye through a pinhole imaging method.

[0066] Exemplary, combined Figure 5 , shows a method for VR glasses to determine the user's primary eye and / or auxiliary eye. It can be understood that the processor in the VR glasses can control the display screen 1 and the display screen 2 to display a virtual image to the user in a three-dimensional space. Figure 5 As shown, the virtual image may include two virtual image planes, such as image plane 1 and image plane 2. Among them, the distance between image plane 1 and the human eye is closer, and the distance between image plane 2 and the human eye is farther. In some implementations, the image plane distance between image plane 2 and image plane 1 can be 10 feet (inches). The distance between image plane 1 and the user can be the user's arm length (such as the left arm length or the right arm length). On image plane 2, prompt information can be displayed, for example, the prompt information can be letters, numbers or text. Figure 5 In the example, the prompt information is the letter H. In some implementations, the height of the letter H can be 1 inch. A virtual small hole can be set on the image plane 1, and other areas on the image plane 1 are opaque. In other words, through the light-transmitting hole, the user can see the virtual image on the image plane 2. In some implementations, the light-transmitting hole can be a circular hole with an aperture of about 1.5 inches.

[0067] Under the prompt of the VR glasses, the user can input operations to the VR glasses to move the position of the pinhole on the image plane 1, so that the letters on the image plane 2 can be seen through the pinhole 1 when observing with both eyes.

[0068] When the user can clearly see the letters through the small hole, it can indicate that the VR glasses can clearly see the letters. In some implementations, the input of the indication can be achieved through voice control, touching, or clicking or double-clicking a preset key. In other implementations of the present application, after being able to clearly see the letters, the user may not input any operation, so that the VR glasses can determine that the letters can be clearly seen by determining that the user has not input any operation within a preset time.

[0069] Combination Figure 5 When the user can clearly see the letters on the image plane 2, only one eye can actually see the letters through the light-transmitting hole (for example Figure 5 The left eye in the figure can see the letter H through the path of line of sight 1), while the other eye will not see the letter through the light hole (for example Figure 5 The right eye in the figure will not see the letter H through the path of line of sight 2). At this time, the left eye can be determined as the main eye, and correspondingly, the right eye can be determined as the auxiliary eye.

[0070] It is understandable that when a user sees a virtual image through both eyes, he actually watches the image 1 on the display screen corresponding to the left eye (such as display screen 1) through the left eye, and watches the image 2 on the display screen corresponding to the right eye (such as display screen 2) through the right eye, and synthesizes the obtained visual perception in the human brain after acquiring the image 1 and the image 2. Combined with the above description, when the user can clearly see the letter H through both eyes, only one image of the image 1 and the image 2 will show the letter H to the user. The image 1 and the image 2 are displayed on the display screen controlled by the processor in the VR glasses. Therefore, the VR glasses can know whether the image showing the letter H is the image 1 displayed on the display screen 1 or the image 2 displayed on the display screen 2 when the user can see the letter H through both eyes. In this example, the VR glasses can determine the human eye corresponding to the display screen showing the letter H as the dominant eye.

[0071] As a specific implementation example, the VR glasses can display the following information to the user when the primary eye and / or the auxiliary eye need to be detected: Figure 6The virtual scene shown in the figure. In this virtual scene, the user can simulate holding a piece of paper with a small hole of 1.5 inches in diameter in the virtual scene by holding the remote control corresponding to the VR glasses. The user can hold the remote control and stretch out his arm to simulate holding the paper in his hand and stretching out his arm in the virtual scene. The user can adjust the spatial position of the remote control to achieve the effect of adjusting the position of the small hole on the paper in the virtual scene. In this virtual scene, you can move the remote control 10 feet away from the image plane of the paper (i.e., Figure 5 The image of the wall is displayed on the image plane 2 shown in the figure. A 1-inch letter H can be hung on the wall. The user can continuously adjust the position of the small hole until the letter H on the wall can be clearly seen through the small hole when observing with both eyes. Then, the user can maintain this posture for a short time, and the VR glasses can determine that the user can clearly see the letters on the wall through the small hole in the current state by the user stopping moving the remote control. In this way, the VR glasses can determine the main viewing eye and / or auxiliary viewing eye according to whether there are letters displayed on the display screen in the current state.

[0072] It should be noted that the above-mentioned pinhole imaging method can be used to determine the dominant eye or the auxiliary eye during the actual use of VR glasses, or the dominant eye and the auxiliary eye can be determined by this method. For example, in some implementations of the present application, the VR glasses can use the above-mentioned pinhole imaging method to determine that when the user can clearly see the letters through both eyes, the human eye corresponding to the display screen displaying letters is the dominant eye. In other implementations, the VR glasses can use the above-mentioned pinhole imaging method to determine that when the user can clearly see the letters through both eyes, the human eye corresponding to the display screen that does not display letters is the auxiliary eye. In other implementations, the VR glasses can use the above-mentioned pinhole imaging method to determine that when the user can clearly see the letters through both eyes, the human eye corresponding to the display screen that does not display letters is the dominant eye, and the human eye corresponding to the display screen that does not display letters is the auxiliary eye.

[0073] The following takes the case where the left eye is the dominant eye and the right eye is the auxiliary eye, and the VR glasses determine the dominant eye through the pinhole imaging method as an example to illustrate the execution process after the dominant eye is determined in this solution.

[0074] After determining that the dominant eye is the left eye, VR glasses can reduce the power consumption of VR devices by adjusting the working mechanism of the eye tracking module.

[0075] Exemplary, combined Figure 4 After determining that the dominant eye is the left eye, the VR glasses can turn off the eye tracking module 2 used for eye tracking of the right eye, that is, turn off the eye tracking of the auxiliary eye, thereby saving power consumption caused by eye tracking of the auxiliary eye.

[0076] As a comparison, when both eye tracking modules are working properly, the VR glasses work like this: Figure 4 As shown in the figure, LED light group 1 and LED light group 2 are both emitting light normally, and IR camera 1 and IR camera 2 are also capturing the corresponding response images of the human eye normally, and eye tracking is performed on the left eye and right eye respectively. Correspondingly, in this example, when the eye tracking module of the right eye is turned off, the working condition of the VR glasses is as follows: Figure 7 As shown. That is, LED light group 1 and IR camera 1 work normally, and perform normal eye tracking on the left eye (i.e., the main eye). LED light group 2 and IR camera 2 are in a dormant state or turned off, and do not perform eye tracking on the right eye (i.e., the auxiliary eye), thereby saving power consumption loss caused by eye tracking of the right eye. For example, it can save power consumption caused by the light emission of LED light group 2, and it can also save power consumption loss caused by IR camera 2 taking pictures of the right eye, and it can also save power consumption loss caused by relevant calculations when the CPU in VR glasses determines the coordinates of the right eye's gaze point and other information.

[0077] It should be noted that in some implementations of the present application, after determining that the dominant eye is the left eye, the image rendering mechanism corresponding to the other eye (such as the right eye) can also be adjusted, thereby saving power consumption and computing bandwidth consumed in the rendering process of the image corresponding to the auxiliary eye.

[0078] Exemplarily, in combination with the foregoing description, before controlling the display screen to display the corresponding image, the VR device needs to control the GPU to perform rendering processing according to the image features of each position in the three-dimensional virtual scene sent by the application, as well as the rendering parameters, to obtain the corresponding image data. Among them, the rendering parameters may include the frame rate, resolution, and gaze point rendering range of the output image. In this example, when the GPU renders the image of the auxiliary eye, it can reduce the rendering parameters to use a lower computing bandwidth to obtain the data stream of the image displayed to the auxiliary eye. The effect displayed according to the data stream may be worse than the display effect for the main eye. Since the auxiliary eye is not sensitive to image quality, the adjustment will not affect the user's visual perception. In this example, power consumption and computing expenses in the rendering process can be saved in this way.

[0079] In some embodiments, the GPU may use a lower output image frame rate when rendering the image corresponding to the right eye (ie, the auxiliary eye).

[0080] It is understandable that, in a normal rendering process, the GPU can perform rendering of the N+1 frame image after rendering the Nth frame image. In this example, the output image frame rate is reduced to half of that in a normal rendering process. The GPU can perform rendering of the N+2 frame image after rendering the Nth frame image, and skip rendering of the N+1 frame. When the data stream of the image after rendering is sent to the display screen for display, the display screen can display continuous images by interpolation according to the data stream. For example, after the display of the Nth frame image is completed, since there is no N+1 frame image in the data stream of the image, the display screen can continue to display the Nth frame image during the time period when the N+1 frame image should be displayed. When the N+2 frame image needs to be displayed, the display screen can display the N+2 frame image according to the data corresponding to the N+2 frame image in the data stream. In this way, the GPU can save half of the rendering calculations, thereby obtaining the effect of saving rendering calculations and power consumption.

[0081] Combination Figure 8 , which shows a comparison of the playback timings of the display screens. The left eye (i.e., the dominant eye) corresponds to display screen 1. On display screen 1, playback can be performed according to playback timing 1, that is, the order of the Nth frame image, the N+1th frame image, and the N+2th frame image is displayed in sequence. If the prior art is used, display screen 2 can be displayed according to the same playback timing as display screen 1 (such as playback timing 2). However, using the solution in this example, display screen 2 can display the Nth frame image within the playback time corresponding to the display screen 1 displaying the Nth frame image and the N+1th frame image. Later, when display screen 1 displays the N+2th frame image, display screen 2 also displays the N+2th frame image, that is, according to the above Figure 8 Just display the playback timing 3 in .

[0082] It is understandable that the display of the image on the display screen is submitted to the display screen for display through the driver after the GPU completes the rendering. Since the number of frame images that need to be displayed in the same length of time is reduced when displaying using the playback timing 3, the cost of the GPU submitting display instructions to the display screen through the driver is saved.

[0083] It should be noted that in the above examples, GPU frame rendering is used as an example. In other implementations of the present application, the GPU can also perform image rendering at other intervals. For example, when the output image frame rate is reduced to 1 / 3 of the normal rendering process, the GPU can skip the N+1 frame and the N+2 frame after completing the rendering of the N-th frame image, and perform the rendering of the N+3 frame image. In this way, 2 / 3 of the rendering calculations of the GPU can be saved. In different implementations, the adjustment of the output image frame rate can be determined by the VR glasses themselves, or it can be determined according to a preset value, and the embodiments of the present application do not limit this.

[0084] In other embodiments, the GPU may use a lower resolution to perform rendering processing on the image corresponding to the right eye (ie, the auxiliary viewing eye).

[0085] Exemplarily, during normal rendering, the GPU can perform rendering processing on each frame of the image at a resolution of A, thereby obtaining a rendering result at a resolution of A, and correspondingly, an image at a resolution of A can be displayed on the display screen. Specifically, during the GPU rendering of the image, the image features of multiple spatial positions in the virtual space can be mapped to a pixel according to the resolution to obtain the image features of the pixel. The larger the resolution, the more pixels the image has, the more times the above mapping is performed, and the greater the corresponding computing power required. According to the scheme in this example, the GPU can perform rendering processing according to the rendering parameters with a resolution of B after the resolution is reduced. Among them, B is less than A. Specifically, the GPU can map the image features of each position in the virtual space to B pixels to obtain the image features of each pixel in the B pixels. The rendering result after the resolution is reduced is obtained. In this way, the GPU only needs to perform B mappings without performing A mapping operations. Therefore, the GPU can reduce the computing power consumption of the GPU rendering process by reducing the number of mappings.

[0086] Combination Fig. 9 , showing a comparison before and after reducing the resolution. For the rendering of the left eye (dominant eye), a higher resolution can be used and displayed on the display screen 1. For example, the display screen 1 corresponding to the left eye can display the following: Fig. 9 In the case of using the prior art, the display screen 2 corresponding to the right eye can display an image with a high resolution similar to that displayed on the display screen 1 (e.g. Fig. 9 However, since the right eye is not sensitive to image quality, it is obviously unnecessary to display a high-resolution image to the right eye. Fig. 9 The image shown in (c) in the figure has a relatively low resolution. This can significantly save the computing power cost of image rendering.

[0087] In other embodiments, the GPU can appropriately adjust the mechanism of gaze point rendering when rendering the image corresponding to the right eye (i.e., the auxiliary eye). Combined with the above description, in order to save the computing power overhead of high-precision rendering, VR glasses can only perform fine rendering (or high-precision rendering) for the area near the gaze point, and render the area far away from the gaze point with lower precision, thereby achieving the purpose of improving the user's visual perception with lower computing power overhead. In this application, since the right eye is not sensitive to image quality, it is not necessary to perform local high-precision rendering of the right eye. In other words, when the GPU performs image rendering, it can not perform gaze point rendering on the image corresponding to the right eye, but use lower precision to render the image corresponding to the right eye. Although the image obtained is of low fineness, it is displayed to the auxiliary eye, so it will not cause a decrease in the user's visual perception.

[0088] Combination Fig.10 , shows a comparison diagram of an image rendered with a focus point. Fig.10 As shown, for the left eye (i.e., the dominant eye), the GPU can perform gaze point rendering according to the gaze point coordinates of the left eye, and accordingly display the following on the display screen 1: Fig.10 The image shown in (a) in FIG. The area near the gaze point is rendered by the gaze point, which improves the fineness. Therefore, when the left eye sees the image, the image quality can be significantly improved. If the existing technology is used, the following can be displayed on the display screen 2: Fig.10 The image shown in (b) in FIG. 1 also includes the high-precision area after the foveation rendering. However, when the right eye views the image, the right eye is not sensitive to image quality and may not be able to feel the improvement in image quality due to the presence of the high-precision area. Therefore, in this example, the following image may be displayed on the display screen 2: Fig.10 The image shown in (c) in FIG. The image may be a full-frame image obtained by normal rendering. The right eye sees Fig.10 The visual perception after seeing the image shown in (b) is similar to that after seeing Fig.10 There is little difference in visual perception after viewing the image shown in (c), and the GPU can save computing power for gaze point rendering. It should be understood that in this example, the example of not performing gaze point rendering on the auxiliary eye (i.e. reducing the area of ​​gaze point rendering to 0) is used for explanation. In other implementations of the present application, the GPU can also save GPU computing power by appropriately reducing the size of the gaze point rendering area of ​​the auxiliary eye image. Among them, since the VR glasses may have turned off eye tracking for the right eye, the gaze point coordinates of the right eye cannot be obtained. In this example, the VR glasses can use the geometric center of the image as the assumed gaze point coordinates for gaze point rendering after a narrowed range.

[0089] In the above example, three possible solutions are provided for reducing the image quality of the auxiliary eye so that the power consumption of VR glasses can be reduced and the GPU computing power overhead can be saved without the user noticing. In actual use, VR glasses can flexibly select one or more of the above three solutions. In addition, VR glasses can also combine the eye tracking mechanism and the above-mentioned quality adjustment solution to further reduce the power consumption of VR glasses. For example, after determining the main eye, VR glasses can turn off the eye tracking module for the auxiliary eye and reduce the image quality corresponding to the auxiliary eye, thereby achieving the effect of reducing the power consumption of VR glasses.

[0090] It should be noted that in some other embodiments of the present application, the VR glasses can also determine whether to take the power consumption reduction measures provided in the above examples (such as the working mechanism of the eye tracking module, and reducing the image rendering parameters of the auxiliary eye) according to the current scene. As an example, the VR glasses can determine that in the current scene, when only the gaze point rendering function is involved, one or more of the above power consumption reduction measures are turned on. Correspondingly, the VR glasses can not turn on the above power consumption reduction measures in user research scenarios based on binocular data, so as to obtain more comprehensive data.

[0091] The above mainly introduces the solution provided by the embodiment of the present application from the perspective of electronic devices (such as VR glasses). In order to achieve the above functions, it includes hardware structures and / or software modules corresponding to the execution of each function. It should be easy for those skilled in the art to realize that, in combination with the units and algorithm steps of each example described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0092] The embodiment of the present application can divide the functional modules of the devices involved therein according to the above method examples. For example, each functional module can be divided corresponding to each function, or two or more functions can be integrated into one processing module. The above integrated modules can be implemented in the form of hardware or in the form of software functional modules. It should be noted that the division of modules in the embodiment of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation.

[0093] Please refer to Fig.11 , a control device 1100 is provided, which is applied to an electronic device, wherein the electronic device includes two eye tracking modules, and the two eye tracking modules correspond to different human eyes. Fig.11As shown, the device may include: a determination unit 1101, configured to determine a dominant eye. A tracking control unit 1102, configured to shut down an eye tracking module corresponding to a non-dominant eye in response to a determination result. An adjustment unit 1103, configured to adjust a rendering parameter of an image corresponding to the non-dominant eye from a first rendering parameter to a second rendering parameter in response to a determination result. The rendering parameter includes an output image frame rate, and the output image frame rate included in the first rendering parameter is greater than the output image frame rate included in the second rendering parameter. The image corresponding to the non-dominant eye is rendered according to the second rendering parameter.

[0094] In one possible design, the device further includes: a display unit 1104, for displaying a light-transmitting hole and prompt information. The image plane where the light-transmitting hole is located is located between the image plane where the prompt information is located and the electronic device. In response to detecting the first operation, the position of the light-transmitting hole on the image plane where the light-transmitting hole is located is adjusted. A determination unit 1101 is used to, in response to detecting the second operation, use the human eye corresponding to the display screen displaying the prompt information as the dominant eye according to the display content on the first display screen and the second display screen. The first display screen is a display screen in the electronic device that displays an image to the first human eye. The second display screen is a display screen in the electronic device that displays an image to the second human eye.

[0095] In a possible design, the image plane where the light-transmitting hole is located is the first virtual image plane, and the image plane where the prompt information is located is the second virtual image plane. The distance between the first virtual image plane and the electronic device is the first preset distance. The distance between the first virtual image plane and the second virtual image plane is the second preset distance.

[0096] In a possible design, the electronic device corresponds to a handheld device. The display unit 1104 is used to determine the position of the light-transmitting hole according to the position of the handheld device, and display the light-transmitting hole at the position of the light-transmitting hole. The distance between the image plane where the prompt information is located and the image plane where the light-transmitting hole is located is a third preset distance. The device also includes an acquisition unit 1105, which is used to detect and acquire the position of the handheld device, or to acquire the position of the handheld device from the handheld device.

[0097] In one possible design, the tracking control unit 1102 is specifically used to turn off the LED light group and / or IR camera in the eye tracking module corresponding to the non-dominant eye.

[0098] In one possible design, the rendering parameters also include resolution, and / or foveated rendering range.

[0099] It should be noted that all relevant contents of each step involved in the above method embodiment can be referred to the functional description of the corresponding functional module and will not be repeated here.

[0100] Please refer to Fig.12, is an electronic device 1200 provided in an embodiment of the present application, wherein a first eye tracking module and a second eye tracking module are provided in the electronic device 1200, wherein the first eye tracking module is used to perform eye tracking on a first person's eye, and the second eye tracking module is used to perform eye tracking on a second person's eye. The electronic device 1200 includes one or more processors 1201 and one or more memories 1202. The one or more memories 1202 are coupled to the one or more processors 1201, and the one or more memories 1202 store computer instructions. When the one or more processors 1201 execute the computer instructions, the electronic device 1200 executes any one of the control methods provided in the embodiments of the present application.

[0101] It should be noted that all relevant contents of each step involved in the above method embodiment can be referred to the functional description of the corresponding functional module and will not be repeated here.

[0102] Please refer to Fig.13 , a chip system 1300 provided in an embodiment of the present application, the chip system 1300 includes an interface circuit 1301 and a processor 1302; the interface circuit 1301 and the processor 1302 are interconnected through lines; the interface circuit 1301 is used to receive signals from the memory and send signals to the processor 1302, the signals include computer instructions stored in the memory; when the processor 1302 executes the computer instructions, the chip system 1300 executes any one of the control methods provided in the embodiments of the present application.

[0103] It should be noted that all relevant contents of each step involved in the above method embodiment can be referred to the functional description of the corresponding functional module and will not be repeated here.

[0104] The functions or actions or operations or steps in the above embodiments can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using a software program, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that a computer can access or may include one or more servers, data centers and other data storage devices that can be integrated with the medium. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a DVD), or a semiconductor medium (eg, a solid state disk (SSD)).

[0105] Although the present application has been described in conjunction with specific features and embodiments thereof, it is obvious that various modifications and combinations may be made thereto without departing from the spirit and scope of the present application. Accordingly, this specification and the drawings are merely exemplary illustrations of the present application as defined by the appended claims, and are deemed to have covered any and all modifications, variations, combinations or equivalents within the scope of the present application. Obviously, those skilled in the art may make various modifications and variations to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A control method, characterized in that: Applied to an electronic device, the electronic device includes two eye tracking modules, the two eye tracking modules correspond to different human eyes, and the method includes: Determine the dominant eye; In response to the determination result, turning off the eye tracking module corresponding to the non-dominant eye; In response to the determination result, adjusting the rendering parameters of the image corresponding to the non-dominant eye from the first rendering parameters to the second rendering parameters; wherein the rendering parameters include an output image frame rate, the output image frame rate included in the first rendering parameters is greater than the output image frame rate included in the second rendering parameters, and the rendering parameters of the image corresponding to the dominant eye are the first rendering parameters; The image corresponding to the dominant eye is rendered according to the first rendering parameters, and the image corresponding to the non-dominant eye is rendered according to the second rendering parameters.

2. The method according to claim 1, characterized in that: Determining the dominant eye comprises: Displaying a light-transmitting hole and prompt information; the image plane where the light-transmitting hole is located is located between the image plane where the prompt information is located and the electronic device; In response to detecting a first operation, adjusting a position of the light-transmitting hole on an image plane where the light-transmitting hole is located; In response to detecting the second operation, according to the display contents on the first display screen and the second display screen, the human eye corresponding to the display screen displaying the prompt information is used as the dominant eye; The first display screen is a display screen in the electronic device that displays images to the first human eye; the second display screen is a display screen in the electronic device that displays images to the second human eye.

3. The method according to claim 2, characterized in that The image plane where the light-transmitting hole is located is the first virtual image plane, and the image plane where the prompt information is located is the second virtual image plane; The distance between the first virtual image plane and the electronic device is a first preset distance; The distance between the first virtual image plane and the second virtual image plane is a second preset distance.

4. The method according to claim 2, characterized in that: The electronic device corresponds to a handheld device; The display of light-transmitting holes and prompt information includes: Determining the position of the light-transmitting hole according to the position of the handheld device, and displaying the light-transmitting hole at the position of the light-transmitting hole; The distance between the image plane where the prompt information is located and the image plane where the light-transmitting hole is located is a third preset distance; The position of the handheld device is detected and acquired by the electronic device, or the position of the handheld device is acquired by the electronic device from the handheld device.

5. The method according to any one of claims 1 to 4, characterized in that The eye tracking module includes an LED light group and an IR camera. The eye tracking module corresponding to the non-dominant eye is turned off, including: Turn off the LED light group and / or IR camera in the eye tracking module corresponding to the non-dominant eye.

6. The method according to claim 5, characterized in that The rendering parameters also include resolution, and / or gaze point rendering range.

7. An electronic device, characterized in that: The electronic device is provided with a first eye tracking module and a second eye tracking module, wherein the first eye tracking module is used to perform eye tracking on a first person's eye, and the second eye tracking module is used to perform eye tracking on a second person's eye; The electronic device comprises one or more processors and one or more memories; the one or more memories are coupled to the one or more processors, and the one or more memories store computer instructions; When the one or more processors execute the computer instructions, the electronic device executes the control method according to any one of claims 1 to 6.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes computer instructions, and when the computer instructions are executed, the control method according to any one of claims 1 to 6 is executed.

9. A chip system, characterized in that: The chip system includes an interface circuit and a processor; the interface circuit and the processor are interconnected through lines; the interface circuit is used to receive a signal from a memory and send the signal to the processor, the signal including a computer instruction stored in the memory; when the processor executes the computer instruction, the chip system executes the control method as described in any one of claims 1-6.

Images