Display method and device of eye movement equipment, electronic equipment, storage medium and computer program product

Abstract

The invention relates to a display method and device of eye movement equipment, electronic equipment, a storage medium and a computer program product, and the method comprises the steps that under the condition that it is detected that at least two application windows are opened, the eye movement equipment displays a target picture comprising the at least two application windows; determining a gazing area of the gazing object on the target picture; determining a display parameter of each application window based on a position relationship between the fixation area and each application window; wherein different position relations correspond to different display parameters, and power consumption generated by displaying according to different display parameters is different. Therefore, the CPU resources and the GPU resources required by each application window can be dynamically allocated, so that the problem of display lagging of a display screen of the eye movement equipment is reduced while the fixation condition of the fixation object is flexibly adapted, the use power consumption of the eye movement equipment is reduced, and the use experience of a user is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of human-computer interaction technology, and in particular to a display method, device, electronic device, storage medium, and computer program product for an eye movement device. Background Art

[0002] Eye tracking devices are wearable virtual display products. The technical principles of current eye tracking devices are roughly divided into virtual reality (VR), augmented reality (AR), mixed reality (MR), and extended reality (XR).

[0003] However, most eye-tracking devices use a fixed rendering strategy. When the display screen of the eye-tracking device displays multiple application windows, it usually provides the same clarity display parameters for the multiple application windows, so that the visual effects presented by each application window are consistent, resulting in the problem of not being able to flexibly adapt to the user's viewing needs, which is not conducive to improving the user experience. Summary of the Invention

[0004] To overcome the problems in the related art, the present disclosure provides a display method, device, electronic device, storage medium and computer program product for an eye-tracking device, which can dynamically allocate the CPU resources and GPU resources required for each application window, while flexibly adapting to the gaze situation of the gaze object, reducing the problem of display freezes on the eye-tracking device's display screen, reducing the power consumption of the eye-tracking device, and improving the user experience.

[0005] According to a first aspect of an embodiment of the present disclosure, a display method of an eye-tracking device is provided, comprising:

[0006] When detecting that at least two application windows are opened, the eye-tracking device displays a target screen including the at least two application windows;

[0007] Determine a gaze area of ​​a gaze object on the target image;

[0008] determining display parameters of each of the application windows based on a positional relationship between the gaze area and each of the application windows;

[0009] Different positional relationships correspond to different display parameters, and the power consumption generated by displaying according to different display parameters is different.

[0010] In some embodiments, the eye-tracking device displays a target screen including the at least two application windows, including:

[0011] The eye-tracking device displays the target image according to the first display parameter;

[0012] The determining, based on the positional relationship between the gaze area and each of the application windows, display parameters of each of the application windows includes:

[0013] Based on the positional relationship between the gaze area and each of the application windows, maintaining the application window to be displayed according to the first display parameter, or adjusting the application window to be displayed according to the second display parameter;

[0014] The power consumption of displaying according to the second display parameter is lower than the power consumption of displaying according to the first display parameter.

[0015] In some embodiments, determining the display parameters of each application window based on the positional relationship between the gaze area and each application window includes:

[0016] In a case where it is determined based on the positional relationship that there is an application window in contact with the gaze area, determining the application window in contact with the gaze area as a target window;

[0017] determining display parameters of each of the application windows based on a contact area between the target window and the gaze area;

[0018] The power consumption generated by displaying the application windows other than the target window is lower than the power consumption generated by displaying the target window.

[0019] In some embodiments, determining the display parameters of each of the application windows based on the contact area between the target window and the gaze area includes:

[0020] When the contact area is smaller than a preset area threshold, displaying the target window according to a third display parameter, and displaying the application windows other than the target window according to a fourth display parameter; wherein power consumption generated by displaying according to the third display parameter is greater than power consumption generated by displaying according to the fourth display parameter;

[0021] When the contact area is greater than or equal to the preset area threshold, the target window is displayed according to the fifth display parameter, and the application windows other than the target window are displayed according to the sixth display parameter; wherein the power consumption generated by displaying according to the fifth display parameter is greater than the power consumption generated by displaying according to the third display parameter and the sixth display parameter.

[0022] In some embodiments, there are at least two target windows;

[0023] The power consumption corresponding to the display parameters of each target window is positively correlated with the contact area between each target window and the gaze area.

[0024] In some embodiments, the method further comprises:

[0025] When it is detected that the positional relationship between the gaze area and the application window changes, the display parameters of each application window are re-determined based on the changed positional relationship.

[0026] In some embodiments, re-determining the display parameters of each of the application windows based on the changed positional relationship includes:

[0027] In a case where it is determined based on the changed positional relationship that the target window is separated from the gaze area, the target window separated from the gaze area is adjusted to be displayed according to the second display parameter.

[0028] In some embodiments, the method further comprises:

[0029] When a first adjustment operation on the display parameters of the application window is detected, the application window is controlled to be displayed according to the display parameters indicated by the first adjustment operation within a preset time period.

[0030] In some embodiments, the method further comprises:

[0031] When a second adjustment operation on the application window is detected, clearing display parameters of the application window;

[0032] The second adjustment operation is used to instruct to stop displaying the application window.

[0033] In some embodiments, determining the gaze area of ​​the gaze object on the target image includes:

[0034] When detecting that the eye-tracking device is in an on state, determining a gaze area of ​​the gaze object on the target image based on the acquired gaze information of the gaze object;

[0035] The gaze information includes: gaze direction and / or gaze position.

[0036] In some embodiments, the method further comprises:

[0037] When it is detected that the eye movement device enters the closed state, the gaze information and the display parameters of each application window are cleared.

[0038] According to a second aspect of an embodiment of the present disclosure, there is provided a display device, including:

[0039] A display module configured to, when detecting that at least two application windows are opened, cause the eye-tracking device to display a target screen including the at least two application windows;

[0040] A first determining module is configured to determine a gaze area of ​​a gaze object on the target image;

[0041] a second determining module configured to determine display parameters of each of the application windows based on a positional relationship between the gaze area and each of the application windows;

[0042] Different positional relationships correspond to different display parameters, and the power consumption generated by displaying according to different display parameters is different.

[0043] In some embodiments, the display module is configured as follows:

[0044] The eye-tracking device displays the target image according to the first display parameter;

[0045] The second determining module is configured to:

[0046] Based on the positional relationship between the gaze area and each of the application windows, maintaining the application window to be displayed according to the first display parameter, or adjusting the application window to be displayed according to the second display parameter;

[0047] The power consumption of displaying according to the second display parameter is lower than the power consumption of displaying according to the first display parameter.

[0048] In some embodiments, the second determining module includes:

[0049] A first submodule is configured to, when it is determined based on the positional relationship that there is an application window in contact with the gaze area, determine the application window in contact with the gaze area as a target window;

[0050] a second submodule configured to determine display parameters of each of the application windows based on a contact area between the target window and the gaze area;

[0051] The power consumption generated by displaying the application windows other than the target window is lower than the power consumption generated by displaying the target window.

[0052] In some embodiments, the second submodule is configured as follows:

[0053] When the contact area is smaller than a preset area threshold, displaying the target window according to a third display parameter, and displaying the application windows other than the target window according to a fourth display parameter; wherein power consumption generated by displaying according to the third display parameter is greater than power consumption generated by displaying according to the fourth display parameter;

[0054] When the contact area is greater than or equal to the preset area threshold, the target window is displayed according to the fifth display parameter, and the application windows other than the target window are displayed according to the sixth display parameter; wherein the power consumption generated by displaying according to the fifth display parameter is greater than the power consumption generated by displaying according to the third display parameter and the sixth display parameter.

[0055] In some embodiments, there are at least two target windows;

[0056] The power consumption corresponding to the display parameters of each target window is positively correlated with the contact area between each target window and the gaze area.

[0057] In some embodiments, the apparatus further comprises:

[0058] The third determining module is configured to, when detecting that a positional relationship between the gaze area and the application window has changed, re-determine display parameters of each application window based on the changed positional relationship.

[0059] In some embodiments, the third determining module is configured to:

[0060] In a case where it is determined based on the changed positional relationship that the target window is separated from the gaze area, the target window separated from the gaze area is adjusted to be displayed according to the second display parameter.

[0061] In some embodiments, the apparatus further comprises:

[0062] The control module is configured to, when a first adjustment operation on the display parameters of the application window is detected, control the application window to be displayed according to the display parameters indicated by the first adjustment operation within a preset time period.

[0063] In some embodiments, the apparatus further comprises:

[0064] a first clearing module configured to clear the display parameters of the application window when a second adjustment operation on the application window is detected;

[0065] The second adjustment operation is used to instruct to stop displaying the application window.

[0066] In some embodiments, the apparatus further comprises:

[0067] a fourth determining module configured to, when detecting that the eye-tracking device is in an on state, determine a gaze area of ​​the gaze object on the target image based on the acquired gaze information of the gaze object;

[0068] The gaze information includes: gaze direction and / or gaze position.

[0069] In some embodiments, the apparatus further comprises:

[0070] The second clearing module is configured to clear the gaze information and the display parameters of each application window when detecting that the eye movement device enters a closed state.

[0071] According to a third aspect of an embodiment of the present disclosure, there is provided an electronic device, including:

[0072] processor;

[0073] memory for storing computer programs or instructions;

[0074] The processor executes the computer program or instruction to implement the steps of any one of the display methods in the first aspect above.

[0075] According to a fourth aspect of an embodiment of the present disclosure, a non-transitory computer-readable storage medium is provided, comprising:

[0076] When the computer program or instructions in the storage medium are executed by a processor, the steps in any one of the display methods in the first aspect are implemented.

[0077] According to a fifth aspect of an embodiment of the present disclosure, a computer program product is provided, comprising a computer program or instructions, which, when executed by a processor, implement the steps of any one of the display methods in the first aspect.

[0078] The technical solutions provided by the embodiments of the present disclosure may have the following beneficial effects:

[0079] In an embodiment of the present disclosure, when it is detected that at least two application windows are open, the eye tracking device displays a target screen including at least two application windows; then determines the gaze area of ​​the gaze object on the target screen; finally, based on the positional relationship between the gaze area and each application window, determines the display parameters of each application window; wherein different positional relationships correspond to different display parameters, and the power consumption generated by displaying according to different display parameters is different.

[0080] On the one hand, compared with the related art, the eye-tracking device adopts unified display parameters for each application window so that the visual effects presented by each application window are consistent, resulting in the problem of not being able to flexibly adapt to the user's viewing needs. The embodiment of the present disclosure determines the positional relationship between the gaze area and each application window, adjusts the display parameters of each application window, so that the display parameters of the application window change with the change of the positional relationship, thereby being able to flexibly adapt to the gaze situation of the gaze object, which is conducive to improving the user's viewing experience.

[0081] On the other hand, compared with the related art, the eye tracking device uses a unified high display parameter to display each application window, which causes the display screen of the eye tracking device to freeze or display delay. In the embodiment of the present disclosure, the power consumption generated by displaying different display parameters is different. Therefore, by determining the positional relationship between the gaze area and each application window, the CPU resources and GPU resources required by each application window are dynamically allocated, and the display parameters of each application window are flexibly adjusted, so that each application window is not displayed according to the unified high display parameter. This effectively avoids the problem of display delay on the display screen while reducing the power consumption of the eye tracking device and improving the user experience.

[0082] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0083] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0084] Figure 1 This is a flow chart of a display method of an eye-tracking device according to an exemplary embodiment. Figure 1 ;

[0085] Figure 2 This is a display scene 1 of a display screen of an eye tracking device according to an exemplary embodiment;

[0086] Figure 3 This is a flow chart of a display method of an eye-tracking device according to an exemplary embodiment. Figure 2 ;

[0087] Figure 4 This is a second display scene of a display screen of an eye tracking device according to an exemplary embodiment;

[0088] Figure 5 This is a display scene 3 of a display screen of an eye tracking device according to an exemplary embodiment;

[0089] Figure 6This is a fourth display scene of a display screen of an eye tracking device according to an exemplary embodiment;

[0090] Figure 7 is a block diagram of a display device according to an exemplary embodiment;

[0091] Figure 8 is a structural block diagram of an electronic device 800 according to an exemplary embodiment;

[0092] Figure 9 is a block diagram showing a device 900 for display according to an exemplary embodiment. DETAILED DESCRIPTION

[0093] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements, unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all possible embodiments consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure, as detailed in the appended claims.

[0094] Figure 1 This is a flow chart of a display method of an eye-tracking device according to an exemplary embodiment. Figure 1 ,like Figure 1 As shown, the display method mainly includes the following steps:

[0095] In step 101, when detecting that at least two application windows are open, the eye tracking device displays a target screen including the at least two application windows;

[0096] In step 102, a gaze area of ​​a gaze object on a target image is determined;

[0097] In step 103, based on the positional relationship between the gaze area and each application window, display parameters of each application window are determined;

[0098] Different positional relationships correspond to different display parameters, and the power consumption generated by displaying according to different display parameters is different.

[0099] Here, the eye movement device may be a virtual reality device (VR), an augmented reality device (AR) or a mixed reality device (MR), and this disclosure does not limit this.

[0100] Taking AR as an example, the eye-tracking device uses computer graphics technology and visualization technology to construct virtual content that does not exist in the display environment, and accurately integrates the virtual content into the real environment through image recognition and positioning technology. With the help of the display module in AR, the virtual content and the real environment are integrated to provide users with a real viewing experience.

[0101] In some embodiments, in order to enhance the user's virtual reality experience, the eye-tracking device can provide the user with some application windows, so that when the user wears the eye-tracking device, he can operate the application window to conveniently perform predetermined functions, such as quickly switching scenes, sending files, and moving objects within the scene, etc., thereby helping to improve user satisfaction.

[0102] It is understandable that application windows include but are not limited to application windows provided by the eye-tracking device, such as video windows, file windows or chat windows, etc.; virtual components (Widgets) provided by the eye-tracking device, such as virtual pendants, virtual characters or virtual objects.

[0103] For example, Figure 2 This is a display scene 1 of a display screen of an eye-tracking device according to an exemplary embodiment. Figure 2 As shown, Figures 24-27 are application windows provided by the eye-tracking device, which may be a video window, a file window, or a chat window, etc., and Figures 21-23 are virtual components provided by the eye-tracking device, which may be a virtual character or a virtual object, etc.

[0104] In a related technology, an eye-tracking device uses the same display parameters for at least two application windows displayed on a target screen to ensure that the visual effects achieved by at least two application windows are consistent. However, this method cannot flexibly adapt to the user's viewing needs. For example, the user wants a certain application window to be displayed according to high display parameters and other application windows to be displayed according to low display parameters, which is not conducive to improving the user experience.

[0105] In another related technology, the eye-tracking device uses the same high display parameters to display multiple application windows on the target screen to ensure high-definition display of each application window. However, this method causes the eye-tracking device to be in high-definition rendering working mode all the time, which not only causes delays or freezes in the display screen of the eye-tracking device, but also increases the power consumption of the eye-tracking device.

[0106] Based on this, in the embodiment of the present disclosure, the gaze area of ​​the gaze object on the target screen is first determined, and then the display parameters of each application window are adjusted based on the positional relationship between the gaze area and the each application window, so that the eye-tracking device can dynamically provide the required resources for displaying the display parameters of each application window, such as CPU resources and graphics processor unit resources (Graphics Processing Unit, GPU), thereby flexibly adapting to the gaze situation of the gaze object while reducing the display delay or display freeze problem of the eye-tracking device, reducing the power consumption of the eye-tracking device, and improving the user experience.

[0107] In some embodiments, eye tracking technology of an eye movement device can be utilized. For example, infrared light can be used to assist the eye movement device in capturing eye images. The position and movement trajectory information of the eyeball can be measured by tracking and calculating the center position of the pupil in the eye image. Then, based on the position and movement trajectory information of the eyeball, the gaze area of ​​the gaze object on the target screen can be determined.

[0108] Here, the size of the gaze area can be set as needed, and the embodiments of the present disclosure are not limited to this. In some embodiments, the size of the gaze area can be associated with the number of application windows displayed on the target screen. For example, the size of the gaze area can be negatively correlated with the number of application windows displayed on the target screen. When there are many application windows displayed on the target screen, the size of the gaze area is set to be smaller. When there are fewer application windows displayed on the target screen, the size of the gaze area is set to be larger, which is conducive to better locking on the window that the user wants to focus on.

[0109] In some embodiments, the shape of the gaze area can also be set as needed. For example, the gaze area can be circular, rectangular, triangular, or a rounded polygon. In some embodiments, the shape of the gaze area can be associated with the shape of the application window displayed on the target screen. For example, if the application windows are all rectangular, the shape of the gaze area can also be set to a rectangle. This helps better lock the window that the user wants to focus on and improves the accuracy of the screen display.

[0110] After determining the gaze area of ​​the gaze object on the target screen, the positional relationship between the gaze area and each application window can be determined first, and it can be judged whether each application window is an application window within the gaze range of the gaze object. Then, based on the judgment result, the display parameters of each application window are determined. Therefore, it is possible to avoid wasting CPU and GPU resources while ensuring smooth display of the eye tracking device's display screen.

[0111] In some embodiments, the positional relationship between the gaze area and each application window includes, but is not limited to, a relative distance, a relative angle, or an overlap between the gaze area and each application window.

[0112] Here, the display parameters include but are not limited to display frame rate, display brightness, display resolution, and display clarity, etc., which are not limited in the embodiments of the present disclosure.

[0113] In some embodiments, after determining the positional relationship between the gaze area and each application window, it is possible to first determine whether the gaze area and each application window are in a distant state or a close state based on the positional relationship. When it is determined that the gaze area is in a distant state from a certain application window, it is possible to determine that the application window is a non-target window (i.e., not the application window being gazed at by the gaze object), and reduce the display parameters of the application window. When it is determined that the gaze area is in a close state from a certain application window, it is possible to determine that the application window is a target window (i.e., the application window being gazed at by the gaze object), and increase the display parameters of the application window. The distant state includes a state in which the distance between the gaze area and the application window is greater than a preset distance threshold, and the close state includes a state in which the distance between the gaze area and the application window is less than or equal to a preset distance threshold. The preset distance threshold can be determined as needed and is not specifically limited here.

[0114] Among them, the distance between the gaze area and the application window can be the distance between the center point of the gaze area and the center point of the application window, or it can be based on the distance between the edge of the gaze area and the edge of the application window. As long as the relative distance between the gaze area and the application window can be represented, the determination method is not specifically limited here.

[0115] For example, the gaze area can be rectangular in shape, and the display parameters of each application window can be adjusted based on the positional relationship between the edge of each application window and the edge of the rectangle. When the edge of the application window is close to the edge of the rectangle, the display parameters of the application window are increased to make the content of the application window appear clearer; when the edge of the application window is far from the edge of the rectangle, the display parameters of the application window are decreased to make the content of the application window appear blurry.

[0116] As another example, the gaze area can be rectangular, with the intersection of the two diagonals of the rectangle being the central gaze point of the gazed object. The display parameters of each application window can be adjusted based on the positional relationship between each application window and the central gaze point. When the center point of an application window is close to the central gaze point, the display parameters of the application window are increased to make the content of the application window appear clearer; when the center point of the application window is far from the central gaze point, the display parameters of the application window are decreased to make the content of the application window appear blurry.

[0117] In the embodiment of the present disclosure, different positional relationships correspond to different display parameters, and the power consumption generated by displaying according to different display parameters is different. Therefore, after each application window is displayed according to the determined display parameters, the power consumption of the eye tracking device can be reduced, thereby increasing the usage time of the eye tracking device and improving the user experience.

[0118] In an embodiment of the present disclosure, when it is detected that at least two application windows are open, the eye tracking device displays a target screen including at least two application windows; then determines the gaze area of ​​the gaze object on the target screen; finally, based on the positional relationship between the gaze area and each application window, determines the display parameters of each application window; wherein different positional relationships correspond to different display parameters, and the power consumption generated by displaying according to different display parameters is different.

[0119] On the one hand, compared with the related art, the eye-tracking device adopts unified display parameters for each application window so that the visual effects presented by each application window are consistent, resulting in the problem of not being able to flexibly adapt to the user's viewing needs. The embodiment of the present disclosure determines the positional relationship between the gaze area and each application window, adjusts the display parameters of each application window, so that the display parameters of the application window change with the change of the positional relationship, thereby being able to flexibly adapt to the gaze situation of the gaze object, which is conducive to improving the user's viewing experience.

[0120] On the other hand, compared with the related art, the eye tracking device uses a unified high display parameter to display each application window, which causes the display screen of the eye tracking device to freeze or display delay. In the embodiment of the present disclosure, the power consumption generated by displaying different display parameters is different. Therefore, by determining the positional relationship between the gaze area and each application window, the CPU resources and GPU resources required by each application window are dynamically allocated, and the display parameters of each application window are flexibly adjusted, so that each application window is not displayed according to the unified high display parameter. This effectively avoids the problem of display delay on the display screen while reducing the power consumption of the eye tracking device and improving the user experience.

[0121] In some embodiments, the eye tracking device displays a target screen including at least two application windows, including:

[0122] The eye-tracking device displays the target image according to the first display parameter;

[0123] Based on the positional relationship between the gaze area and each application window, display parameters of each application window are determined, including:

[0124] Based on the positional relationship between the gaze area and each application window, maintaining the application window displayed according to the first display parameter, or adjusting the application window to be displayed according to the second display parameter;

[0125] The power consumption of displaying according to the second display parameter is lower than the power consumption of displaying according to the first display parameter.

[0126] It should be noted that in order to ensure that the eye-tracking device can provide the user with a real viewing experience, the eye-tracking device can display the target picture according to the first display parameter with better display effect, so that the display effect of the target picture (for example, the picture clarity and picture brightness of the target picture) meets the user's viewing needs, thereby improving user satisfaction.

[0127] In some embodiments, the display parameters of the eye movement device for each application window, namely the first display parameter and the second display parameter, can be pre-set. By determining the positional relationship between the gaze area and each application window, it is determined whether the application window is displayed according to the first display parameter or the application window is displayed according to the second display parameter.

[0128] Since the eye tracking device displays the target screen according to the first display parameters by default, after obtaining the positional relationship between the gaze area and each application window, the application window can be maintained displayed according to the first display parameters, or the application window can be displayed according to the second display parameters.

[0129] Since the power consumption of displaying according to the second display parameters is lower than that of displaying according to the first display parameters, the power consumption of the eye tracking device can be effectively reduced by flexibly adjusting the display parameters of each application window in the embodiment of the present disclosure.

[0130] For example, after obtaining the positional relationship between the gaze area and each application window, it can be determined whether the gaze area and each application window are close to each other or far away from each other. When the gaze area and the application window are close to each other, the application window is maintained in display according to the first display parameter; when the gaze area and the application window are far away, the application window is controlled to be displayed according to the second display parameter.

[0131] In the disclosed embodiment, the eye-tracking device first displays the target image according to first display parameters. Then, based on the positional relationship between the gaze area and each application window, it determines whether to adjust the display parameters of the application window, i.e., whether to maintain the application window display according to the first display parameters or adjust the application window display to the second display parameters. This allows for flexible adjustment of the CPU and GPU resources allocated to each application window, helping to reduce the power consumption of the eye-tracking device and extend its usability.

[0132] Figure 3 This is a flow chart of a display method of an eye-tracking device according to an exemplary embodiment. Figure 2 ,like Figure 3 As shown, the display method mainly includes:

[0133] In step 301, when detecting that at least two application windows are open, the eye tracking device displays a target screen including the at least two application windows;

[0134] In step 302, a gaze area of ​​a gaze object on a target image is determined;

[0135] In step 303, when it is determined that there is an application window in contact with the gaze area based on the positional relationship between the gaze area and each application window, the application window in contact with the gaze area is determined as a target window;

[0136] In step 304, display parameters of each application window are determined based on the contact area between the target window and the gaze area;

[0137] The power consumption generated by displaying application windows other than the target window is lower than the power consumption generated by displaying the target window.

[0138] It should be noted that in order to better reduce the power consumption of the eye-tracking device, after obtaining the positional relationship between the gaze area and each application window, the target window can be selected from multiple application windows based on each positional relationship, and the power consumption generated by displaying application windows other than the target window can be ensured to be lower than the power consumption generated by displaying the target window, so as to extend the usage time of the eye-tracking device.

[0139] Here, the application window that is in contact with the gaze area may be determined as a target window, and the application window that is not in contact with the gaze area may be determined as a non-target window.

[0140] For example, Figure 4 This is a second display scene of a display screen of an eye-tracking device according to an exemplary embodiment. Figure 4 As shown, the diagram 40 represents the gaze area, and 41-47 represent application windows. 40 is in the shape of a ring. Application window 41 that is in contact with 40 is the target window, and application windows 42-47 that are not in contact with 40 are non-target windows. In the diagram, the display quality of application window 41 is higher than that of application windows 42-47. Therefore, the power consumption generated by displaying application window 41 is higher than that of application windows 42-47.

[0141] It is understandable that in order to ensure that the eye movement device accurately allocates rendering resources (such as CPU resources and GPU resources), the contact area between the target window and the gaze area can be calculated, the display parameters of each application window can be determined, and then the rendering resources can be allocated based on the determined display parameters.

[0142] Here, the higher the display parameter, the more rendering resources the eye tracking device accurately allocates, and the lower the display parameter, the fewer rendering resources the eye tracking device accurately allocates.

[0143] In this disclosed embodiment, the application window in contact with the gaze area is identified as the target window. By obtaining the contact area between the target window and the gaze area, the display parameters of each application window are determined, thereby accurately obtaining display parameters and improving the user experience. At the same time, the power consumption generated by displaying application windows other than the target window is ensured to be lower than that generated by displaying the target window, thereby reducing the power consumption of the eye tracking device and extending its usage time.

[0144] In some embodiments, determining display parameters of each application window based on the contact area between the target window and the gaze area includes:

[0145] When the contact area is smaller than the preset area threshold, displaying the target window according to the third display parameter, and displaying application windows other than the target window according to the fourth display parameter; wherein the power consumption generated by displaying according to the third display parameter is greater than the power consumption generated by displaying according to the fourth display parameter;

[0146] When the contact area is greater than or equal to a preset area threshold, the target window is displayed according to the fifth display parameter, and application windows other than the target window are displayed according to the sixth display parameter; wherein the power consumption generated by displaying according to the fifth display parameter is greater than the power consumption generated by displaying according to the third display parameter and the sixth display parameter.

[0147] It should be noted that to ensure that the eye tracking device accurately allocates rendering resources (e.g., CPU and GPU resources), a preset area threshold can be set. After obtaining the contact area between the target window and the gaze area, this contact area can be compared with the preset area threshold to obtain a comparison result. Based on the comparison result, different display parameters are determined for the target window and non-target windows, and rendering resources are then allocated based on the determined display parameters.

[0148] Specifically, when the contact area is smaller than a preset area threshold, it can be determined that the contact area between the target window and the gaze area is small, and the target window can be displayed according to the first rendering strategy and the third display parameters, and application windows other than the target window can be displayed according to the fourth display parameters.

[0149] When the contact area is smaller than the preset area threshold, it can be determined that the contact area between the target window and the gaze area is large, then the target window can be displayed according to the second rendering strategy and the fifth display parameter, and the application windows other than the target window can be displayed according to the sixth display parameter; wherein the rendering accuracy corresponding to the second rendering strategy is higher than the rendering accuracy corresponding to the first rendering strategy.

[0150] Here, the preset area threshold can be set arbitrarily, and the embodiment of the present disclosure does not limit this.

[0151] Since when the contact area is smaller than the preset area threshold, the rendering resources (such as CPU resources and GPU resources) consumed by the display parameters of the target window are greater than the rendering resources (such as CPU resources and GPU resources) consumed by the display parameters of the non-target window, the power consumption generated by displaying according to the third display parameters is greater than the power consumption generated by displaying according to the fourth display parameters.

[0152] However, since the display parameters of the target window consume more rendering resources when the contact area is greater than or equal to the preset area threshold than the rendering resources consumed when the display parameters of the target window are less than the preset area threshold, the power consumption generated by displaying according to the fifth display parameter is greater than the power consumption generated by displaying according to the third display parameter and the sixth display parameter.

[0153] In the disclosed embodiment, after obtaining the contact area between the target window and the gaze area, the contact area is compared with a preset area threshold. Based on the different comparison results, the display parameters of the target window and the display parameters of application windows other than the target window are determined, thereby improving the accuracy of the display parameter determination. At the same time, the power consumption generated by displaying application windows other than the target window is ensured to be lower than the power consumption generated by displaying the target window, thereby improving the user's viewing experience while reducing the power consumption of the eye tracking device.

[0154] In some embodiments, there are at least two target windows;

[0155] The power consumption corresponding to the display parameters of each target window is positively correlated with the contact area between each target window and the gaze area.

[0156] It should be noted that in order to better reduce the power consumption of the eye tracking device, when there are at least two target windows, the display parameters of each target window can be adjusted according to the contact area between each target window and the gaze area, so that the sum of the power consumption corresponding to the display parameters of all target windows shows a downward trend, thereby extending the usage time of the eye tracking device.

[0157] Here, the power consumption corresponding to the display parameters of each target window is positively correlated with the contact area between each target window and the gaze area. It can be understood that when the contact area between the target window and the gaze area is larger, the power consumption corresponding to the display parameters of the target window is larger, and when the contact area between the target window and the gaze area is smaller, the power consumption corresponding to the display parameters of the target window is smaller.

[0158] In some embodiments, when the contact area between the target window and the gaze area is large, it can be determined that the distance between the target window and the center point of the gaze area is close, and the display parameters of the target window can be set to be larger, so that the visual effect of the target window is better. When the contact area between the target window and the gaze area is small, it can be determined that the distance between the target window and the center point of the gaze area is far, and the display parameters of the target window can be set to be smaller, so that the visual effect of the target window is less optimal.

[0159] For example, Figure 5 This is a display scene 3 of a display screen of an eye-tracking device according to an exemplary embodiment. Figure 5 As shown, 51 is the gaze area, which is in the shape of a ring. 52-57 are application windows. The ring includes a first concentric circle and a second concentric circle, with the radius of the first concentric circle being smaller than the radius of the second. Target windows include application window 52, ​​which is in contact with the first concentric circle, and application windows 53 and 54, which are in contact with the second concentric circle. Non-target windows include application windows 55-57, which are not in contact with the gaze area. In the figure, the display quality of application window 52 is higher than that of application windows 53-54. Therefore, the power consumption of displaying application window 52 is higher than that of application windows 53-54.

[0160] In some embodiments, if the gaze area is a ring, the ring includes a first concentric circle and a second concentric circle, the radius of the first concentric circle is smaller than the radius of the second concentric circle, the first concentric circle represents the range of the gaze object's fine gaze point, and the second concentric circle represents the range of the gaze object's less fine gaze point. Based on the positional relationship between the ring and each application window, the application window in contact with the first concentric circle is controlled to be displayed according to high-definition display parameters, the application window in contact with the second concentric circle is controlled to be displayed according to medium-definition display parameters, and the application window not in contact with the second concentric circle is controlled to be displayed according to low-definition display parameters.

[0161] In the disclosed embodiment, since the power consumption corresponding to the display parameters of each target window is positively correlated with the contact area between each target window and the gaze area, the power consumption of the eye movement device can be reduced while ensuring the visual effect of the target window, which is conducive to improving the user experience.

[0162] In some embodiments, the method further comprises:

[0163] When a change in the positional relationship between the gaze area and the application window is detected, the display parameters of each application window are re-determined based on the changed positional relationship.

[0164] It should be noted that in order to quickly respond to changes in the user's vision, when a change in the positional relationship between the gaze area and the application window is detected, the target window can be re-determined based on the changed positional relationship, and the display parameters of each application window can be determined to enhance the user experience.

[0165] For example, Figure 6 This is a display scene 4 of a display screen of an eye-tracking device according to an exemplary embodiment. Figure 6 As shown, figure 61 is the gaze area, and the arrow 62 in the figure represents that the eye movement device detects a change in the vision of the gaze object, that is, it detects that the position of the gaze area changes to the lower left, and it can be determined that the current target window is the application window 63-65. Then, the target window can be controlled to be displayed according to the first display parameter, and the application windows other than the target window can be displayed according to the second display parameter, wherein the power consumption of displaying according to the second display parameter is lower than the power consumption of displaying according to the first display parameter.

[0166] In the embodiment of the present disclosure, when a change in the positional relationship between the gaze area and the application window is detected, the display parameters of each application window can be re-determined based on the changed positional relationship, thereby quickly responding to changes in the user's vision and improving the user experience.

[0167] In some embodiments, re-determining display parameters of each application window based on the changed positional relationship includes:

[0168] When it is determined based on the changed positional relationship that the target window is separated from the gaze area, the target window separated from the gaze area is adjusted to be displayed according to the second display parameter.

[0169] It can be understood that when it is determined that the target window is separated from the gaze area based on the changed positional relationship, it can be determined that there is no contact between the target window and the gaze area, and the target window changes to a non-target window, then the target window separated from the gaze area is adjusted to be displayed according to the second display parameters.

[0170] Since the power consumption of displaying using the second display parameters is lower, when it is determined based on the changed positional relationship that the target window is separated from the gaze area, the target window is promptly adjusted to be displayed according to the second display parameters, thereby reducing the power consumption of the eye tracking device.

[0171] In the disclosed embodiment, if the target window is determined to be separated from the gaze area based on the changed positional relationship, the target window separated from the gaze area is adjusted from being displayed according to the first display parameters to being displayed according to the second display parameters. In this way, the target window separated from the gaze area can be determined as a non-target window (i.e., not the application window being gazed at by the gaze subject), and the display parameters of the application window can be adjusted in a timely manner, thereby improving the user's viewing experience while reducing the power consumption of the eye tracking device.

[0172] In some embodiments, the method further comprises:

[0173] When a first adjustment operation on the display parameters of the application window is detected, the application window is controlled to be displayed according to the display parameters indicated by the first adjustment operation within a preset time period.

[0174] It is understandable that, considering that users may adjust the display parameters of any application window, the embodiment of the present disclosure, when detecting a first adjustment operation on the display parameters of an application window, can control the application window to be displayed according to the display parameters indicated by the first adjustment operation within a preset time period, so that the eye tracking device can quickly respond to user needs, thereby improving the user's operating experience.

[0175] Here, the user may perform the first adjustment operation on one application window, or may perform the first adjustment operation on multiple application windows, which is not limited in this embodiment of the present disclosure.

[0176] Here, the application window is controlled to be displayed according to the display parameters indicated by the first adjustment operation within a preset time period. It can be understood that the display parameters of the application window have been fixed to the display parameters indicated by the first adjustment operation, that is, the display parameters of the application window will not change with the change of the contact area between the gaze area and the application window.

[0177] In some embodiments, the preset duration can be set arbitrarily. For example, the preset duration can be 1 hour, which is not limited in the embodiments of the present disclosure.

[0178] In some embodiments, the first adjustment operation is an operation of adjusting the display parameters of the application window. For example, the first adjustment operation may be an operation of adjusting the display frame rate, display brightness, display resolution, and display clarity of the application window. This is not limited to the embodiments of the present disclosure.

[0179] In the disclosed embodiment, upon detecting a first adjustment operation on the display parameters of an application window, the application window is controlled to be displayed according to the display parameters indicated by the first adjustment operation for a preset duration. Thus, upon detecting that a user has adjusted the display parameters of any application window as required, the application window can be guaranteed to be displayed with the adjusted display parameters for the preset duration, thereby improving the user experience.

[0180] In some embodiments, the method further comprises:

[0181] When a second adjustment operation on the application window is detected, clearing display parameters of the application window;

[0182] The second adjustment operation is used to instruct to stop displaying the application window.

[0183] It is understandable that, considering that the user's demand may be to stop displaying the application window, when the second adjustment operation on the application window is detected, the display parameters of the application window can be cleared, and the CPU resources and GPU resources occupied by the application window can be released in time, thereby improving the user's operating experience while reducing the power consumption of the eye tracking device and extending the usage time of the eye tracking device.

[0184] Here, the second adjustment operation is used to instruct to stop displaying the application window. It can be understood that the second adjustment operation is a close operation or a minimize operation, so that the application window processed by the second adjustment operation stops being displayed on the display screen of the eye tracking device.

[0185] In an embodiment of the present disclosure, the second adjustment operation is used to instruct to stop displaying the application window. When the second adjustment operation for the application window is detected, the display parameters of the application window can be cleared so as to promptly release the CPU resources and GPU resources occupied by the application window, thereby reducing the power consumption of the eye movement device while ensuring the display effect of the display screen of the eye movement device.

[0186] In some embodiments, determining a gaze area of ​​a gaze object on a target screen includes:

[0187] When detecting that the eye tracking device is in an on state, determining a gaze area of ​​the gaze object on the target image based on the acquired gaze information of the gaze object;

[0188] The gaze information includes gaze direction and / or gaze position.

[0189] It is understandable that in order to improve the user experience, when the eye movement device is detected to be in the turned-on state, the eye movement device can automatically perform the eye tracking function, that is, it can quickly obtain the gaze information of the gaze object and determine the gaze area of ​​the gaze object on the target screen, wherein the gaze information includes: gaze direction and / or gaze position.

[0190] Here, the gaze area of ​​the gaze object can be obtained through methods such as eye tracking, infrared pupil reflection tracking, and iris tracking of the video image segmentation method, which is not limited in the embodiments of the present disclosure.

[0191] In some embodiments, various electronic and optical detection methods can be used to obtain the current gaze direction of the gaze subject. Based on the primary viewing direction, the gaze area of ​​the gaze subject on the target image is determined. Certain eye structures and features whose relative positions remain unchanged during eye movement are used as reference features. Gaze change parameters are extracted between the eye position change features and the reference features. The gaze direction is then determined using a geometric model or a pre-trained mapping model. Finally, based on the mapping relationship between the gaze direction and the gaze, the coordinates of the pixel points on the target image corresponding to that gaze direction are obtained, and the area corresponding to the pixel points is determined as the gaze area.

[0192] In other embodiments, considering that different background lights may cause changes in the eyes (for example, pupil scaling, changes in pupil center position), making eye tracking inaccurate, the current pupil center and the current pupil radius can be obtained based on the current background light of the display screen of the eye tracking device to further determine the gaze position; and then, based on the gaze position, the gaze area of ​​the gazed object on the target screen can be determined.

[0193] Specifically, when the display screen of the eye tracking device is under current background light information, the current pupil center and current pupil radius of the eyeball of the gaze subject are obtained; a first gaze position corresponding to the current pupil center position is determined on the target image; and based on an offset mapping relationship, a current offset corresponding to the current pupil radius and the first gaze position is obtained, wherein the offset mapping relationship includes a correspondence between multiple offsets, multiple pupil radii, and the positions of multiple gaze areas. Based on the current pupil center position and the current offset, a target pupil center position is obtained; a second gaze position corresponding to the target pupil center position is determined on the target image, and the second gaze position is determined as the gaze area under the current background light information.

[0194] In an embodiment of the present disclosure, when it is detected that the eye movement device is in the turned-on state, the gaze area of ​​the gaze object on the target screen is determined based on the gaze information of the gaze object obtained, thereby improving the accuracy of determining the gaze area and laying the foundation for adjusting the display parameters of each application window.

[0195] In some embodiments, the method further comprises:

[0196] When it is detected that the eye tracking device has entered the off state, the gaze information and the display parameters of each application window are cleared.

[0197] It is understandable that when it is detected that the eye-tracking device has entered the off state, it can be determined that the user's experience of the eye-tracking device has ended, and the gaze information obtained by the eye-tracking device and the display parameters of each application window can be automatically cleared.

[0198] In an embodiment of the present disclosure, when it is detected that the eye-tracking device has entered a closed state, the gaze information and the display parameters of each application window are cleared to release the CPU resources and GPU resources consumed during the current use of the eye-tracking device, thereby laying the foundation for the next use of the eye-tracking device.

[0199] Figure 7 is a block diagram of a display device according to an exemplary embodiment. Figure 7 As shown, the device 700 includes:

[0200] The display module 701 is configured to, when detecting that at least two application windows are opened, cause the eye tracking device to display a target screen including the at least two application windows;

[0201] A first determining module 702 is configured to determine a gaze area of ​​a gaze object on the target image;

[0202] A second determining module 703 is configured to determine display parameters of each application window based on a positional relationship between the gaze area and each application window;

[0203] Different positional relationships correspond to different display parameters, and the power consumption generated by displaying according to different display parameters is different.

[0204] In some embodiments, the display module 701 is configured as follows:

[0205] The eye-tracking device displays the target image according to the first display parameter;

[0206] The second determining module 703 is configured to:

[0207] Based on the positional relationship between the gaze area and each of the application windows, maintaining the application window to be displayed according to the first display parameter, or adjusting the application window to be displayed according to the second display parameter;

[0208] The power consumption of displaying according to the second display parameter is lower than the power consumption of displaying according to the first display parameter.

[0209] In some embodiments, the second determining module 703 includes:

[0210] A first submodule is configured to, when it is determined based on the positional relationship that there is an application window in contact with the gaze area, determine the application window in contact with the gaze area as a target window;

[0211] a second submodule configured to determine display parameters of each of the application windows based on a contact area between the target window and the gaze area;

[0212] The power consumption generated by displaying the application windows other than the target window is lower than the power consumption generated by displaying the target window.

[0213] In some embodiments, the second submodule is configured as follows:

[0214] When the contact area is smaller than a preset area threshold, displaying the target window according to a third display parameter, and displaying the application windows other than the target window according to a fourth display parameter; wherein power consumption generated by displaying according to the third display parameter is greater than power consumption generated by displaying according to the fourth display parameter;

[0215] When the contact area is greater than or equal to the preset area threshold, the target window is displayed according to the fifth display parameter, and the application windows other than the target window are displayed according to the sixth display parameter; wherein the power consumption generated by displaying according to the fifth display parameter is greater than the power consumption generated by displaying according to the third display parameter and the sixth display parameter.

[0216] In some embodiments, there are at least two target windows;

[0217] The power consumption corresponding to the display parameters of each target window is positively correlated with the contact area between each target window and the gaze area.

[0218] In some embodiments, the apparatus 700 further includes:

[0219] The third determining module is configured to, when detecting that a positional relationship between the gaze area and the application window has changed, re-determine display parameters of each application window based on the changed positional relationship.

[0220] In some embodiments, the third determining module is configured to:

[0221] In a case where it is determined based on the changed positional relationship that the target window is separated from the gaze area, the target window separated from the gaze area is adjusted to be displayed according to the second display parameter.

[0222] In some embodiments, the apparatus 700 further includes:

[0223] The control module is configured to, when a first adjustment operation on the display parameters of the application window is detected, control the application window to be displayed according to the display parameters indicated by the first adjustment operation within a preset time period.

[0224] In some embodiments, the apparatus further comprises:

[0225] a first clearing module configured to clear the display parameters of the application window when a second adjustment operation on the application window is detected;

[0226] The second adjustment operation is used to instruct to stop displaying the application window.

[0227] In some embodiments, the apparatus 700 further includes:

[0228] a fourth determining module configured to, when detecting that the eye-tracking device is in an on state, determine a gaze area of ​​the gaze object on the target image based on the acquired gaze information of the gaze object;

[0229] The gaze information includes: gaze direction and / or gaze position.

[0230] In some embodiments, the apparatus 700 further includes:

[0231] The second clearing module is configured to clear the gaze information and the display parameters of each application window when detecting that the eye movement device enters a closed state.

[0232] Regarding the apparatus in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0233] Figure 8 8 is a block diagram showing a structure of an electronic device 800 according to an exemplary embodiment. For example, the device 800 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0234] Reference Figure 8, the device 800 may include one or more of the following components: a processing component 802 , a memory 804 , a power component 806 , a multimedia component 808 , an audio component 810 , an input / output (I / O) interface 812 , a sensor component 814 , and a communication component 816 .

[0235] The processing component 802 generally controls the overall operation of the device 800, such as operations associated with at least one of display, phone calls, data communications, camera operation, and recording operations. The processing component 802 may include one or more processors 820 to execute instructions to perform all or part of the steps of the above-described method. In addition, the processing component 802 may include one or more modules to facilitate interaction between the processing component 802 and other components. For example, the processing component 802 may include a multimedia module to facilitate interaction between the multimedia component 808 and the processing component 802.

[0236] The memory 804 is configured to store various types of data to support operations on the device 800. Examples of such data include at least one of the following: instructions for any application or method operating on the device 800, contact data, phone book data, messages, pictures, and videos. The memory 804 can be implemented by any type of volatile or non-volatile storage device, or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk.

[0237] The power supply component 806 provides power to the various components of the device 800. The power supply component 806 can include at least one of the following: a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the device 800.

[0238] The multimedia component 808 includes a screen that provides an output interface between the device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, slides, and gestures on the touch panel. The touch sensor can not only sense the boundaries of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 808 includes a front camera and / or a rear camera. When the device 800 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera can receive external multimedia data. Each front camera and rear camera can be a fixed optical lens system or have focal length and optical zoom capabilities.

[0239] The audio component 810 is configured to output and / or input audio signals. For example, the audio component 810 includes a microphone (MIC), which is configured to receive external audio signals when the device 800 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signal can be further stored in the memory 804 or transmitted via the communication component 816. In some embodiments, the audio component 810 also includes a speaker for outputting audio signals.

[0240] I / O interface 812 provides an interface between processing component 802 and peripheral interface modules, such as a keyboard, click wheel, and buttons. These buttons may include, but are not limited to, a home button, volume buttons, a start button, and a lock button.

[0241] The sensor assembly 814 includes one or more sensors for providing various aspects of the status assessment of the device 800. For example, the sensor assembly 814 can detect the open / closed state of the device 800, the relative positioning of components, such as the display and keypad of the device 800. The sensor assembly 814 can also detect changes in the position of the device 800 or a component thereof, the presence or absence of user contact with the device 800, the orientation or acceleration / deceleration of the device 800, and changes in the temperature of the device 800. The sensor assembly 814 can include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 814 can also include an optical sensor, such as a complementary metal oxide semiconductor (CMOS) or charge coupled device (CCD) image sensor, for use in imaging applications. In some embodiments, the sensor assembly 814 can also include, but is not limited to, at least one of the following: an accelerometer, a gyroscope, a magnetic sensor, a pressure sensor, and a temperature sensor.

[0242] The communication component 816 is configured to facilitate wired or wireless communication between the device 800 and other devices. The device 800 can access a wireless network based on a communication standard, such as Wi-Fi, 4G, 5G, or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0243] In an exemplary embodiment, the device 800 can be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components.

[0244] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 804 including executable instructions or a computer program. The instructions or computer program can be executed by the processor 820 of the apparatus 800 to perform the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a compact disc read-only memory (CD-ROM), a magnetic tape, a floppy disk, an optical data storage device, etc.

[0245] A non-transitory computer-readable storage medium, when the instructions in the storage medium are executed by a processor of a mobile terminal, enables the mobile terminal to perform any of the above-mentioned display methods of the embodiments of the present disclosure. For example, the method includes:

[0246] When detecting that at least two application windows are opened, the eye tracking device displays a target screen including the at least two application windows;

[0247] Determining a gaze area of ​​a gaze object on a target image;

[0248] Determining display parameters of each application window based on a positional relationship between the gaze area and each application window;

[0249] Different positional relationships correspond to different display parameters, and the power consumption generated by displaying according to different display parameters is different.

[0250] The present disclosure provides a computer program product comprising a computer program or executable instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer program or executable instructions from the computer-readable storage medium and executes the computer program or executable instructions, causing the computer device to perform any of the display methods described above in the present disclosure.

[0251] Figure 9 1 is a block diagram of a device 900 for display according to an exemplary embodiment. For example, the device 900 can be provided as a server. Figure 9 The apparatus 900 includes a processing component 922, which further includes one or more processors, and a memory resource represented by a memory 932 for storing instructions executable by the processing component 922, such as an application. The application stored in the memory 932 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 922 is configured to execute the instructions to perform the above-mentioned display method:

[0252] When detecting that at least two application windows are opened, the eye tracking device displays a target screen including the at least two application windows;

[0253] Determining a gaze area of ​​a gaze object on a target image;

[0254] Determining display parameters of each application window based on a positional relationship between the gaze area and each application window;

[0255] Different positional relationships correspond to different display parameters, and the power consumption generated by displaying according to different display parameters is different.

[0256] The device 900 may also include a power supply component 926 configured to perform power management of the device 900, a wired or wireless network interface 950 configured to connect the device 900 to a network, and an input / output (I / O) interface 958. The device 900 may operate an operating system stored in the memory 932, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, or the like.

[0257] Other embodiments of the present disclosure will readily occur to those skilled in the art after considering the specification and practicing the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, with the true scope and spirit of the present disclosure being indicated by the following claims.

[0258] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes can be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A display method for an eye-movement device, characterized in that: The method comprises: When detecting that at least two application windows are opened, the eye-tracking device displays a target screen including the at least two application windows; Determine a gaze area of ​​a gaze object on the target image; determining display parameters of each of the application windows based on a positional relationship between the gaze area and each of the application windows; Different positional relationships correspond to different display parameters, and the power consumption generated by displaying according to different display parameters is different.

2. The method according to claim 1, characterized in that The eye-tracking device displays a target screen including the at least two application windows, including: The eye-tracking device displays the target image according to the first display parameter; The determining, based on the positional relationship between the gaze area and each of the application windows, display parameters of each of the application windows includes: Based on the positional relationship between the gaze area and each of the application windows, maintaining the application window to be displayed according to the first display parameter, or adjusting the application window to be displayed according to the second display parameter; The power consumption of displaying according to the second display parameter is lower than the power consumption of displaying according to the first display parameter.

3. The method according to claim 1, characterized in that The determining, based on the positional relationship between the gaze area and each of the application windows, display parameters of each of the application windows includes: In a case where it is determined based on the positional relationship that there is an application window in contact with the gaze area, determining the application window in contact with the gaze area as a target window; determining display parameters of each of the application windows based on a contact area between the target window and the gaze area; The power consumption generated by displaying the application windows other than the target window is lower than the power consumption generated by displaying the target window.

4. The method according to claim 3, characterized in that The determining, based on the contact area between the target window and the gaze area, display parameters of each application window includes: When the contact area is smaller than a preset area threshold, displaying the target window according to a third display parameter, and displaying the application windows other than the target window according to a fourth display parameter; wherein power consumption generated by displaying according to the third display parameter is greater than power consumption generated by displaying according to the fourth display parameter; When the contact area is greater than or equal to the preset area threshold, the target window is displayed according to the fifth display parameter, and the application windows other than the target window are displayed according to the sixth display parameter; wherein the power consumption generated by displaying according to the fifth display parameter is greater than the power consumption generated by displaying according to the third display parameter and the sixth display parameter.

5. The method according to claim 3, characterized in that There are at least two target windows; The power consumption corresponding to the display parameters of each target window is positively correlated with the contact area between each target window and the gaze area.

6. The method according to any one of claims 1 to 5, characterized in that The method further comprises: When it is detected that the positional relationship between the gaze area and the application window changes, the display parameters of each application window are re-determined based on the changed positional relationship.

7. The method according to claim 6, characterized in that The re-determining the display parameters of each of the application windows based on the changed positional relationship includes: In a case where it is determined based on the changed positional relationship that the target window is separated from the gaze area, the target window separated from the gaze area is adjusted to be displayed according to the second display parameter.

8. The method according to any one of claims 1 to 5, characterized in that The method further comprises: When a first adjustment operation on the display parameters of the application window is detected, the application window is controlled to be displayed according to the display parameters indicated by the first adjustment operation within a preset time period.

9. The method according to any one of claims 1 to 5, characterized in that The method further comprises: When a second adjustment operation on the application window is detected, clearing display parameters of the application window; The second adjustment operation is used to instruct to stop displaying the application window.

10. The method according to any one of claims 1 to 5, characterized in that The determining the gaze area of ​​the gaze object on the target picture includes: When detecting that the eye-tracking device is in an on state, determining a gaze area of ​​the gaze object on the target image based on the acquired gaze information of the gaze object; The gaze information includes: gaze direction and / or gaze position.

11. The method according to claim 10, characterized in that The method further comprises: When it is detected that the eye movement device enters the closed state, the gaze information and the display parameters of each application window are cleared.

12. A display device, characterized in that: The device comprises: A display module configured to, when detecting that at least two application windows are opened, cause the eye-tracking device to display a target screen including the at least two application windows; A first determining module is configured to determine a gaze area of ​​a gaze object on the target image; a second determining module configured to determine display parameters of each of the application windows based on a positional relationship between the gaze area and each of the application windows; Different positional relationships correspond to different display parameters, and the power consumption generated by displaying according to different display parameters is different.

13. An electronic device, characterized in that: include: processor; memory for storing computer programs or instructions; The processor executes the computer program or instructions to implement the steps of the method according to any one of claims 1 to 11.

14. A non-transitory computer-readable storage medium storing a computer program or instruction, characterized in that: When the computer program or instructions in the storage medium are executed by a processor, the steps of the method according to any one of claims 1 to 11 are implemented.

15. A computer program product comprising a computer program or instructions, characterized in that When the computer program or instruction is executed by a processor, the steps of the method according to any one of claims 1 to 11 are implemented.

Images