Message display method and electronic device

Abstract

A message display method and electronic device. In this method, when a message window needs to be displayed, it is directly displayed around the user's eye gaze position in the display area. Implementing the technical solution provided by this application can display the message window based on the user's eye gaze position, improving the efficiency of message notifications.

Description

Technical Field

[0001] The present application relates to the field of terminals and communication technologies, and in particular to a message display method and an electronic device. Background Art

[0002] Message display is a very important function in electronic devices. It can provide users with the information they need in a timely manner, prompt users on how to operate, or inform users of the current progress.

[0003] Current message windows are generally displayed in a fixed position. For example, information notification windows are generally displayed in a fixed top area of an electronic device display, and feedback prompt windows are generally displayed in a fixed bottom area of an electronic device display.

[0004] However, the message window is displayed in a fixed area or even a fixed position, which often causes users to fail to notice it. Or when the user changes their viewing angle to view the message, the message window has disappeared. This reduces the efficiency of message notifications and may require users to call up the message list to view it, which increases the number of steps for users to view messages. Summary of the Invention

[0005] The present application provides a message display method and an electronic device for displaying a message window according to the user's eye gaze position, thereby improving the efficiency of message notification.

[0006] In a first aspect, the present application provides a message display method, which includes: an electronic device determines that there is a message window that needs to be displayed; the electronic device determines in real time the current eye gaze position of the user in the display area of the electronic device; the electronic device determines a first preset visual area based on the current eye gaze position, and the first preset visual area is around the current eye gaze position on the display area; the electronic device displays the message window in the first preset visual area.

[0007] In the above embodiment, when the electronic device determines that there is a message window that needs to be displayed, it determines a first preset visual area around the user's current eye gaze position in the display area according to the user's current eye gaze position, and displays the message window in the first preset visual area, so that the message window is directly displayed around the user's current eye gaze position, so that the user can see the message window at the first time, thereby improving the efficiency of message notification.

[0008] In combination with some embodiments of the first aspect, in some embodiments, the first preset visual area is on the display area and the center or starting position is within a first preset distance from the current eye gaze position, and the size of the first preset visual area is not less than the size of the message window.

[0009] In the above embodiment, the first preset visual area is located on the display area and its center or starting position is within a first preset distance from the current eye gaze position, thereby ensuring that the first preset visual area is located around the user's current eye gaze position. The size of the first preset visual area is no smaller than the size of the message window, so that the first preset visual area can be used to display the message window.

[0010] In combination with some embodiments of the first aspect, in some embodiments, the electronic device determines a first preset visual area based on the current eye gaze position, specifically including: the electronic device determines a preset display orientation and a preset display distance that match the relative position based on the relative position of the current eye gaze position on the display area; the electronic device determines that the center or starting position of the first preset visual area is at the preset display distance of the preset display orientation of the current eye gaze position.

[0011] In the above embodiment, the preset display orientation and preset display distance matching the relative position of the current eye gaze position on the display area can be determined, and then the position of the first preset visual area can be determined, so that the determination of the first preset visual area is more in line with the needs of the customer's current scene.

[0012] In combination with some embodiments of the first aspect, in some embodiments, the electronic device determines a first preset visual area based on the current eye gaze position, specifically including: the electronic device determines whether there is an area with no displayed content and not less than the size of the message window within a range of a first preset distance from the current eye gaze position on the display area; when it is determined that there is an area with no displayed content and not less than the size of the message window, the electronic device determines that the area is the first preset visual area.

[0013] In the above embodiment, the electronic device preferentially determines the area in the display area where no content is displayed as the first preset visual area, thereby avoiding blocking of the content displayed in the display area.

[0014] In combination with some embodiments of the first aspect, in some embodiments, the electronic device determines the movement speed of the user's current eye gaze position in the display area of the electronic device.

[0015] In the above embodiment, the electronic device can also determine the moving speed of the current eye gaze position, thereby determining the display of the message window based on the current eye gaze position and its speed, thereby determining the current state of the user and making the display of the message window more humane.

[0016] In combination with some embodiments of the first aspect, in some embodiments, the method may also include: the electronic device determines whether the moving speed of the current eye gaze position is not greater than a preset first speed value; the electronic device displays the message window in the first preset visual area, specifically including: when the electronic device determines that the moving speed of the current eye gaze position is not greater than the preset first speed value, the electronic device displays the message window in the first preset visual area.

[0017] In the above embodiment, the electronic device displays the message window directly in the first preset visual area around the current eye gaze position only when the movement speed of the current eye gaze position is no greater than a preset first speed value. In this case, the movement speed of the user's current eye gaze position is not high, and the displayed message window can be easily seen by the user.

[0018] In combination with some embodiments of the first aspect, in some embodiments, the method may further include: when the electronic device determines that the moving speed of the current eye gaze position is greater than a preset first speed value, the electronic device determines whether the average moving speed of the user's eye gaze position within the preset first time period is not greater than a preset second speed value; when the electronic device determines that the average moving speed of the user's eye gaze position within the preset first time period is not greater than the preset second speed value, the electronic device displays the message window in a first preset visual area determined according to the current eye gaze position; when the electronic device determines that the average moving speed of the user's eye gaze position within the preset first time period is greater than the preset second speed value, the electronic device displays the message window in the first preset visual area determined according to the current eye gaze position, and moves the message window following the movement of the user's current eye gaze position.

[0019] In the above embodiment, when the movement speed of the current eye gaze position is greater than a preset first speed value, it indicates that the movement speed of the eye gaze position at this time is relatively high. The electronic device can determine whether the average movement speed of the user's eye gaze position within a subsequent preset first time period is not greater than a preset second speed value. When the average speed is not greater than the preset second speed value, it indicates that the user can still see the message window after the message window is displayed. The electronic device can determine the first preset visual area based on the current eye gaze position and display the message window. When the average speed is greater than the preset second speed value, it indicates that the movement speed of the user's eye gaze position is relatively fast, and the user may not be able to see the message window if only the message window is displayed. Therefore, after the electronic device determines the first preset visual area based on the current eye gaze position and displays the message window, it can move the message window according to the movement of the user's current eye gaze position, thereby making the message window more easily visible to users with a higher eye gaze movement speed.

[0020] In combination with some embodiments of the first aspect, in some embodiments, the method also includes: when the electronic device determines that the moving speed of the current eye gaze position is greater than a preset first speed value, the electronic device determines whether the cumulative distance of the user's eye gaze position moving within the preset fourth time period exceeds the preset cumulative distance; when the electronic device determines that the cumulative distance of the user's eye gaze position moving within the preset fourth time period does not exceed the preset cumulative distance, the electronic device displays the message window in the first preset visual area determined according to the current eye gaze position; when the electronic device determines that the cumulative distance of the user's eye gaze position moving within the preset fourth time period exceeds the preset cumulative distance, the electronic device displays the message window in the first preset visual area determined according to the current eye gaze position, and moves the message window following the movement of the user's current eye gaze position.

[0021] In the above embodiment, since the speed of the user's eye gaze position movement is unstable, whether the average speed of the user's eye gaze position movement exceeds a preset value can be determined by determining whether the cumulative distance the user's eye gaze position moves within the preset fourth time period exceeds a preset cumulative distance. Determining the cumulative distance of the eye gaze position within a time period can more accurately reflect the average speed of the user's eye gaze position, rather than directly determining the average speed within that time period, and thus can more accurately determine whether the message window needs to move in accordance with the movement of the user's eye gaze position.

[0022] In combination with some embodiments of the first aspect, in some embodiments, the method also includes: the electronic device determines whether the user's current eye gaze position moves into the area of the message window within a preset second time period after the message window is displayed; if the user's current eye gaze position moves into the area of the message window at a moment within the preset second time period, the electronic device stops moving the message window.

[0023] In the above embodiment, if the user's current eye gaze moves into the area of the message window at some point within the preset second time period after the message window is displayed, it means that the user has seen the message window. The electronic device can stop moving the message window to avoid affecting the user's subsequent operations.

[0024] In combination with some embodiments of the first aspect, in some embodiments, the method also includes: if the user's eye gaze position does not move into the area of the message window within the preset second time period, the electronic device determines whether the movement distance of the user's eye gaze position exceeds the preset second distance within the preset third time period after the message window is displayed; if the movement distance of the user's eye gaze position does not exceed the preset second distance within the preset third time period, the electronic device stops moving the message window.

[0025] In the above embodiment, if the user does not see the message window within the preset second duration, and the user's eye gaze position does not move beyond the preset second distance within the preset third duration, it indicates that the user does not want to see the message window. The electronic device may then stop moving the message window to prevent it from affecting the user's subsequent operations.

[0026] In combination with some embodiments of the first aspect, in some embodiments, the method also includes: the electronic device determines whether the user's eye gaze position moves into the area of the message window within a preset second time period after the message window is displayed; when the electronic device determines that the user's eye gaze position has not moved into the area of the message window within the preset second time period, and the movement distance of the user's eye gaze position exceeds the preset second distance within a preset third time period after the message window is displayed, the electronic device displays the message window in the first preset visual area determined according to the current eye gaze position, or the electronic device executes the step of the electronic device determining whether the average movement speed of the user's eye gaze position within the preset first time period is not greater than the preset second speed value.

[0027] In the above embodiment, if the user does not see the message window within the preset second duration, and the user's eye gaze position moves beyond the preset second distance within the preset third duration, this indicates that the user's eye gaze position has significantly fluctuated within the preset third duration. Therefore, the user may not have seen the message window. The electronic device can directly move the message window to the first preset visual area around the user's current eye gaze position. Alternatively, the electronic device can re-perform the aforementioned average speed determination process and display the message window accordingly based on the determination result.

[0028] In combination with some embodiments of the first aspect, in some embodiments, the method also includes: the electronic device determines whether the user's eye gaze position moves into the area of the message window within a preset second time period after the message window is displayed; when the electronic device determines that the user's eye gaze position has not moved into the area of the message window within the preset second time period, and the movement distance of the user's eye gaze position exceeds the preset second distance within a preset third time period after the message window is displayed, the electronic device displays the message window in the first preset visual area determined according to the current eye gaze position, or the electronic device executes the step of the electronic device determining whether the cumulative distance of the user's eye gaze position movement within the preset fourth time period exceeds the preset cumulative distance.

[0029] In the above embodiment, if the user does not see the message window within the preset second duration, and the user's eye gaze position moves beyond the preset second distance within the preset third duration, this indicates that the user's eye gaze position has significantly fluctuated within the preset third duration. Therefore, the user may not have seen the message window. The electronic device can directly move the message window to the first preset visual area around the user's current eye gaze position. Alternatively, the electronic device can re-perform the aforementioned cumulative distance determination process and display the message window accordingly based on the determination result.

[0030] In a second aspect, an embodiment of the present application provides an electronic device, which includes: one or more processors and a memory; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code includes computer instructions, and the one or more processors call the computer instructions to enable the electronic device to execute: determining that there is a message window that needs to be displayed; determining in real time the current eye gaze position of the user in the display area of the electronic device; determining a first preset visual area based on the current eye gaze position, the first preset visual area being around the current eye gaze position on the display area; and displaying the message window in the first preset visual area.

[0031] In the above embodiment, when the electronic device determines that there is a message window that needs to be displayed, it determines a first preset visual area around the user's current eye gaze position in the display area according to the user's current eye gaze position, and displays the message window in the first preset visual area, so that the message window is directly displayed around the user's current eye gaze position, so that the user can see the message window at the first time, thereby improving the efficiency of message notification.

[0032] In combination with some embodiments of the second aspect, in some embodiments, the first preset visual area is on the display area and the center or starting position is within a first preset distance from the current eye gaze position, and the size of the first preset visual area is not less than the size of the message window.

[0033] In combination with some embodiments of the second aspect, in some embodiments, the one or more processors are specifically used to call the computer instructions to enable the electronic device to execute: determining a preset display orientation and a preset display distance that match the relative position based on the relative position of the current eye gaze position on the display area; and determining that the center or starting position of the first preset visual area is at the preset display distance of the preset display orientation of the current eye gaze position.

[0034] In combination with some embodiments of the second aspect, in some embodiments, the one or more processors are specifically used to call the computer instructions to cause the electronic device to execute: determining whether there is an area with no displayed content and not less than the size of the message window within a first preset distance from the current eye gaze position on the display area; when it is determined that there is an area with no displayed content and not less than the size of the message window, determining that the area is the first preset visual area.

[0035] In combination with some embodiments of the second aspect, in some embodiments, the one or more processors are also used to call the computer instructions to enable the electronic device to execute: determining the movement speed of the user's current eye gaze position in the display area of the electronic device.

[0036] In combination with some embodiments of the second aspect, in some embodiments, the one or more processors are also used to call the computer instructions so that the electronic device executes: determining whether the moving speed of the current eye gaze position is not greater than a preset first speed value; the one or more processors are specifically used to call the computer instructions so that the electronic device executes: when it is determined that the moving speed of the current eye gaze position is not greater than the preset first speed value, displaying the message window in the first preset visual area.

[0037] In combination with some embodiments of the second aspect, in some embodiments, the one or more processors are also used to call the computer instructions to enable the electronic device to execute: when it is determined that the moving speed of the current eye gaze position is greater than a preset first speed value, determining whether the average moving speed of the user's eye gaze position within the preset first time period is not greater than a preset second speed value; when it is determined that the average moving speed of the user's eye gaze position within the preset first time period is not greater than the preset second speed value, displaying the message window in a first preset visual area determined according to the current eye gaze position; when it is determined that the average moving speed of the user's eye gaze position within the preset first time period is greater than the preset second speed value, displaying the message window in the first preset visual area determined according to the current eye gaze position, and moving the message window to follow the movement of the user's current eye gaze position.

[0038] In combination with some embodiments of the second aspect, in some embodiments, the one or more processors are further used to call the computer instructions to cause the electronic device to execute: when it is determined that the moving speed of the current eye gaze position is greater than a preset first speed value, determining whether the cumulative distance of the user's eye gaze position movement within the preset fourth time period exceeds the preset cumulative distance; when it is determined that the cumulative distance of the user's eye gaze position movement within the preset fourth time period does not exceed the preset cumulative distance, displaying the message window in the first preset visual area determined according to the current eye gaze position; when it is determined that the cumulative distance of the user's eye gaze position movement within the preset fourth time period exceeds the preset cumulative distance, displaying the message window in the first preset visual area determined according to the current eye gaze position, and moving the message window to follow the movement of the user's current eye gaze position.

[0039] In combination with some embodiments of the second aspect, in some embodiments, the one or more processors are also used to call the computer instructions to enable the electronic device to execute: determining whether the user's current eye gaze position moves into the area of the message window within a preset second time period after the message window is displayed; if the user's current eye gaze position moves into the area of the message window at a moment within the preset second time period, the electronic device stops moving the message window.

[0040] In combination with some embodiments of the second aspect, in some embodiments, the one or more processors are also used to call the computer instructions to enable the electronic device to execute: if the user's eye gaze position does not move into the area of the message window within the preset second time period, determine whether the movement distance of the user's eye gaze position exceeds the preset second distance within the preset third time period after the message window is displayed; if the movement distance of the user's eye gaze position does not exceed the preset second distance within the preset third time period, stop moving the message window.

[0041] In combination with some embodiments of the second aspect, in some embodiments, the one or more processors are also used to call the computer instructions to enable the electronic device to execute: determining whether the user's eye gaze position moves into the area of the message window within the preset second time period after the message window is displayed; when it is determined that the user's eye gaze position has not moved into the area of the message window within the preset second time period, and the movement distance of the user's eye gaze position exceeds the preset second distance within the preset third time period after the message window is displayed, displaying the message window in the first preset visual area determined according to the current eye gaze position, or executing the step of determining whether the average movement speed of the user's eye gaze position within the preset first time period is not greater than the preset second speed value.

[0042] In combination with some embodiments of the second aspect, in some embodiments, the one or more processors are also used to call the computer instructions to enable the electronic device to execute: determining whether the user's eye gaze position moves into the area of the message window within the preset second time period after the message window is displayed; when it is determined that the user's eye gaze position has not moved into the area of the message window within the preset second time period, and the movement distance of the user's eye gaze position exceeds the preset second distance within the preset third time period after the message window is displayed, displaying the message window in the first preset visual area determined according to the current eye gaze position, or executing the step of determining whether the cumulative distance of the user's eye gaze position movement within the preset fourth time period exceeds the preset cumulative distance.

[0043] In a third aspect, an embodiment of the present application provides a chip system, which is applied to an electronic device, and the chip system includes one or more processors, which are used to call computer instructions to enable the electronic device to execute the method described in the first aspect and any possible implementation method of the first aspect.

[0044] It is understood that the chip system may include a Figure 17 The processor 110 in the electronic device 100 shown may also include multiple Figure 17 The processor 110 in the electronic device 100 shown may also include one or more other chips in the chip system, which is not limited here.

[0045] In a fourth aspect, an embodiment of the present application provides a computer program product comprising instructions, which, when the computer program product is run on an electronic device, enables the electronic device to execute the method described in the first aspect and any possible implementation of the first aspect.

[0046] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium comprising instructions. When the instructions are executed on an electronic device, the electronic device executes the method described in the first aspect and any possible implementation of the first aspect.

[0047] It is understandable that the electronic device provided in the second aspect, the chip system provided in the third aspect, the computer program product provided in the fourth aspect, and the computer storage medium provided in the fifth aspect are all used to execute the methods provided in the embodiments of the present application. Therefore, the beneficial effects that can be achieved can be referred to the beneficial effects of the corresponding methods and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 This is a schematic diagram of a scenario of the eye tracking technology in an embodiment of the present application;

[0049] Figure 2This is a schematic diagram of the effect of the eyeball reflecting infrared light in an embodiment of the present application;

[0050] Figure 3 This is another schematic diagram of the effect of the eyeball reflecting infrared light in an embodiment of the present application;

[0051] Figure 4 This is a flowchart of the eye tracking technology in the embodiment of the present application;

[0052] Figure 5 This is an exemplary diagram of a message window in an embodiment of the present application;

[0053] Figure 6 is another exemplary schematic diagram of the message window in an embodiment of the present application;

[0054] Figure 7 is another exemplary schematic diagram of the message window in an embodiment of the present application;

[0055] Figure 8 is another exemplary schematic diagram of the message window in an embodiment of the present application;

[0056] Figure 9 is another exemplary schematic diagram of the message window in an embodiment of the present application;

[0057] Figure 10 is an exemplary schematic diagram of a display area of an electronic device in an embodiment of the present application;

[0058] Figure 11 is another exemplary schematic diagram of the display area of the electronic device in an embodiment of the present application;

[0059] Figure 12 is another exemplary schematic diagram of the display area of the electronic device in an embodiment of the present application;

[0060] Figure 13 is another exemplary schematic diagram of the display area of the electronic device in an embodiment of the present application;

[0061] Figure 14 is another exemplary schematic diagram of the display area of the electronic device in an embodiment of the present application;

[0062] Figure 15 This is an exemplary schematic diagram of the first preset area in the embodiment of the present application;

[0063] Figure 16 This is a schematic diagram of an exemplary scenario of the message display method in an embodiment of the present application;

[0064] Figure 17 is a schematic structural diagram of an exemplary electronic device 100 provided in an embodiment of the present application;

[0065] Figure 18 is a software structure block diagram of an exemplary electronic device 100 provided in an embodiment of the present application;

[0066] Figure 19 This is a flow chart of a message display method in an embodiment of the present application;

[0067] Figure 20 This is an exemplary diagram of the user interface in the embodiment of the present application;

[0068] Figure 21 This is another exemplary schematic diagram of the user interface in the embodiment of the present application;

[0069] Figure 22a This is another exemplary schematic diagram of the user interface in the embodiment of the present application;

[0070] Figure 22b This is another exemplary schematic diagram of the user interface in the embodiment of the present application;

[0071] Figure 23 This is a set of exemplary schematic diagrams of user interfaces in the embodiments of the present application;

[0072] Figure 24 2 is a schematic structural diagram of an exemplary electronic device 200 provided in an embodiment of the present application. DETAILED DESCRIPTION

[0073] The terms used in the following examples of the present application are for the purpose of describing specific embodiments only and are not intended to limit the present application. As used in the specification and appended claims of the present application, the singular expressions "a," "an," "said," "above," "the," and "this" are intended to include plural expressions as well, unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used in the present application refers to and encompasses any or all possible combinations of one or more of the listed items.

[0074] In the following, the terms "first" and "second" are used for descriptive purposes only and should not be understood to imply or suggest relative importance or implicitly indicate the number of the technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of this application, unless otherwise specified, "plurality" means two or more.

[0075] To facilitate understanding, the following first introduces the relevant terms and concepts involved in the embodiments of this application.

[0076] (1) Eye tracking technology:

[0077] There have been many studies on eye tracking technology. Currently, the common implementation principles of eye tracking technology are as follows:

[0078] Principle 1: Tracking based on changes in the features of the eyeball and its surroundings;

[0079] Principle 2: Tracking based on iris angle changes;

[0080] Principle 3: Actively project infrared light beams onto the iris to extract features.

[0081] These implementations are based on the fact that when a person looks in different directions, their eyes undergo subtle changes. These changes generate extractable features, which electronic devices can extract through image capture or scanning, thereby tracking eye changes in real time and determining the eye's gaze position.

[0082] The main equipment for eye tracking technology can include infrared devices, image acquisition devices, or even cameras on general electronic devices. Eye tracking can also be achieved with the support of software.

[0083] Generally, eye tracking based on Principle 1 or Principle 2 is not as accurate as eye tracking based on Principle 3. The following is a brief description of eye tracking technology based on Principle 3:

[0084] For example, Figure 1 The following is a schematic diagram of a scene of the eye tracking technology in the embodiment of the present application. The eye tracking technology according to Principle 3 requires the hardware to include an infrared light emitter and an infrared camera. Figure 1 As shown in FIG, the electronic device is equipped with three infrared light emitters and one infrared camera. Figure 1 As shown by the dotted arrow in the middle, when the infrared light emitter emits infrared light and is reflected by the human eye, the video of the emitted infrared light will be captured by the infrared camera.

[0085] like Figure 2 The figure below is a schematic diagram of the effect of infrared light reflected by the eyeball in an embodiment of the present application. Each time the user looks at a position on the display area, the eyeball reflects infrared light emitted by infrared light emitters at different positions on the electronic device at different reflection points. Therefore, for each eye, the electronic device can obtain a set of reflection point locations of infrared light reflected by the user's pupil and cornea.

[0086] like Figure 3Figure 2 shows another schematic diagram of the effect of infrared light reflected by the eyeball in an embodiment of the present application. When the user's eyes focus on another location on the display area, the electronic device can obtain another set of reflection points of infrared light reflected by the user's pupil and cornea for each eye. Because the user is now focusing on a different location, the angle of the eyeball will change slightly compared to the previous one. The reflection points of infrared light reflected by the user's pupil and cornea will shift compared to the previous one.

[0087] The electronic device can process the infrared light reflected by the human eye to obtain the eye gaze position on the display area of the electronic device. Figure 4 As shown, it is a flowchart of the eye tracking technology in the embodiment of this application:

[0088] S401, an infrared light emitter in the electronic device emits infrared light;

[0089] S402, an infrared camera in an electronic device captures a video of a human eye;

[0090] S403: The electronic device processes the human eye video to obtain the positions of the user's pupil and corneal reflection points;

[0091] S404: The electronic device determines the sight direction according to the offset between the pupil and the corneal reflection point, and ultimately determines the eye gaze position on the display area.

[0092] Since how to implement eye tracking technology is not the focus of the embodiments of this application, the embodiments of this application will not elaborate on the specific algorithm.

[0093] In the embodiments of the present application, the eye tracking technology used is not limited to the principle. As long as the eye tracking technology used can ultimately determine the user's eye gaze position on the display area, no limitation is made here.

[0094] Generally, after an electronic device determines the eye gaze position using eye tracking technology, it can output the coordinates of the eye gaze position on the display area. It is understood that the reference coordinate system of the coordinates can be the default coordinate system of the electronic device or another coordinate system. If the reference coordinate system of the output coordinates is another coordinate system, the electronic device can also convert them to the default coordinate system of the electronic device, which is not limited here.

[0095] (2) Message window:

[0096] The message window in the embodiment of the present application may include various windows in the electronic device that provide messages to the user, such as a message window for information notification, a message window for feedback prompts, etc., which are not limited here.

[0097] For example, a message window for information notification may include notification messages from various applications or systems. The message window for information notification may appear when a window of an application is displayed on the display area of the electronic device, or may appear when the main interface window of the system is displayed on the display area of the electronic device, without limitation.

[0098] Figure 5 1 is an exemplary diagram of a message window in an embodiment of the present application. The display area 501 of the electronic device displays a window of an application. At this time, a message window 502 can be displayed at the top of the display area.

[0099] The message window 502 may include an application identification area 502A and a content area 502B.

[0100] In the application identification area 502A, an indicator of which application and / or source the message window originated from may be displayed. For example, the indicator may include "Mom" or "Text message." The indicator is not limited to text information and may also be an icon, which is not limited here.

[0101] The content area 502B may display information notification content, for example, "Go home quickly for dinner."

[0102] Figure 6 This is another exemplary diagram of a message window in an embodiment of the present application. The display area of an electronic device displays the system's main interface window 601. At this point, a message window 602 may also be displayed at the top of the display area. Message window 602 may include an application identification area 602A and a content area 602B. For details, please refer to application identification area 502A and content area 502B, which will not be further described here.

[0103] Exemplarily, the message window for feedback prompts may include at least the following message windows:

[0104] 1. Toast:

[0105] Toast is a lightweight feedback prompt that often appears in the form of a subtitle reminder. It usually appears automatically within 1 to 2 seconds.

[0106] Figure 7This is another exemplary diagram of a message window in an embodiment of the present application. After the electronic device is powered on, the network signal connected to it is poor and it is not connected to a Wi-Fi network. At this point, the system's main interface window 701 is displayed in the display area of the electronic device. A message window 702 may pop up at the bottom of the display area. Message window 702 contains a line of subtitles stating "Network connection unavailable, please try again later." This allows the user to promptly obtain the network connection status of the electronic device for timely processing.

[0107] 2. Snackbar:

[0108] Snackbar inherits all the features of toast, that is, it is a small pop-up window that disappears automatically.

[0109] But there are several differences:

[0110] ①. 0 to 1 operations can appear in the message window, excluding the cancel button;

[0111] ②. Click outside the message window and the Snackbar will disappear;

[0112] ③. It usually appears at the bottom of the display area of the electronic device by default.

[0113] Figure 8 This is another exemplary schematic diagram of a message window in an embodiment of the present application. A photo application management window 701 is displayed in the display area of the electronic device. When a user deletes a photo, the photo application may pop up a snackbar message window 802 in the display area of the electronic device. This message window 802 may display the message "Photo successfully deleted" to prompt the user that the photo has been successfully deleted. A clickable "Undo" button may also be displayed to prompt the user that they can click the Undo button to undo the deletion operation and restore the deleted photo.

[0114] 3. Dialog:

[0115] Dialog is a feedback prompt in the form of a dialog box. Dialog usually requires user action (click OK or Close) before it disappears.

[0116] Figure 9This is another exemplary schematic diagram of the message window in the embodiment of the present application. The display area of the electronic device displays the main interface window 901 of the system. When the user logs in to the system account of the electronic device on the PC web page, a Dialog message window 902 may pop up in the display area of the electronic device. The message window 902 may display the content "Log in to the system account on the web page" to prompt the user that someone has logged in to the system account of the electronic device on the web page. A clickable "Cancel" button may also be displayed to prompt the user that the user can click the cancel button to fail the login on the web page; and a "Confirm" button may be displayed to prompt the user that the user can click the confirmation button to successfully log in on the web page.

[0117] It is understandable that, in addition to the various types of message windows exemplified above, the message windows in the embodiments of the present application may also include various other types of message windows for sending messages or prompts to users, which are not limited here.

[0118] (3) Display area of electronic equipment:

[0119] The electronic device in the embodiments of the present application can be any electronic device that supports eye tracking technology and has a display area, for example, it can be a mobile electronic device, a personal computer electronic device, an in-vehicle electronic device, a virtual reality (VR) electronic device, or an augmented reality (AR) electronic device, etc., without limitation here.

[0120] For example, Figure 5 Shown may be a display area of a mobile electronic device.

[0121] For example, Figure 10 FIG. 1 is another exemplary schematic diagram of the display area of an electronic device in an embodiment of the present application. It may be a display area of a mobile electronic device with a folding screen. Figure 10 As shown, the display area of the electronic device can be composed of two foldable display areas, screen A and screen B. When screen A and screen B are folded, they can each serve as an independent display area. When screen A and screen B are unfolded, they can together form a larger display area.

[0122] For example, Figure 11 FIG. 1 is another exemplary schematic diagram of the display area of the electronic device in the embodiment of the present application. Figure 11 The display area 1101 in the figure may be a display area of a personal computer electronic device.

[0123] For example, Figure 12FIG. 1 is another exemplary schematic diagram of the display area of the electronic device in the embodiment of the present application. Figure 12 The display area 1201 in the figure may be a display area of an in-vehicle electronic device.

[0124] For example, Figure 13 FIG. 1 is another exemplary schematic diagram of the display area of the electronic device in the embodiment of the present application. Figure 13 The display area 1301 in the VR electronic device may be a display area of a VR electronic device. The display area of the VR electronic device will only display the screen displayed by the electronic device, and will not display the real screen. Figure 13 As shown, although the VR electronic device worn by user 1303 is facing the direction of a scene 1304 in which two people are moving, the real-life scene 1304 of two people moving does not appear in display area 1301. It is understandable that although the image of the VR electronic device is displayed on the two lenses 1302A and 1302B of the VR electronic device, the image ultimately seen by user 1303 is display area 1301. Therefore, in this example, the display area of the electronic device in the embodiment of the present application is display area 1301.

[0125] For example, Figure 14 FIG. 1 is another exemplary schematic diagram of the display area of the electronic device in the embodiment of the present application. Figure 14 The display area 1401 in the figure may be a display area of an AR electronic device. The display area of the AR electronic device may not only display the screen displayed by the electronic device, but also display the screen in reality. Figure 14 As shown, the AR electronic device worn by user 1403 is facing a scene 1404 of two people moving. Therefore, the real-life scene 1404 of two people moving appears in display area 1401. It is understandable that although the image of the AR electronic device is displayed on the two lenses 1402A and 1402B of the AR electronic device, the image ultimately seen by user 1403 is display area 1401. Therefore, in this example, the display area of the electronic device in the embodiment of the present application is display area 1401.

[0126] It is understandable that there are many different display areas of electronic devices, which are not limited here. Figure 5 The display area of the electronic device shown is used as an example to describe the message display method in the embodiment of the present application. The process and type of executing the message display method on electronic devices with other types of display areas are not described in detail in the embodiment of the present application.

[0127] (4) First preset visual area:

[0128] In the embodiment of the present application, the message window to be displayed is preferentially displayed in a first preset visual area. The first preset visual area is around the current eye gaze position on the display area of the electronic device.

[0129] Specifically, the first preset visual area is on the display area of the electronic device and its center or starting position is within a first preset distance from the current eye gaze position. The size of the first preset visual area is not less than the size of the message window to be displayed.

[0130] Preferably, the first preset distance is less than half the width of the display area. Optionally, the first preset distance is less than 2 / 3 the width of the display area. The first preset distance can also be other values, which can be factory preset or user-defined, and are not limited here.

[0131] The first preset visual area can vary depending on the size of the message window to be displayed. In some embodiments, the size and shape of the first preset visual area can be the same as the size and shape of the message window to be displayed. In some embodiments, the size of the first preset visual area can be larger than the size of the message window to be displayed. This is not limited here.

[0132] The position of the first preset visual area will change according to the change of the current eye gaze position.

[0133] In some embodiments, the center of the first preset visual area may coincide with the current eye gaze position.

[0134] In some embodiments, the center of the first preset visual area does not coincide with the current eye gaze position. Depending on the position of the current eye gaze position on the display area of the electronic device, the first preset visual area may be at a different orientation relative to the eye gaze position.

[0135] Specifically, the electronic device can determine the preset display orientation and preset display distance that match the relative position of the current eye gaze position on the display area; the electronic device can determine that the center or starting position of the first preset visual area is at the preset display distance of the preset display orientation of the eye gaze position.

[0136] Exemplarily, the relative position may include being within a preset first left side distance from the left side of the display area of the electronic device. In this case, the matching preset display orientation may be to the right of the current eye gaze position, and the matching preset display distance may be a preset second left side distance. The preset second left side distance may be greater than the preset first left side distance but less than the first preset distance.

[0137] Exemplarily, the relative position may include being within a preset first upper edge distance from the upper edge of the display area of the electronic device. In this case, the matching preset display orientation may be below the current eye gaze position, and the matching preset display distance may be a preset second upper edge distance. The preset second upper edge distance may be greater than the preset first upper edge distance but less than the first preset distance.

[0138] Exemplarily, the relative position may include being within a preset first center distance from the center of the display area of the electronic device. In this case, the matching preset display orientation may be in a direction opposite to the direction of movement of the current eye gaze position, and the matching preset display distance may be a preset second center distance. The preset second center distance may be greater than the preset first center distance but less than the first preset distance.

[0139] It is understandable that there may be many relative positions of the current eye gaze position on the display area, as well as preset display orientations and preset display distances of the first preset visual area that match the relative positions. For example, the eye gaze position is closer to the right side of the display area, the eye gaze position is closer to the lower side of the display area, the eye gaze position is closer to the upper left corner of the display area, etc. The specific settings can be based on actual conditions and needs, and are not limited here.

[0140] In some embodiments, the first preset area is determined preferentially in an area in the display area of the electronic device where no content is displayed. If there is no area in the display area where no content is displayed, the first preset area can be determined in other ways, which are not limited here.

[0141] For example, Figure 15 FIG. 1 is an exemplary diagram of the first preset area in an embodiment of the present application. The first preset area 1502 is located around the eye gaze position 1501 on the display area 1505 , and the distance 1503 between the center of the first preset area and the eye gaze position 1501 is less than the first preset distance 1504 .

[0142] It can be understood that as long as the center or starting position of the first preset visual area is within the first preset distance from the current eye gaze position, there can be many other preset ways to determine the orientation of the first preset visual area relative to the current eye gaze position, which are not limited here.

[0143] In the prior art, when an electronic device has a message window that needs to be displayed, the message window is generally displayed in a fixed position. Figure 5 As shown, the message window 502 is fixedly displayed at the top of the display area 501. Figure 8 As shown, the message window 802 is fixedly displayed at the bottom of the display area 801 .

[0144] Displaying the message window in a fixed position like this can cause the user to be looking elsewhere in the display area and not notice the message window, resulting in the user not receiving the message in a timely manner. Alternatively, the user may need to turn their head and shift their gaze to see the message window, interrupting their current tasks. If the message window is not displayed, the user must also call up the message list to see the message, which increases the number of steps required for the user to view the message.

[0145] In an embodiment of the present application, when there is a message window that needs to be displayed in the electronic device, the message window is directly displayed around the user's eye gaze position in the display area. Figure 16 This is a schematic diagram of an exemplary scenario of the message display method in the embodiment of the present application. Figure 16 As shown in (a) of FIG, the current eye gaze position of the user is a first eye gaze position 1601. If the electronic device needs to display a message window at this time, the electronic device directly displays the message window 1602 around the first eye gaze position 1601.

[0146] Because the message window is displayed directly around the user's current eye gaze, the user can see the message window directly without having to shift their viewing angle. This allows users to quickly and conveniently receive notifications, not only without affecting the user's current tasks but also improving the efficiency of message notifications.

[0147] The following first introduces an exemplary electronic device 100 provided in an embodiment of the present application.

[0148] Figure 17 1 is a schematic structural diagram of an electronic device 100 provided in an embodiment of the present application.

[0149] The following embodiments are described in detail using electronic device 100 as an example. It should be understood that electronic device 100 may have more or fewer components than shown in the figure, may combine two or more components, or may have a different component configuration. The various components shown in the figure may be implemented in hardware, including one or more signal processing and / or application-specific integrated circuits, software, or a combination of hardware and software.

[0150] The electronic device 100 may include: a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, a subscriber identification module (SIM) card interface 195, and an infrared light emitter 196, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0151] It should be understood that the structure illustrated in the embodiments of the present invention does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown, or may combine or separate certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0152] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). The different processing units may be independent devices or integrated into one or more processors.

[0153] The controller may be the nerve center and command center of the electronic device 100. The controller may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.

[0154] Processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in processor 110 is a cache memory. This memory can store instructions or data that have just been used or are being recycled by processor 110. If processor 110 needs to use the same instruction or data again, it can directly access the memory. This avoids duplicate accesses, reduces processor 110 latency, and thus improves system efficiency.

[0155] In some embodiments, the processor 110 may include one or more interfaces. The interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface.

[0156] The I2C interface is a bidirectional synchronous serial bus that includes a serial data line (SDA) and a serial clock line (SCL). In some embodiments, processor 110 may include multiple I2C busses. Processor 110 may be coupled to touch sensor 180K, a charger, a flash, camera 193, and the like via different I2C bus interfaces. For example, processor 110 may be coupled to touch sensor 180K via an I2C interface, enabling communication between processor 110 and touch sensor 180K via the I2C bus interface, thereby enabling touch functionality in electronic device 100.

[0157] The I2S interface can be used for audio communication. In some embodiments, the processor 110 can include multiple I2S buses. The processor 110 can be coupled to the audio module 170 via the I2S bus to enable communication between the processor 110 and the audio module 170. In some embodiments, the audio module 170 can transmit audio signals to the wireless communication module 160 via the I2S interface, enabling the function of answering calls through a Bluetooth headset.

[0158] The PCM interface can also be used for audio communication, sampling, quantizing, and encoding analog signals. In some embodiments, the audio module 170 and the wireless communication module 160 can be coupled via a PCM bus interface. In some embodiments, the audio module 170 can also transmit audio signals to the wireless communication module 160 via the PCM interface, enabling the function of answering calls via a Bluetooth headset. Both the I2S interface and the PCM interface can be used for audio communication.

[0159] The UART interface is a universal serial data bus used for asynchronous communication. This bus can be a bidirectional communication bus. It converts the data to be transmitted between serial communication and parallel communication. In some embodiments, the UART interface is typically used to connect the processor 110 and the wireless communication module 160. For example, the processor 110 communicates with the Bluetooth module in the wireless communication module 160 via the UART interface to implement Bluetooth functionality. In some embodiments, the audio module 170 can transmit audio signals to the wireless communication module 160 via the UART interface, enabling the function of playing music through Bluetooth headphones.

[0160] The MIPI interface can be used to connect the processor 110 to peripheral devices such as the display 194 and the camera 193. MIPI interfaces include the camera serial interface (CSI) and the display serial interface (DSI). In some embodiments, the processor 110 and the camera 193 communicate via the CSI interface to enable the camera function of the electronic device 100. The processor 110 and the display 194 communicate via the DSI interface to enable the display function of the electronic device 100.

[0161] The GPIO interface can be configured via software. The GPIO interface can be configured as either a control signal or a data signal. In some embodiments, the GPIO interface can be used to connect the processor 110 to the camera 193, display 194, wireless communication module 160, audio module 170, sensor module 180, etc. The GPIO interface can also be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, etc.

[0162] The SIM interface can be used to communicate with the SIM card interface 195 to implement the function of transmitting data to the SIM card or reading data in the SIM card.

[0163] The USB interface 130 is an interface that complies with USB standards and may be a Mini USB interface, a Micro USB interface, a USB Type-C interface, or the like. The USB interface 130 can be used to connect a charger to charge the electronic device 100, or to transfer data between the electronic device 100 and peripheral devices. It can also be used to connect headphones to play audio. This interface can also be used to connect other electronic devices, such as augmented reality devices.

[0164] It is understood that the interface connection relationship between the modules illustrated in the embodiment of the present invention is merely an illustrative illustration and does not constitute a structural limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection methods from the above embodiments, or a combination of multiple interface connection methods.

[0165] The charging management module 140 is configured to receive charging input from a charger, which may be a wireless charger or a wired charger.

[0166] The power management module 141 is used to connect the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140 to provide power to the processor 110, the internal memory 121, the external memory, the display 194, the camera 193, and the wireless communication module 160.

[0167] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0168] Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve antenna utilization. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In other embodiments, the antennas can be used in conjunction with a tuning switch.

[0169] The mobile communication module 150 can provide wireless communication solutions, including 2G / 3G / 4G / 5G, for the electronic device 100. The mobile communication module 150 can include at least one filter, a switch, a power amplifier, a low-noise amplifier (LNA), and the like. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, filter and amplify the received electromagnetic waves, and transmit them to the modem processor for demodulation. The mobile communication module 150 can also amplify the signals modulated by the modem processor and convert them into electromagnetic waves for radiation via the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be located in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be located in the same device as at least some of the modules of the processor 110.

[0170] The modem processor may include a modulator and a demodulator. The modulator is used to modulate the low-frequency baseband signal to be transmitted into a medium- or high-frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After processing by the baseband processor, the low-frequency baseband signal is passed to the application processor. The application processor outputs audio signals through an audio device (including but not limited to the speaker 170A, the receiver 170B, etc.) or displays images or videos on the display 194. In some embodiments, the modem processor may be a standalone device. In other embodiments, the modem processor may be independent of the processor 110 and may be located in the same device as the mobile communication module 150 or other functional modules.

[0171] The wireless communication module 160 can provide wireless communication solutions applied to the electronic device 100, including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, frequency modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110. The wireless communication module 160 can also receive signals to be transmitted from the processor 110, frequency modulate them, amplify them, and convert them into electromagnetic waves for radiation through the antenna 2.

[0172] In some embodiments, antenna 1 of electronic device 100 is coupled to mobile communication module 150, and antenna 2 is coupled to wireless communication module 160, so that electronic device 100 can communicate with a network and other devices via wireless communication technologies. The wireless communication technologies may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), Bluetooth, GNSS, WLAN, NFC, FM, and / or IR technology. The GNSS may include global positioning system (GPS), global navigation satellite system (GLONASS), Beidou navigation satellite system (BDS), quasi-zenith satellite system (QZSS), and / or satellite-based augmentation system (SBAS).

[0173] Electronic device 100 implements display functionality through a GPU, display screen 194, and an application processor. A GPU is a microprocessor for image processing that connects display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs that execute program instructions to generate or modify display information.

[0174] Display screen 194 is used to display images, videos, and the like. Display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a MiniLED, a MicroLED, a Micro-oLed, or a quantum dot light-emitting diode (QLED). In some embodiments, electronic device 100 may include one or N display screens 194, where N is a positive integer greater than one.

[0175] It can be understood that the display screen 194 of the exemplary electronic device 100 provides a display area in the embodiment of the present application.

[0176] The electronic device 100 can implement a shooting function through an ISP, a camera 193, a video codec, a GPU, a display screen 194, and an application processor.

[0177] The ISP processes data fed back by camera 193. For example, when taking a photo, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, which is then passed to the ISP for processing and transformed into a visible image. The ISP can also perform algorithmic optimization for image noise, brightness, and skin tone. It can also optimize parameters such as exposure and color temperature of the captured scene. In some embodiments, the ISP can be located within camera 193.

[0178] The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, and then passes the electrical signal to the ISP for conversion into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the electronic device 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.

[0179] In some embodiments, the electronic device 100 may include 1 or N infrared cameras, which can capture videos containing infrared light.

[0180] Infrared light emitter 196 is used to emit infrared light. In some embodiments, the infrared light emitted by infrared light emitter 196 is reflected by the user's eyeball. Electronic device 100 may include 1 or N infrared light emitters 196, where N is a positive integer greater than 1.

[0181] In the embodiment of the present application, the infrared camera and the infrared light emitter 196 in the electronic device 100 can constitute the hardware basis for implementing the eye tracking technology using the principle 3 in the above-mentioned term introduction (1) eye tracking technology.

[0182] In practical applications, if the eye tracking technology is implemented by adopting the principle 1 or principle 2 in the eye tracking technology described in the above terminology (1), the electronic device 100 may or may not include an infrared camera and an infrared light emitter 196, which is not limited here.

[0183] The digital signal processor is used to process digital signals. In addition to processing digital image signals, it can also process other digital signals. For example, when the electronic device 100 selects a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

[0184] Video codecs are used to compress or decompress digital video. Electronic device 100 may support one or more video codecs. This allows electronic device 100 to play or record videos in various encoding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, and MPEG4.

[0185] The NPU is a neural-network (NN) computing processor. Drawing on the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, it rapidly processes input information and can continuously self-learn. The NPU enables intelligent cognitive applications in electronic device 100, such as image recognition, face recognition, speech recognition, and text comprehension.

[0186] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 via the external memory interface 120 to implement data storage functions. For example, files such as music and videos can be stored on the external memory card.

[0187] The internal memory 121 can be used to store computer executable program codes, which include instructions. The processor 110 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 121. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area can store an operating system, applications required for at least one function (such as face recognition function, fingerprint recognition function, mobile payment function, etc.), etc. The data storage area can store data created during the use of the electronic device 100 (such as face information template data, fingerprint information template, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0188] The electronic device 100 can implement audio functions such as music playback and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0189] The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 170 can also be used to encode and decode audio signals. In some embodiments, the audio module 170 can be provided in the processor 110, or some functional modules of the audio module 170 can be provided in the processor 110.

[0190] The speaker 170A, also called a "speaker", is used to convert audio electrical signals into sound signals. The electronic device 100 can listen to music or listen to hands-free calls through the speaker 170A.

[0191] The receiver 170B, also called a "handset", is used to convert audio electrical signals into sound signals. When the electronic device 100 receives a call or a voice message, the user can place the receiver 170B close to the ear to hear the voice.

[0192] Microphone 170C, also known as "microphone" or "microphone", is used to convert sound signals into electrical signals. When making a call or sending a voice message, the user can speak by putting their mouth close to the microphone 170C to input the sound signal into the microphone 170C. The electronic device 100 can be provided with at least one microphone 170C. In other embodiments, the electronic device 100 can be provided with two microphones 170C, which can not only collect sound signals but also realize noise reduction function. In other embodiments, the electronic device 100 can also be provided with three, four or more microphones 170C to collect sound signals, reduce noise, identify the source of sound, realize directional recording function, etc.

[0193] The headphone jack 170D is used to connect a wired headphone and can be the USB interface 130 or a 3.5mm open mobile terminal platform (OMTP) standard interface or a cellular telecommunications industry association of the USA (CTIA) standard interface.

[0194] Pressure sensor 180A is used to sense pressure signals and convert them into electrical signals. In some embodiments, pressure sensor 180A can be located on display screen 194. There are many types of pressure sensors 180A, such as resistive, inductive, and capacitive. A capacitive pressure sensor can include at least two parallel plates made of conductive material. When force acts on pressure sensor 180A, the capacitance between the electrodes changes. Electronic device 100 determines the intensity of the pressure based on this change in capacitance. When a touch operation is applied to display screen 194, electronic device 100 detects the touch intensity based on pressure sensor 180A. Electronic device 100 can also calculate the touch location based on the detection signal from pressure sensor 180A. In some embodiments, touch operations applied to the same touch location but with different touch intensities can correspond to different operation instructions. For example, when a touch operation with an intensity less than a first pressure threshold is applied to a short message application icon, a command to view short messages is executed. When a touch operation with an intensity greater than or equal to the first pressure threshold is applied to a short message application icon, a command to create a new short message is executed.

[0195] The gyroscope sensor 180B can be used to determine the motion posture of the electronic device 100. In some embodiments, the angular velocity of the electronic device 100 around three axes (i.e., x, y, and z axes) can be determined by the gyroscope sensor 180B. The gyroscope sensor 180B can be used for anti-shake shooting. For example, when the shutter is pressed, the gyroscope sensor 180B detects the angle of the electronic device 100 shaking, calculates the distance that the lens module needs to compensate based on the angle, and allows the lens to offset the shaking of the electronic device 100 through reverse movement to achieve anti-shake. The gyroscope sensor 180B can also be used for navigation and somatosensory game scenes.

[0196] The air pressure sensor 180C is used to measure air pressure. In some embodiments, the electronic device 100 calculates the altitude using the air pressure value measured by the air pressure sensor 180C to assist in positioning and navigation.

[0197] The magnetic sensor 180D includes a Hall sensor. The electronic device 100 can use the magnetic sensor 180D to detect the opening and closing of the flip case. In some embodiments, when the electronic device 100 is a flip phone, the electronic device 100 can detect the opening and closing of the flip cover based on the magnetic sensor 180D. Based on the detected opening and closing status of the case or flip cover, features such as automatic unlocking of the flip cover can be configured.

[0198] Accelerometer 180E can detect the magnitude of acceleration of electronic device 100 in all directions (generally three axes). When electronic device 100 is stationary, it can detect the magnitude and direction of gravity. It can also be used to identify the electronic device's posture, enabling applications such as switching between landscape and portrait modes and pedometers.

[0199] The distance sensor 180F is used to measure distance. The electronic device 100 can measure distance using infrared or laser. In some embodiments, when shooting a scene, the electronic device 100 can use the distance sensor 180F to measure distance to achieve fast focusing.

[0200] The proximity light sensor 180G may include, for example, a light emitting diode (LED) and a light detector, such as a photodiode. The light emitting diode may be an infrared light emitting diode. The electronic device 100 emits infrared light outward through the light emitting diode. The electronic device 100 uses a photodiode to detect infrared reflected light from nearby objects. When sufficient reflected light is detected, it can be determined that there is an object near the electronic device 100. When insufficient reflected light is detected, the electronic device 100 can determine that there is no object near the electronic device 100. The electronic device 100 can use the proximity light sensor 180G to detect that the user is holding the electronic device 100 close to the ear to talk, so as to automatically turn off the screen to save power. The proximity light sensor 180G can also be used in leather case mode and pocket mode to automatically unlock and lock the screen.

[0201] Ambient light sensor 180L is used to sense ambient light brightness. Electronic device 100 can adaptively adjust the brightness of display screen 194 based on the perceived ambient light. Ambient light sensor 180L can also be used to automatically adjust white balance when taking photos. Ambient light sensor 180L can also work with proximity light sensor 180G to detect whether electronic device 100 is in a pocket to prevent accidental touches.

[0202] The fingerprint sensor 180H is used to collect fingerprints. The electronic device 100 can use the collected fingerprint characteristics to implement fingerprint unlocking, access application locks, fingerprint photography, fingerprint call answering, etc.

[0203] The temperature sensor 180J is used to detect temperature. In some embodiments, the electronic device 100 uses the temperature detected by the temperature sensor 180J to execute a temperature processing strategy. For example, when the temperature reported by the temperature sensor 180J exceeds a threshold, the electronic device 100 reduces the performance of the processor located near the temperature sensor 180J to reduce power consumption and implement thermal protection. In other embodiments, when the temperature is lower than another threshold, the electronic device 100 heats the battery 142 to prevent the electronic device 100 from shutting down abnormally due to low temperature. In other embodiments, when the temperature is lower than another threshold, the electronic device 100 boosts the output voltage of the battery 142 to prevent abnormal shutdown due to low temperature.

[0204] The touch sensor 180K is also called a "touch panel." The touch sensor 180K can be disposed on the display screen 194. The touch sensor 180K and the display screen 194 form a touch screen, also called a "touch screen." The touch sensor 180K is used to detect touch operations applied thereto or in the vicinity thereof. The touch sensor can transmit the detected touch operations to the application processor to determine the type of touch event. Visual output related to the touch operations can be provided via the display screen 194. In other embodiments, the touch sensor 180K can also be disposed on the surface of the electronic device 100, in a location different from that of the display screen 194.

[0205] The buttons 190 include a power button, a volume button, and the like. The buttons 190 may be mechanical buttons or touch buttons. The electronic device 100 may receive key inputs and generate key signal inputs related to user settings and function control of the electronic device 100.

[0206] Motor 191 can generate vibration alerts. This can be used for incoming call vibration alerts and touch vibration feedback. For example, touch operations applied to different applications (such as taking photos, playing audio, etc.) can correspond to different vibration feedback effects. Motor 191 can also generate different vibration feedback effects for touch operations applied to different areas of the display 194. Different application scenarios (such as time reminders, receiving messages, alarm clocks, games, etc.) can also correspond to different vibration feedback effects. Touch vibration feedback effects can also be customized.

[0207] The indicator 192 may be an indicator light, which may be used to indicate the charging status, power level changes, messages, missed calls, notifications, etc.

[0208] The SIM card interface 195 is used to connect a SIM card. The SIM card can be connected to and disconnected from the electronic device 100 by inserting it into or removing it from the SIM card interface 195. The electronic device 100 can support 1 or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 195 can support Nano SIM cards, Micro SIM cards, SIM cards, and the like. Multiple cards can be inserted into the same SIM card interface 195 at the same time. The types of the multiple cards can be the same or different. The SIM card interface 195 can also be compatible with different types of SIM cards. The SIM card interface 195 can also be compatible with external memory cards. The electronic device 100 interacts with the network through the SIM card to implement functions such as calls and data communications.

[0209] Figure 18 It is a software structure block diagram of the electronic device 100 according to an embodiment of the present application.

[0210] A layered architecture divides software into several layers, each with distinct roles and responsibilities. Layers communicate with each other through software interfaces. In some embodiments, the system is divided into four layers: application layer, application framework layer, runtime and system libraries, and kernel layer, from top to bottom.

[0211] The application layer can include a series of application packages.

[0212] like Figure 18 As shown, the application package may include message notification module, camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, short message and other applications (also referred to as applications).

[0213] Among them, the message notification module is used to determine the user's eye gaze position from the eye gaze position calculation module, determine the message window to be displayed from the notification manager, and execute the message display method in the embodiment of the present application according to the user's eye gaze position.

[0214] The application framework layer provides an application programming interface (API) and programming framework for applications in the application layer. The application framework layer includes some predefined functions.

[0215] like Figure 18 As shown, the application framework layer may include a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager, an eye gaze position calculation module, and the like.

[0216] The window manager is used to manage window programs. The window manager can obtain the display size, determine whether there is a status bar, lock the screen, take screenshots, etc.

[0217] Content providers are used to store and retrieve data and make it accessible to applications. The data may include videos, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.

[0218] The view system includes visual controls, such as those for displaying text and images. The view system is used to build applications. A display interface can consist of one or more views. For example, a display interface containing a text notification icon might include a view for displaying text and a view for displaying images.

[0219] The phone manager is used to provide communication functions of the electronic device 100, such as management of call status (including answering, hanging up, etc.).

[0220] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0221] The Notification Manager allows applications to display notifications in the status bar. These messages can be displayed briefly and then disappear automatically, without requiring user interaction. For example, the Notification Manager can be used to notify users of completed downloads, message reminders, and so on. The Notification Manager can also display notifications in the top status bar of the system as icons or scrolling text, such as notifications from background applications, or as dialog interfaces on the screen. Examples include text messages in the status bar, beeps, vibrations on electronic devices, and flashing indicator lights.

[0222] In some embodiments, when the message notification module is running, the notification manager no longer notifies the message notification module of messages that the message window is displayed.

[0223] In some embodiments, the message notification module may be located not in the application layer but in the application framework layer, which is not limited here.

[0224] In some embodiments, the message notification module is a submodule in the notification manager, which is not limited here.

[0225] The eye gaze position calculation module is used to obtain real-time video of the user's eye movement from the camera driver of the kernel layer, and calculate the user's eye gaze position on the display screen in real time based on eye tracking technology.

[0226] The eye gaze position calculation module can provide an eye gaze position interface, and the control movement module of the application layer can obtain the user's eye gaze position on the display screen in real time by calling the eye gaze position interface.

[0227] The runtime includes the core library and the virtual machine. The runtime is responsible for system scheduling and management.

[0228] The core library consists of two parts: one is the function that the programming language (for example, Java language) needs to call, and the other is the core library of the system.

[0229] The application layer and application framework layer run in a virtual machine. The virtual machine executes application layer and application framework layer programming files (for example, Java files) as binary files. The virtual machine manages object lifecycles, stack management, thread management, security and exception management, and garbage collection.

[0230] The system library can include multiple functional modules, such as the surface manager, media libraries, 3D graphics processing library (such as OpenGL ES), and 2D graphics engine (such as SGL).

[0231] The surface manager is used to manage the display subsystem and provides the fusion of two-dimensional (2D) and three-dimensional (3D) layers for multiple applications.

[0232] The media library supports playback and recording of a variety of common audio and video formats, as well as static image files. The media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

[0233] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0234] A 2D graphics engine is a drawing engine for 2D drawings.

[0235] The kernel layer is the layer between hardware and software. The kernel layer includes at least a display driver, a camera driver, an audio driver, a sensor driver, and a virtual card driver. In some embodiments, the kernel layer may also include an infrared light emitter driver.

[0236] The following describes the workflow of the software and hardware of the electronic device 100 in conjunction with capturing a photo scene.

[0237] When touch sensor 180K receives a touch operation, a corresponding hardware interrupt is sent to the kernel layer. The kernel layer processes the touch operation into a raw input event (including touch coordinates, touch operation timestamp, and other information). The raw input event is stored in the kernel layer. The application framework layer obtains the raw input event from the kernel layer and identifies the control corresponding to the input event. For example, if the touch operation is a single-click operation and the control corresponding to the single-click operation is the control of the camera application icon, the camera application calls the application framework layer interface to start the camera application, which then calls the kernel layer to start the camera driver and capture still images or video through camera 193.

[0238] In some embodiments, the eye gaze position calculation module may also reside in other layers of the software architecture. For example, it may reside in the kernel layer, the system library, or the application layer, without limitation. When the eye gaze position calculation module and the message notification module are located in the same layer, it may be a module independent of the message notification module or a submodule within the message notification module, without limitation.

[0239] The following describes the method in the embodiment of the present application in detail with reference to the accompanying drawings and the software and hardware structure of the exemplary electronic device 100.

[0240] Figure 19 This is a flow chart of a message display method in an embodiment of the present application:

[0241] S1901: The electronic device determines that there is a message window that needs to be displayed;

[0242] In the embodiment of the present application, the electronic device can determine whether there is a message window that needs to be displayed in various ways:

[0243] 1. When the electronic device determines that there is a message to be displayed in the notification manager, it can determine that there is a message window to be displayed;

[0244] 2. When the electronic device determines that the system or application has a prompt message that needs to be displayed, it can determine that there is a message window that needs to be displayed;

[0245] 3. When the electronic device determines that the system or application has entities such as controls and windows to be displayed, it can determine that there is a message window that needs to be displayed.

[0246] It is understandable that the electronic device may also determine that there is a message window that needs to be displayed in other ways, which are not limited here.

[0247] S1902: The electronic device determines whether the user's current eye gaze position is within the display area.

[0248] It is understandable that electronic devices can determine the user's eye gaze position in real time through eye tracking technology. For details, please refer to (1) eye tracking technology in the above term description, which will not be elaborated here.

[0249] When the electronic device determines that the user's current eye gaze position is within the display area of the electronic device, the electronic device may execute step S1904;

[0250] When the electronic device determines that the current eye gaze position of the user is not within the display area of the electronic device, the electronic device may execute step S1903.

[0251] It is understandable that the electronic device can determine that the user's eye gaze position is outside the display area through the eye tracking technology, thereby determining that the user's current eye gaze position is not in the display area; it can also determine that the user's current eye gaze position is not in the display area by failing to detect the user's eye gaze position in the display area of the electronic device, which is not limited here.

[0252] S1903: The electronic device displays the message window at a default position;

[0253] When the electronic device determines that the user's current eye gaze position is not in the display area, it means that the user is not currently looking at the electronic device. At this time, the electronic device can display the message window in the default position of the message window.

[0254] Optionally, after the electronic device displays the message window in the default position, it can use additional reminders to remind the user of new messages. For example, the electronic device can increase the message reminder sound; for example, the electronic device can increase the message reminder vibration, etc., which are not limited here.

[0255] In some embodiments, step S1903 may not be executed, which is not limited here.

[0256] S1904: The electronic device determines, in real time, the current eye gaze position of the user in the display area of the electronic device;

[0257] When the electronic device determines that the current eye gaze position of the user is in the display area of the electronic device, the electronic device determines the current eye gaze position of the user in the display area of the electronic device in real time.

[0258] Specifically, the electronic device can determine the current eye gaze position of the user in the display area of the electronic device in real time in a continuous process. The electronic device can continuously track and determine the current eye gaze position of the user in the display area of the electronic device in real time or periodically.

[0259] It is understood that if the current eye gaze position of the electronic device in the display area has been obtained in step S1902, step S1904 may not be performed. If step S1902 only determines that the eye gaze position is within the display area but does not determine the specific eye gaze position, the electronic device may determine the specific current eye gaze position of the user in the display area of the electronic device in step S1904, which is not limited here.

[0260] S1905: The electronic device determines a moving speed of a current eye gaze position of the user in a display area of the electronic device;

[0261] When the electronic device determines that the current eye gaze position of the user is in the display area of the electronic device, the electronic device may determine a moving speed of the current eye gaze position of the user in the display area of the electronic device.

[0262] It is understandable that the execution order of steps S1904 and S1905 is not limited.

[0263] In some embodiments, step S1905 may not be performed, and the display position of the message window may be determined only according to the current eye gaze position, which is not limited here.

[0264] For example, Figure 20 This is an exemplary diagram of a user interface in an embodiment of the present application. On display area 2003, the electronic device determines that the user is reading in the direction of arrow 2001. The electronic device can determine that the user's current eye gaze position is within the display area and that the current eye gaze position is second eye gaze position 2002. Furthermore, using eye tracking technology, the electronic device can determine that the speed of movement of the user's current eye gaze position within display area 2003 is V2002.

[0265] S1906: The electronic device determines whether the moving speed of the current eye gaze position is not greater than a preset first speed value;

[0266] After the electronic device determines the movement speed of the user's current eye gaze position in the display area of the electronic device, it can determine whether the movement speed is not greater than a preset first speed value;

[0267] If the movement speed is greater than the preset first speed value, it means that the current eye gaze position of the user moves faster, and the electronic device may execute step S1907;

[0268] If the movement speed is not greater than the preset first speed value, it means that the movement speed of the user's current eye gaze position is slow, and the electronic device can execute step S1908.

[0269] S1907: The electronic device determines whether the average moving speed of the user's eye gaze position within the preset first duration is not greater than a preset second speed value;

[0270] When the electronic device determines that the current eye gaze position movement speed is greater than a preset first speed value, it may determine an average movement speed of the user's eye gaze position over a subsequent preset first time period, and then determine whether the average movement speed is no greater than a preset second speed value;

[0271] It is understandable that the preset second speed value may be greater than, less than or equal to the preset first speed value, which is not limited here.

[0272] The user's current eye gaze position in the display area of the electronic device moves at a relatively fast speed. The electronic device continuously monitors the moving speed of the eye gaze position within the preset first time period to determine whether the average moving speed of the eye gaze position meets the display requirements.

[0273] When the electronic device determines that the average movement speed of the user's eye gaze position within the preset first time period is not greater than the preset second speed value, it indicates that the average movement speed of the user's eye gaze position meets the display requirement, and the electronic device may execute step S1908;

[0274] When the electronic device determines that the average movement speed of the user's eye gaze position within the preset first time period is greater than the preset second speed value, it means that the average movement speed of the user's eye gaze position is high, and the electronic device can execute step S1909.

[0275] It can be understood that the average movement speed of the user's eye gaze position within the preset first time period is the average value of the movement speed of the current eye gaze position at each moment monitored in real time by the electronic device within the first time period.

[0276] Optionally, in actual applications, the speed of the user's eye movement position within a period of time can be determined not only by the average speed, but also by the cumulative movement distance of the user's eye movement position within a period of time.

[0277] Specifically, when the electronic device determines that the current eye gaze position movement speed is greater than a preset first speed value, it may determine a cumulative distance of movement of the user's eye gaze position within a subsequent preset fourth time period, and then determine whether the cumulative distance exceeds a preset cumulative distance;

[0278] When the electronic device determines that the cumulative distance of movement of the user's eye gaze position within the preset first time period is not greater than the preset cumulative distance, the electronic device may execute step S1908;

[0279] When the electronic device determines that the cumulative distance of movement of the user's eye gaze position within the preset first time period is greater than the preset cumulative distance, the electronic device may execute step S1909.

[0280] S1908: The electronic device displays a message window in the first preset visual area;

[0281] The first preset visual area is around the user's current eye gaze position. For a description of the first preset visual area, please refer to (4) the first preset visual area in the above term description, and will not be repeated here.

[0282] When the electronic device determines that the speed of movement of the user's current eye gaze position in the display area of the electronic device is not greater than a preset first speed value, the electronic device displays a message window in the first preset visual area; or, after the electronic device determines that the speed of movement of the user's current eye gaze position in the display area of the electronic device is greater than the preset first speed value, when the electronic device determines that the average speed of movement of the user's eye gaze position within the preset first time period is not greater than a preset second speed value, the electronic device displays the message window in the first preset visual area. The first preset visual area is determined based on the current eye gaze position at that time.

[0283] In some embodiments, after the electronic device determines in step S1901 that there is a message window that needs to be displayed, it can directly execute step S1908.

[0284] In some embodiments, before displaying the message window in the first preset visual area, the electronic device may first determine the first preset visual area based on the user's current eye gaze position in the display area, and then display the message window in the first preset visual area.

[0285] It is understood that the electronic device determining the first preset visual area based on the user's current eye gaze position can be a real-time, continuously updated process. That is, as the user's current eye gaze position moves, the electronic device can update the first preset visual area corresponding to each new current eye gaze position determined by the electronic device.

[0286] For example, Figure 21 This is an exemplary diagram of the user interface in the embodiment of the present application. Assume that the first speed value is preset as YV1 and the second speed value is preset as YV2 in the electronic device. The electronic device determines Figure 20When the speed value V2002 of the movement speed of the current eye gaze position is not greater than the preset first speed value YV1, a first preset visual area 2101 can be determined around the second eye gaze position 2002, and then a message window 2102 to be displayed can be displayed in the first preset visual area 2101. In this example, the size of the first preset visual area 2101 is larger than the message window 2102 to be displayed.

[0287] For example, Figure 22a This is an exemplary diagram of the user interface in the embodiment of the present application. Assume that the first speed value is preset as YV1 and the second speed value is preset as YV2 in the electronic device. The electronic device determines Figure 20 If the speed value V2002 of the movement speed of the user's eye gaze position is greater than the preset first speed value YV1, the electronic device determines whether the average movement speed of the user's eye gaze position during the preset first duration is no greater than the preset second speed value YV2. If, after the preset first duration, the user's eye gaze position moves to a third eye gaze position 2201, the electronic device determines that the average movement speed of the user's eye gaze position during the preset first duration, V2201, is less than the preset second speed YV2, and can determine a first preset visual area 2202 around the third eye gaze position 2201, and then display the message window 2203 to be displayed in the first visual area 2202. In this example, the size and shape of the first preset visual area 2202 are the same as those of the message window 2203 to be displayed.

[0288] For example, Figure 22b This is another exemplary diagram of the user interface in the embodiment of the present application. Assume that the first speed value preset in the electronic device is YV1 and the preset cumulative distance is SL. The electronic device determines Figure 20 If the speed value V2002 of the movement speed of the eye gaze position is greater than the preset first speed value YV1, it is determined whether the cumulative distance moved by the user's eye gaze position within the preset fourth time duration is greater than the preset cumulative distance. If, within the preset fourth time duration, the user's eye gaze position first moves a distance S1 in the direction of the arrow 2201 shown in the figure, and then moves a distance S2 in the direction of the arrow 2202 shown in the figure, reaching the current eye gaze position 2203. Then, the cumulative distance moved by the user's eye gaze position within the preset fourth time duration is S1+S2. When the electronic device determines that the cumulative distance S1+S2 is less than the preset cumulative distance SL, the electronic device determines a first preset visual area 2204 around the current eye gaze position 2203, and then displays the message window 2205 to be displayed in the first visual area 2204.

[0289] S1909: The electronic device displays a message window in the first preset visual area, and moves the message window in accordance with the movement of the current eye gaze position;

[0290] When the electronic device determines that the movement speed of the user's current eye gaze position in the display area of the electronic device is greater than a preset first speed value, and determines that the average movement speed of the user's eye gaze position in the subsequent preset first time period is greater than a preset second speed value, or determines that the cumulative distance of the user's eye gaze position movement in the preset fourth time period at this time is greater than the preset cumulative distance, the electronic device displays a message window in the first preset visual area and moves the message window following the movement of the eye gaze position.

[0291] The electronic device can move the message window in accordance with the movement of the eye gaze position in a variety of different ways:

[0292] Optionally, the electronic device may synchronously move the message window according to the direction and distance of the eye gaze position movement.

[0293] For example, Figure 23 This is a set of interface diagrams in the embodiment of the present application. Assume that the first speed value is preset as YV1 and the second speed value is preset as YV2 in the electronic device. The electronic device determines Figure 20 The speed value V2002 of the eyeball gaze position movement speed is greater than the preset first speed value YV1, and it is determined that the average movement speed of the user's eyeball gaze position in the subsequent preset first time period is greater than the preset second speed value YV2. Figure 23 As shown in (a) of FIG, after the preset first time duration has passed, the electronic device determines that the user has moved the eye gaze position to a fourth eye gaze position 2302 in the direction of arrow 2301. The electronic device determines a first preset visual area 2303 around the fourth eye gaze position 2302 and displays a message window 2304 in the first preset visual area 2303. Figure 23 As shown in (b) in the figure, the user's eye gaze position continues to move. In the process of the user's eye gaze position moving from the fourth eye gaze position 2302 to the fifth eye gaze position 2306 according to the direction vector 2305, the message window 2304 also moves to the message window 2308 according to the direction vector 2307 that is the same as the direction vector 2305.

[0294] Optionally, when the eye gaze position moves, the electronic device can update the first preset visual area in real time according to the user's current eye gaze position, and move the message window to the first preset visual area that is updated in real time.

[0295] For example, Figure 23As shown in (c) of FIG. 1 , the user's eye gaze position continues to move. As the user's eye gaze position moves from fifth eye gaze position 2306 to sixth eye gaze position 2310 along direction vector 2309, the electronic device can update the first preset visual area in real time based on the user's current eye gaze position. When the eye gaze position moves to sixth eye gaze position 2310, the electronic device can update the first preset visual area to first preset visual zone 2311 and move message window 2312 into the first preset visual area.

[0296] It is understandable that the electronic device may also adopt various other methods to move the message window following the movement of the eye gaze position, which is not limited here.

[0297] Preferably, in some embodiments, when the electronic device moves the message window following the movement of the eye gaze position, the electronic device may determine whether the movement speed of the current eye gaze position is not greater than a preset third speed value;

[0298] When it is determined that the moving speed of the current eye gaze position is not greater than a preset third speed value, the electronic device may stop moving the message window.

[0299] In some embodiments, step S1909 may not be performed, which is not limited here.

[0300] S1910: The electronic device determines whether the user's current eye gaze position moves into the area of the message window within a preset second time period after the message window is displayed;

[0301] After the electronic device displays the message window in the first preset visual area in step S1908 or step S1909, the electronic device may determine whether the user's eye gaze position moves into the area of the message window within a preset second time period after the message window is displayed;

[0302] The preset second duration may be a factory preset, may vary according to different message windows, or may be a user-defined setting, which is not limited here.

[0303] When the electronic device determines that the user's eye gaze position has not moved into the area of the message window within the preset second time period after the message window is displayed, it indicates that the user has not viewed the message window, and the electronic device may execute step S1911;

[0304] When the electronic device determines that the user's current eye gaze position moves into the area of the message window at a moment within the preset second time period after the message window is displayed, it indicates that the user is looking at the message window, and the electronic device can execute step S1912.

[0305] In some embodiments, step S1910 may not be performed, which is not limited here.

[0306] S1911. The electronic device determines whether the movement distance of the eye gaze position exceeds a preset second distance within a preset third time period after the message window is displayed.

[0307] When the electronic device determines that the current eye gaze position of the user has not moved into the area of the message window within a preset second time period after the message window is displayed, the electronic device determines whether a movement distance of the eye gaze position exceeds a preset second distance within a preset third time period after the message window is displayed, that is, whether the user's eye gaze position has a significant displacement;

[0308] When the electronic device determines that the movement distance of the eye gaze position exceeds the preset second distance within the preset third time period after the message window is displayed, that is, the user's eye gaze position has a large displacement, the electronic device may execute step S1907;

[0309] When the electronic device determines that the movement distance of the eye gaze position within the preset third time period after the message window is displayed does not exceed the preset second distance, that is, when the user's eye gaze position does not have a large displacement, the electronic device can execute step S1912.

[0310] In some embodiments, step S1911 may not be performed, which is not limited here.

[0311] S1912. The electronic device stops moving the message window.

[0312] When the electronic device determines that the user's current eye gaze position moves into the area of the message window at a moment within the preset second time period after the message window is displayed, that is, when the user is looking at the message window, the electronic device may stop moving the message window;

[0313] Alternatively, when the electronic device determines that the user's eye gaze position has not moved into the area of the message window within a preset second time period after the message window is displayed, and the electronic device determines that the movement distance of the eye gaze position has not exceeded the preset second distance within a preset third time period after the message window is displayed, that is, although the user has not looked at the message window, the user's eye gaze position has not had a significant displacement, the electronic device can stop moving the message window.

[0314] It is understandable that the electronic device stopping moving the message window is not a processing operation that the electronic device must perform, but can be a state. That is, when the electronic device executes step S1912, it can stop moving the message window without performing any operation, which is not limited here.

[0315] In an embodiment of the present application, when an electronic device has a message window that needs to be displayed, the message window is directly displayed around the user's current eye gaze position in the display area. Since the message window is directly displayed around the user's current eye gaze position, the user can directly see the message window without having to shift their viewing angle. This allows the user to quickly and conveniently obtain notification messages, improving the efficiency of message notifications. Furthermore, the electronic device can intelligently follow the movement of the user's current eye gaze position and move or stop the message window based on the speed of the user's current eye gaze position, greatly improving the convenience of the user in viewing the message window.

[0316] It is understandable that the message display method in the embodiment of the present application can be applied not only to the electronic device 100 described above, but also to other types of electronic devices. The electronic device in the embodiment of the present application can also adopt other hardware structures, which are not limited here.

[0317] For example, Figure 24 It is a structural diagram of an exemplary electronic device 200 provided in an embodiment of the present application.

[0318] The electronic device 200 may include: an input device 2401, an output device 2402, a processor 2403 and a memory 2404 (wherein the number of the processor 2403 in the electronic device 200 may be one or more, Figure 24 In some embodiments of the present application, the input device 2401, the output device 2402, the processor 2403 and the memory 2404 may be connected via a bus or other means, wherein: Figure 24 The bus connection is taken as an example.

[0319] The input device 2401 may include the hardware required to implement eye tracking technology, such as a high-definition camera or an infrared camera. The output device 2402 may also include the hardware required to implement eye tracking technology, such as a display area, an infrared light emitter, etc. The specific implementation method may be determined based on the implementation method of the eye tracking technology and is not limited here.

[0320] The processor 2403 causes the electronic device 200 to execute the message display method in the embodiment of the present application by calling the operation instructions stored in the memory 2404. The specific process is similar to the above electronic device 100 executing the message display method in the embodiment of the present application, and will not be repeated here.

[0321] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

[0322] As used in the above embodiments, the term “when” may be interpreted to mean “if” or “after” or “in response to determining that” or “in response to detecting that”, depending on the context. Similarly, the phrases “upon determining that” or “if (stated condition or event) is detected” may be interpreted to mean “if determining that” or “in response to determining that” or “upon detecting (stated condition or event)” or “in response to detecting (stated condition or event)”, depending on the context.

[0323] In the above embodiments, all or part of the embodiments can be implemented using software, hardware, firmware, or any combination thereof. When implemented using software, all or part of the embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present application are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, hard disk, tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive).

[0324] Those skilled in the art will appreciate that all or part of the process steps in the above-described method embodiments can be implemented by a computer program instructing the relevant hardware. The program can be stored in a computer-readable storage medium, and when executed, the program can include the process steps in the above-described method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM or random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A message display method, characterized in that: include: The electronic device determines that there is a message window that needs to be displayed; The electronic device determines in real time the current eye gaze position of the user in the display area of the electronic device; The electronic device determines a first preset visual area according to the current eye gaze position, where the first preset visual area is around the current eye gaze position on the display area; The electronic device determines whether a moving speed of the current eye gaze position is not greater than a preset first speed value; If so, the electronic device displays the message window in the first preset visual area; If not, the electronic device determines whether the average movement speed of the user's eye gaze position within the preset first time period is not greater than a preset second speed value, or the electronic device determines whether the cumulative distance of the user's eye gaze position movement within the preset fourth time period exceeds a preset cumulative distance; When the average moving speed is not greater than the preset second speed value, or the cumulative distance does not exceed the preset cumulative distance, the electronic device displays the message window in a first preset visual area determined according to the current eye gaze position; When the average moving speed is greater than the preset second speed value, or the cumulative distance exceeds the preset cumulative distance, the electronic device displays the message window in a first preset visual area determined according to the current eye gaze position, and moves the message window in accordance with the movement of the user's current eye gaze position; The electronic device determines whether the current eye gaze position of the user moves into the area of the message window within a preset second time period after the message window is displayed; If the user's current eye gaze position moves into the area of the message window at one moment within the preset second time period, the electronic device stops moving the message window; If the user's eye gaze position does not move into the area of the message window within the preset second time period, the electronic device determines whether the movement distance of the user's eye gaze position exceeds a preset second distance within a preset third time period after the message window is displayed; If the movement distance of the user's eye gaze position does not exceed the preset second distance within the preset third time period, the electronic device stops moving the message window; If the moving distance of the user's eye gaze position within the preset third time period exceeds the preset second distance, the electronic device displays the message window in the first preset visual area determined according to the current eye gaze position, or the electronic device executes the step of determining whether the average moving speed of the user's eye gaze position within the preset first time period is not greater than the preset second speed value, or the electronic device executes the step of determining whether the cumulative distance of the user's eye gaze position moving within the preset fourth time period exceeds the preset cumulative distance.

2. The method according to claim 1, characterized in that The first preset visual area is on the display area and its center or starting position is within a first preset distance from the current eye gaze position, and the size of the first preset visual area is not less than the size of the message window.

3. The method according to claim 1 or 2, characterized in that The electronic device determines a first preset visual area according to the current eye gaze position, specifically including: The electronic device determines, based on the relative position of the current eye gaze position on the display area, a preset display orientation and a preset display distance that match the relative position; The electronic device determines that the center or starting position of the first preset visual area is at a preset display distance of the preset display orientation of the current eye gaze position.

4. The method according to claim 1 or 2, characterized in that The electronic device determines a first preset visual area according to the current eye gaze position, specifically including: The electronic device determines whether there is an area on the display area that has no displayed content and is not smaller than the size of the message window within a range of a first preset distance from the current eye gaze position; When it is determined that there is an area that has no displayed content and is not smaller than the size of the message window, the electronic device determines that the area is the first preset visual area.

5. The method according to any one of claims 1 or 2, characterized in that The method further comprises: The electronic device determines a moving speed of a current eye gaze position of a user in a display area of the electronic device.

6. An electronic device, characterized in that: The electronic device includes: one or more processors and memory; The memory is coupled to the one or more processors, and the memory is used to store computer program code, where the computer program code includes computer instructions. The one or more processors call the computer instructions to cause the electronic device to execute: Determine if there is a message window that needs to be displayed; Determine in real time the current eye gaze position of the user on the display area of the electronic device; Determining a first preset visual area according to the current eye gaze position, where the first preset visual area is around the current eye gaze position on the display area; Determining whether the moving speed of the current eye gaze position is not greater than a preset first speed value; If so, displaying the message window in the first preset visual area; If not, the electronic device determines whether the average movement speed of the user's eye gaze position within the preset first time period is not greater than a preset second speed value, or the electronic device determines whether the cumulative distance of the user's eye gaze position movement within the preset fourth time period exceeds a preset cumulative distance; When the average moving speed is not greater than the preset second speed value, or the cumulative distance does not exceed the preset cumulative distance, the electronic device displays the message window in a first preset visual area determined according to the current eye gaze position; When the average moving speed is greater than the preset second speed value, or the cumulative distance exceeds the preset cumulative distance, the electronic device displays the message window in a first preset visual area determined according to the current eye gaze position, and moves the message window in accordance with the movement of the user's current eye gaze position; determining whether the user's current eye gaze position moves into the area of the message window within a preset second time period after the message window is displayed; If the user's current eye gaze position moves into the area of the message window at one moment within the preset second time period, the electronic device stops moving the message window; If the user's eye gaze position does not move into the area of the message window within the preset second time period, the electronic device determines whether the movement distance of the user's eye gaze position exceeds a preset second distance within a preset third time period after the message window is displayed; If the movement distance of the user's eye gaze position does not exceed the preset second distance within the preset third time period, the electronic device stops moving the message window; If the moving distance of the user's eye gaze position within the preset third time period exceeds the preset second distance, the electronic device displays the message window in the first preset visual area determined according to the current eye gaze position, or the electronic device executes the step of determining whether the average moving speed of the user's eye gaze position within the preset first time period is not greater than the preset second speed value, or the electronic device executes the step of determining whether the cumulative distance of the user's eye gaze position moving within the preset fourth time period exceeds the preset cumulative distance.

7. The electronic device according to claim 6, wherein: The first preset visual area is on the display area and its center or starting position is within a first preset distance from the current eye gaze position, and the size of the first preset visual area is not less than the size of the message window.

8. The electronic device according to claim 6 or 7, characterized in that: The one or more processors are specifically configured to call the computer instructions to enable the electronic device to execute: Determining, according to the relative position of the current eye gaze position on the display area, a preset display orientation and a preset display distance that match the relative position; Determine that the center or starting position of the first preset visual area is at a preset display distance of the preset display orientation of the current eye gaze position.

9. The electronic device according to claim 6 or 7, characterized in that: The one or more processors are specifically configured to call the computer instructions to enable the electronic device to execute: Determining whether there is an area on the display area that has no displayed content and is not smaller than the size of the message window within a range of a first preset distance from the current eye gaze position; When it is determined that there is an area with no displayed content and the area is not smaller than the size of the message window, the area is determined to be the first preset visual area.

10. The electronic device according to any one of claims 6 or 7, characterized in that: The one or more processors are further configured to call the computer instructions to cause the electronic device to execute: Determine the movement speed of the user's current eye gaze position on the display area of the electronic device.

11. A chip system, applied to an electronic device, comprising one or more processors, wherein the processors are configured to call computer instructions so that the electronic device executes the method according to any one of claims 1 to 5.

12. A computer program product comprising instructions, characterized in that When the computer program product is run on an electronic device, the electronic device is enabled to perform the method according to any one of claims 1 to 5.

13. A computer-readable storage medium comprising instructions, characterized in that: When the instruction is executed on an electronic device, the electronic device is caused to execute the method according to any one of claims 1 to 5.

Images