Subject display method, electronic device, readable storage medium, and program product

Abstract

The application relates to the technical field of terminals, in particular to a theme display method, an electronic device, a readable storage medium and a program product. The method can determine a gaze area in a theme interface currently displayed by an electronic device and a first theme element displayed in the gaze area according to data of a user's gaze on the theme interface. Furthermore, the electronic device can adjust theme elements displayed in the gaze area according to data representing the user's current emotion, for example, replace the first theme element in the gaze area with a second theme element, or display a dynamic effect of the first theme element and the second theme element switching with each other in the gaze area. The second theme element can be determined based on the emotion corresponding to the first data. In this way, feedback interaction can be provided for the change of the user's emotion, the interest of theme display is improved, and the user experience is improved.

Description

Technical Field

[0001] This application relates to the field of terminal technology, specifically to a theme display method, electronic device, readable storage medium, and program product. Background Technology

[0002] Themes applied to terminal electronic devices (hereinafter referred to as electronic devices) include background styles, text styles, sticker styles, and control styles displayed on the theme interface. These backgrounds, texts, stickers, and controls can be called theme elements. The theme interface can include the lock screen interface, the desktop, etc. Taking the lock screen interface as an example, the lock screen interface is the foreground display interface when the electronic device is in standby and locked state. It can include theme elements such as lock screen wallpaper, lock screen text, lock screen stickers, and lock screen controls. The theme elements displayed on the lock screen interface can be static or dynamic elements, where dynamic elements refer to elements that change dynamically with specific effects. Electronic devices can replace the theme elements and display styles of the lock screen interface, such as background, text, and stickers, by loading theme resources, thereby achieving a complete change of the visual effect of the lock screen interface.

[0003] However, the theme elements provided by these theme resources can only be displayed according to preset styles. For example, dynamic elements can only change in a fixed loop with preset effects. The expression is relatively simple, which can easily cause visual fatigue for users and is not conducive to improving the user experience. Summary of the Invention

[0004] This application provides a theme display method, an electronic device, a readable storage medium, and a program product for improving user experience.

[0005] In a first aspect, this application provides a theme display method applied to an electronic device. The method includes: displaying a first interface; determining a first gaze area of ​​a user on the first interface, wherein a first theme element is displayed within the first gaze area; displaying a second theme element in the first gaze area based on first data, or displaying a dynamic effect of switching between the first theme element and the second theme element in the first gaze area, wherein the first data is used to represent the user's emotions, the second theme element is determined based on the first data, and the second theme element is different from the first theme element.

[0006] In this embodiment, the electronic device can adjust the displayed thematic elements within the defined first gaze area based on first data that can characterize the user's emotions. This includes displaying a second thematic element determined based on the first data within the gaze area, and also displaying a dynamic effect of switching between the aforementioned thematic elements within the gaze area. Thus, based on the thematic display method provided in this application, the electronic device can support feedback interaction to changes in the user's emotions, increasing the interest of the thematic interface and thereby improving the user experience.

[0007] For example, the first interface mentioned above can be an interface to which the theme resources can be applied, i.e., a theme interface, such as a lock screen or desktop. The electronic device mentioned above can be a mobile phone, tablet, or other terminal electronic device. The second data mentioned above can be, for example, gaze perception data, and the first data mentioned above can be, for example, emotion perception data. Based on the gaze perception data, the electronic device can determine the user's gaze area on the theme interface, such as the first gaze area mentioned above.

[0008] In one possible implementation of the first aspect above, the above-mentioned displaying a second subject element in the first gaze area based on the first data, or displaying a dynamic effect of switching between the first subject element and the second subject element in the first gaze area, includes: obtaining the second subject element that matches the first data based on the first data; displaying the second subject element in the first gaze area, or displaying a dynamic effect of switching between the first subject element and the second subject element in the first gaze area.

[0009] In this embodiment, the electronic device can acquire a matching second theme element based on the first data. Thus, the electronic device can use the acquired theme element to adjust the display effect of the theme element within the user's gaze area.

[0010] For example, electronic devices can first obtain corresponding matching topic elements, such as second topic elements, based on the emotion perception data obtained above.

[0011] In one possible implementation of the first aspect above, the first gaze area is determined based on second data, the second data including data on the user's gaze at the first interface, the second data including the duration of the user's gaze at the first gaze area, and the display of a second theme element in the first gaze area, or the display of a dynamic effect of switching between the first theme element and the second theme element in the first gaze area, includes: displaying the second theme element in the first gaze area based on the duration of the user's gaze at the first gaze area being greater than or equal to a first threshold; or displaying a dynamic effect of switching between the first theme element and the second theme element in the first gaze area based on the duration of the user's gaze at the first gaze area being greater than 0 and less than the first threshold.

[0012] In this embodiment, the electronic device can control the display of a second thematic element in the first gaze area based on the duration of the user's gaze over the first gaze area being greater than or equal to a first threshold; or, based on the duration of the user's gaze over the first gaze area being greater than 0 and less than the first threshold, it can display a dynamic effect of switching between the first and second thematic elements in the first gaze area. In this way, the electronic device can dynamically adjust the effect of displaying thematic elements matching emotion perception data within the gaze area based on the duration of the user's gaze over the first gaze area, which helps to increase interactivity and interest, thereby improving the user experience.

[0013] It is understandable that the duration of a user's gaze at the first gaze area, i.e., the gaze duration, can reflect the intensity of the user's gaze at the first gaze area to a certain extent. For example, the longer the gaze duration, the higher the user's gaze intensity; the shorter the gaze duration, the lower the user's gaze intensity. The first threshold can be a preset time threshold corresponding to the gaze duration. Based on this, the electronic device can determine that the user is gazing intensely at the first gaze area based on the gaze duration being greater than or equal to the first threshold, and in this case, the matching second subject element can be displayed. The electronic device can also determine that the user is gazing infrequently at the first gaze area based on the gaze duration being less than the first threshold, and in this case, a dynamic effect of switching between the subject elements can be displayed.

[0014] It is understandable that the gaze duration can be greater than 0. If the user's gaze duration is 0, it means that the user's gaze behavior was not detected.

[0015] In one possible implementation of the first aspect above, the second data includes the distance at which the user gazes at the first gaze area. The display of the second subject element in the first gaze area, or the display of a dynamic effect of switching between the first subject element and the second subject element in the first gaze area, includes: displaying the second subject element in the first gaze area based on the distance at which the user gazes at the first gaze area being less than a second threshold; or displaying a dynamic effect of switching between the first subject element and the second subject element in the first gaze area based on the distance at which the user gazes at the first gaze area being greater than or equal to the second threshold.

[0016] In this embodiment, the electronic device can control the display of a second theme element in the first gaze area based on the user's gaze distance being less than a second threshold; or it can control the display of a dynamic effect that switches between the first and second theme elements in the first gaze area based on the user's gaze distance being greater than or equal to the second threshold. In this way, the electronic device can dynamically adjust the effect of displaying theme elements matching emotion perception data within the gaze area according to the user's gaze distance, which helps increase interactivity and interest, thereby improving the user experience.

[0017] For example, the distance at which the user gazes at the first gaze area, i.e., the gaze distance, can be the distance between the user's eyes and the screen of the electronic device, or other reasonably measured distances, without limitation. The gaze distance can also reflect the intensity of the user's gaze at the first gaze area to some extent. For example, within a certain preset distance range (such as the first distance range), the smaller the gaze distance, the higher the user's gaze intensity; the larger the gaze distance, the lower the user's gaze intensity. The second threshold can be a preset distance threshold corresponding to the gaze distance. Based on this, the electronic device can determine that the user is gazing intensely at the first gaze area based on a gaze distance less than the second threshold, and in this case, the matching second subject element can be displayed. The electronic device can also determine that the user is gazing infrequently at the first gaze area based on a gaze distance greater than or equal to the second threshold, and in this case, a dynamic effect of switching between the subject elements can be displayed.

[0018] In one possible implementation of the first aspect described above, the distance at which the user gazes at the first gaze region falls within a first distance range.

[0019] In one possible implementation of the first aspect above, the second data includes the area related to the first gaze region; the display of the second subject element in the first gaze region, or the display of a dynamic effect of switching between the first subject element and the second subject element in the first gaze region, includes: displaying the second subject element in the first gaze region based on the area of ​​the first gaze region being less than a third threshold; or displaying a dynamic effect of switching between the first subject element and the second subject element in the first gaze region based on the area of ​​the first gaze region being greater than or equal to the third threshold.

[0020] In this implementation, the electronic device can control the display of a second theme element in the first gaze area based on the area of ​​the first gaze area being less than a third threshold; or it can control the display of a dynamic effect that switches between the first theme element and the second theme element in the first gaze area based on the area of ​​the first gaze area being greater than or equal to the third threshold. In this way, the electronic device can dynamically adjust the effect of displaying theme elements that match emotion perception data within the gaze area according to the area of ​​the first gaze area, which helps to increase interactivity and interest, thereby improving the user experience.

[0021] It is understandable that the area of ​​the first gaze region can reflect the user's level of concentration to a certain extent, and thus the intensity of the user's gaze on the first gaze region. For example, the smaller the area of ​​the gaze region, the higher the user's level of concentration, and the higher the intensity of the gaze; the larger the area of ​​the gaze region, the lower the user's level of concentration, and the lower the intensity of the gaze. The third threshold can be a preset area threshold corresponding to the area of ​​the gaze region. Based on this, the electronic device can determine that the user is gazing intensely at the first gaze region if the area of ​​the first gaze region is less than the third threshold, and in this case, the matching second theme element can be displayed. The electronic device can also determine that the user is gazing weakly at the first gaze region if the area of ​​the first gaze region is greater than or equal to the third threshold, and in this case, the dynamic effect of switching between the theme elements can be displayed.

[0022] In one possible implementation of the first aspect described above, the area of ​​the first gaze region is determined based on saccade features corresponding to the second data, wherein the saccade features include one or more of the following: saccade rate, saccade amplitude, and saccade frequency. Thus, the electronic device can calculate the area of ​​the gaze region actually covered by the user's line of sight based on the aforementioned saccade features.

[0023] For example, the greater the saccade amplitude, the faster the saccade rate, and the higher the saccade frequency, the more dispersed the eye movement is, and the larger the corresponding area of ​​the gaze area is.

[0024] In one possible implementation of the first aspect above, displaying the second theme element in the first gaze area includes: if the first theme element is a wallpaper element and includes a face area, then displaying the second theme element in such a way that the second theme element covers a portion of the first theme element; if the first theme element is not a wallpaper element, then removing the first theme element and displaying the second theme element in the first gaze area.

[0025] For example, the first theme element mentioned above could be a person image element (an example of a wallpaper element), and the second theme element could be a sticker element that matches emotion perception data, such as an emoji sticker. In this case, the way the image is overlaid on a portion of the first theme element could be, for example, displayed over a face area.

[0026] In one possible implementation of the first aspect described above, displaying the second theme element by covering a portion of the first theme element with the second theme element includes any of the following: if the size of the face region is within a first size range and the face orientation angle within the face region is within a first angle range, then the second theme element is displayed by covering the face region with the second theme element; if the size of the face region is greater than the upper limit of the first size range, then the second theme element is displayed by covering a first region of the first theme element with the second theme element, wherein the distance between the center point of the first region and the center point of the face region is less than a fourth threshold; if the size of the face region is less than the lower limit of the first size range, or the face orientation angle is greater than the upper limit of the first angle range, then the second theme element is not displayed.

[0027] In this embodiment, the electronic device can provide various ways to display the second theme element by covering a portion of the first theme element with the second theme element, based on multiple comparison results between the size of the face region and the first size range. This increases the visual appeal of the lock screen or desktop interface displayed by the electronic device using the theme display method provided in this application, thereby improving the user experience.

[0028] It is understood that the size of the aforementioned face area falling within the first size range can be used to determine whether the face in the person image element is appropriate, too large, or too small. If the face is appropriate, such as if the size of the face area falls within the first size range, the electronic device can control the display of the emoji sticker by covering the face area (an example of the aforementioned second theme element). If the face is too large, such as if the size of the face area exceeds the upper limit of the first size range, the electronic device can control the display of the emoji sticker in the area near the face. In this case, the emoji sticker can be displayed close to the face to create an interactive effect. In some embodiments, the aforementioned display close to the face can be determined, for example, by whether the distance between the center point of the first area covered by the emoji sticker in the person image element and the center point of the face area is less than the aforementioned fourth threshold. If the face is too small, such as if the size of the face area is less than the lower limit of the first size range, the electronic device can control the display of the emoji sticker, and in this case, the electronic device may not load the relevant theme resources.

[0029] In one possible implementation of the first aspect above, before determining the user's first gaze area on the first interface based on the second data, the method includes: activating a first interactive function; determining the user's first gaze area on the first interface based on the second data includes: acquiring the second data when the first interactive function is activated; and determining the user's first gaze area on the first interface based on the second data.

[0030] In this embodiment, the electronic device can determine the user's first gaze area on the first interface based on second data when it detects that the first interactive function is activated. This helps to improve the user experience.

[0031] For example, the first interactive function mentioned above could be a gaze-aware interactive function.

[0032] In one possible implementation of the first aspect above, activating the first interactive function includes: displaying a second interface, wherein the second interface includes a switch control for activating or deactivating the first interactive function; and activating the first interactive function in response to a user's activation operation on the switch control.

[0033] For example, the aforementioned switch control can be a switch control for gaze-aware interaction functionality, used to turn gaze-aware interaction functionality on or off. The activation operation can be, for example, a user clicking or touching the switch control. In some embodiments, the switch control can also be called an expression interaction switch, emotion interaction switch, emotion-aware switch, interaction switch, etc., without limitation.

[0034] In one possible implementation of the first aspect above, obtaining the second topic element that matches the first data based on the first data includes: determining the emotion type of the user's current emotion based on the first data; and obtaining the second topic element that matches the emotion type.

[0035] For example, an electronic device can first determine the user's emotion type based on emotion perception data, and then determine a matching topic element, such as the second topic element mentioned above, based on that emotion type. In some embodiments, the electronic device can identify the emotion type corresponding to the user's current emotion based on an emotion score or emotion feature vector, and then match the second topic element mentioned above based on the emotion type.

[0036] In other embodiments, the electronic device may first determine the emotion tag corresponding to the user's emotion type based on emotion perception data, and then determine the matching topic element based on the emotion tag. The element resource library accessible to the electronic device may store the matching relationship, or correspondence, between topic elements and emotion tags, which will not be elaborated here.

[0037] In one possible implementation of the first aspect above, determining the emotion type of the user's current emotion based on the first data includes: determining a score for the user's current emotion based on the first data; determining a first emotion type of the user's current emotion based on the score; and obtaining the second theme element that matches the first emotion type.

[0038] In one possible implementation of the first aspect described above, the first theme element matches a second emotion type, and the second emotion type is different from the first emotion type. For example, both the first and second theme elements are theme elements generated based on images input by the user, such as sticker elements. These theme elements can express the style of the image input by the user, or the expression of the person in the image input by the user, etc., without limitation.

[0039] In one possible implementation of the first aspect above, before displaying the second theme element in the first gaze area, or before displaying the dynamic effect of switching between the first theme element and the second theme element in the first gaze area, the method further includes: at a first moment, acquiring the user's first gaze behavior data on the first interface, and determining the user's first gaze point position on the first interface based on the first gaze behavior data; at a second moment, acquiring the user's second gaze behavior data on the first interface, and determining the user's second gaze point position on the first interface based on the second gaze behavior data, wherein the second moment is later than the first moment; predicting the position of the third gaze point that the user is about to gaze at based on the first gaze point position and the second gaze point position; and preloading the second theme element in response to the third gaze point position being located in the first gaze area.

[0040] In this embodiment, the electronic device can predict the location of the user's next gaze point based on the user's first gaze point location at a first moment and the user's second gaze point location at a second moment, and preload the second theme element displayed in the first gaze area where the third gaze point is located. In this way, the above-mentioned preloading of element resources can prepare theme elements in advance for the area where the user's next gaze point is located, which helps to improve the rendering efficiency of theme elements, and thus helps to improve the smoothness of the related theme interface refresh.

[0041] For example, an electronic device can track a user's gaze behavior over a period of time, such as tracking the area of ​​focus of the user's gaze, thereby fitting a gaze trajectory. This period of time could be, for example, the time interval between a first moment and a second moment. Furthermore, based on this gaze trajectory, the electronic device can predict the area where the user's next gaze point will be located, i.e., predict the next gaze area, such as the first gaze area mentioned above.

[0042] In one possible implementation of the first aspect above, the first gaze region is determined based on second data. After displaying the first interface, the method further includes: determining a second gaze region of the user on the first interface based on the second data, wherein the second gaze region is a different region from the first gaze region; displaying a third theme element in the first gaze region and a fourth theme element in the second gaze region based on one or more of the following: the duration of the user's gaze on the first gaze region is greater than the duration of the user's gaze on the second gaze region; or, the distance at which the user gazes on the first gaze region is less than the distance at which the user gazes on the second gaze region; or, the area of ​​the first gaze region is less than the area of ​​the second gaze region; wherein the rendering complexity of the third theme element is higher than the rendering complexity of the fourth theme element.

[0043] In this embodiment, the electronic device can determine the first and second gaze regions of the user on the first interface based on the second data. Furthermore, if the intensity of the user's gaze on the first gaze region is greater than the intensity of the user's gaze on the second gaze region, the device can control the display of a third theme element with higher rendering complexity within the first gaze region and a fourth theme element with lower rendering complexity within the second gaze region. Thus, based on the theme display method provided in this application, the electronic device can achieve reasonable scheduling of rendering resources and improve the rendering efficiency of theme elements and the theme interface.

[0044] It can be understood that the duration of a user's gaze at the first or second gaze area is the gaze duration. The distance the user gazes at the first or second gaze area is the gaze distance. Any one of the gaze duration, gaze distance, and the area of ​​the first gaze area can be used to determine the intensity of the user's gaze at the first and second gaze areas. For example, if the user's gaze duration at the first gaze area is greater than the user's gaze duration at the second gaze area, or the user's gaze distance at the first gaze area is less than the user's gaze distance at the second gaze area, or the area of ​​the first gaze area is smaller than the area of ​​the second gaze area, it can indicate that the user's gaze intensity at the first gaze area is higher than the gaze intensity at the second gaze area. In this case, the electronic device can control the display of more complex subject elements for the first gaze area (high-intensity gaze) and less complex subject elements for the second gaze area (low-intensity gaze).

[0045] In a second aspect, this application provides an electronic device, including: one or more processors; one or more memories; wherein the one or more memories store one or more programs, and when the one or more programs are executed by the one or more processors, the electronic device performs the subject display method provided in the first aspect and various possible implementations of the first aspect.

[0046] Thirdly, this application provides a computer-readable storage medium storing instructions that, when executed on a computer, cause the computer to perform the subject display method provided in the first aspect and various possible implementations of the first aspect.

[0047] Fourthly, this application provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the subject display method provided in the first aspect and various possible implementations of the first aspect.

[0048] Fifthly, this application provides an apparatus comprising: a module for performing the subject display method provided in the first aspect and various possible implementations thereof.

[0049] In a sixth aspect, this application provides a chip including at least one processor and at least one memory coupled for executing a computer program stored in the at least one memory, such that the chip performs the subject display method provided in the first aspect and various possible implementations of the first aspect.

[0050] The beneficial effects of the second to sixth aspects mentioned above can be referred to the relevant descriptions in the first aspect and its various possible implementations, which will not be repeated here. Attached Figure Description

[0051] Figure 1 The image shown is a schematic diagram of a theme interface display scenario provided in an embodiment of this application.

[0052] Figure 2A The diagram shown is a schematic representation of the interface changes when changing theme elements, implemented by a theme display method provided in an embodiment of this application.

[0053] Figure 2B The diagram shown is a schematic representation of the interface changes that implement the dynamic effect of switching between display theme elements according to a theme display method provided in this application embodiment.

[0054] Figure 3A The diagram shown is a schematic representation of the interface changes for displaying emoji stickers based on the face orientation angle of a face region, according to an embodiment of this application.

[0055] Figure 3B The diagram shows an interface change for another method of displaying emoji stickers based on the face orientation angle of the face region, as provided in an embodiment of this application.

[0056] Figure 3C The diagram shows an interface change of a method for determining the display of emoji stickers based on the size of the face region, according to an embodiment of this application.

[0057] Figure 3D The diagram shown is a schematic representation of an interface change for controlling the number of emoji stickers used to represent a user's current emotion, according to an embodiment of this application.

[0058] Figure 4 The diagram shown is a schematic representation of the implementation process of a topic display method provided in an embodiment of this application.

[0059] Figure 5AThe image shown is a schematic diagram of the theme settings interface when the gaze-aware interaction function is not enabled, according to an embodiment of this application.

[0060] Figure 5B The image shown is a schematic diagram of the theme settings interface after the gaze-aware interaction function is enabled, according to an embodiment of this application.

[0061] Figure 6 The diagram shown is a schematic representation of the interface changes for quickly enabling or disabling the gaze-aware interaction function according to an embodiment of this application.

[0062] Figure 7 The diagram shown is a schematic representation of another interface change provided in an embodiment of this application for quickly enabling or disabling the gaze-aware interaction function.

[0063] Figure 8 The diagram shown is a schematic block diagram of the software structure of an operating system provided in an embodiment of this application.

[0064] Figure 9 The diagram shown is a schematic representation of the interactive implementation process of a theme display method that enables theme elements to interact with the user's emotions, according to an embodiment of this application.

[0065] Figure 10 The diagram shown is an interactive implementation flow diagram of a theme display method that can dynamically adjust the rendering complexity of the theme elements displayed within the gaze area, according to an embodiment of this application.

[0066] Figure 11 The diagram shown illustrates the implementation process and principle of a theme display method provided in this application for preloading theme element resources.

[0067] Figure 12 The diagram shown is a schematic block diagram of the software structure of a HarmonyOSTM system architecture provided in an embodiment of this application.

[0068] Figure 13 The diagram shown illustrates the implementation principle of a theme display method provided in this application within an operating system.

[0069] Figure 14 The diagram shown is a schematic block diagram of the hardware structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0070] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions in the embodiments of this application will be described in detail below with reference to the accompanying drawings and specific implementation methods.

[0071] The method provided in this application can be applied to any electronic device, including but not limited to mobile stations (MS) and mobile terminals (MT). For example, electronic devices can be mobile phones, smart TVs, wearable devices, tablets, personal computers (PCs, including desktop computers and laptops), virtual reality (VR) devices, augmented reality (AR) devices, terminals in smart cities, terminals in smart homes, and foldable screen devices. Laptops can include laptops, and foldable screen devices can include foldable phones, foldable tablets, and foldable laptops. This application does not limit the form of the electronic device.

[0072] The term "user interface (UI)" used in the following embodiments of this application refers to the medium interface through which an application or operating system interacts and exchanges information with a user. It realizes the conversion between the internal form of information and a form acceptable to the user. The user interface is source code written in a specific computer language such as Java or Extensible Markup Language (XML). The interface source code is parsed and rendered on an electronic device, ultimately presenting content that the user can recognize. A common form of user interface is the graphical user interface (GUI), which refers to a user interface related to computer operation displayed graphically.

[0073] Figure 1 This diagram illustrates a scenario where a themed interface is displayed.

[0074] like Figure 1 As shown, the lock screen interface 101 displayed by the electronic device 10 may include multiple theme elements, such as wallpaper elements as backgrounds, as well as emoticon elements 102, first text elements 103, second text elements 104, etc.

[0075] It is understandable that after the user completes the theme settings and applies them to the lock screen, the electronic device 10 can display each theme element according to a fixed preset style when displaying the lock screen 101. For example, Figure 1 The emoticon element 102, the first text element 103, the second text element 104, etc. shown are all displayed using fixed preset styles.

[0076] The current theme display solution can only display according to the preset style, which can easily cause visual fatigue for users and is not conducive to improving the user experience.

[0077] To address the aforementioned issues, this application provides a theme display method. First, the user's gaze area on the interface and a first theme element displayed within that gaze area are determined. Then, the electronic device can adjust the theme element displayed within the gaze area based on data representing the user's current emotion. This could involve replacing the first theme element with a second theme element, or displaying a dynamic effect showing the switching between the first and second theme elements. The second theme element can be determined based on the emotion corresponding to the first data. This allows for feedback interaction based on changes in the user's emotion, enhancing the engaging nature of the theme interface and ultimately improving the user experience.

[0078] It is understood that the data used to characterize the user's current emotions refers to data that can reflect the user's emotional state. In the following text, it can also be referred to as emotion perception data, such as the user's facial image data, pupil image data, etc., or facial features obtained based on facial image data analysis, or pupil features obtained based on pupil image data analysis, etc.

[0079] In some embodiments, the electronic device or a database accessible to the electronic device may pre-set a matching relationship, or correspondence, between theme elements and emotion tags. The theme elements may be theme elements adapted to the theme interface (i.e., the first interface, such as a lock screen or desktop). Based on this, the emotion perception data can be used to determine the user's current emotion score or emotion feature vector. The electronic device can then identify the emotion type corresponding to the user's current emotion based on the emotion score or emotion feature vector, and match the second theme element based on the emotion type.

[0080] In some embodiments, the electronic device can determine the user's gaze area based on data about the user's gaze at the currently displayed theme interface (referred to as gaze perception data). In some embodiments, the gaze perception data can be used to determine the intensity of the user's gaze at the gaze area. The gaze intensity can indicate the degree of focus and / or concentration of the user's attention when looking at the screen or interface. In some embodiments, the electronic device can determine the user's gaze intensity at the lock screen or desktop, etc., based on one or more of the following: gaze duration, gaze distance, degree of concentration (or area of ​​the gaze area). Taking the user's gaze area as the first gaze area as an example, the gaze duration refers to the length of time the user gazes at the first gaze area. As an example, the longer the gaze duration, the higher the user's gaze intensity; the shorter the gaze duration, the lower the user's gaze intensity. It is understood that the gaze duration can be greater than 0; if the user's gaze duration is 0, it indicates that no user gaze behavior was detected.

[0081] Fixation distance refers to the physical distance at which a user fixates on a first fixation area. In embodiments of this application, this distance can be the distance between the user's eyes and the screen of an electronic device, or other reasonably measured distances. In some embodiments, this distance can be determined based on detection data from devices such as distance sensors, and is not limited thereto. As an example, within a certain preset distance range (such as a first distance range), the smaller the fixation distance, the higher the user's fixation intensity can be; the larger the fixation distance, the lower the user's fixation intensity can be. The user's fixation intensity can be negatively correlated with the fixation distance, that is, the closer the fixation distance, the higher the user's level of focus or visual engagement, and the higher the corresponding fixation intensity.

[0082] In some embodiments, the above-mentioned gaze intensity can also be related to the user's level of concentration. For example, the user's level of concentration can be determined based on the size of the area where the user's gaze point is focused (i.e., the gaze region). For instance, a smaller gaze region indicates a higher level of concentration, and the user's gaze intensity can be higher; a larger gaze region indicates a lower level of concentration, and the user's gaze intensity can be lower.

[0083] In some embodiments, the area of ​​the gaze region can be determined based on saccade feature analysis of the user's gaze behavior on the first interface. This saccade feature can be obtained based on the second data analysis described above, and may include one or more of the following: saccade rate, saccade amplitude, and saccade frequency.

[0084] For example, a user's saccade amplitude can determine the distance of a single eye movement, saccade rate can reflect how fast a user's eye moves, and saccade frequency can reflect how frequently a user's eye switches. The larger the saccade amplitude, the faster the saccade rate, and the higher the saccade frequency, the more dispersed the eye activity, and the larger the corresponding area of ​​the gaze.

[0085] In some embodiments, the electronic device system may deploy a computational model or algorithm for calculating the user's gaze region for later invocation. As an example, the computational model or algorithm deployed in the electronic device may use the saccade amplitude as a baseline value, combined with the boundary expansion coefficient corresponding to the saccade rate and the distribution coefficient corresponding to the saccade rate, to comprehensively calculate the major axis length and minor axis length of an elliptical gaze region, thereby determining the area of ​​the elliptical gaze region as the area of ​​the user's gaze region. In this embodiment, when the user's gaze region coincides with the area of ​​a theme element (such as a first theme element) in the theme interface, or when the user's gaze region covers the area of ​​the theme element, the electronic device can determine that the first theme element exists within the user's gaze region.

[0086] For example, with a scan amplitude of 120 pixels (px), a scan rate of 250 pixels / second (px / s), and a scan frequency of 2 scans / second, the electronic device can use twice the scan amplitude as a reference, and take d0 = 2 × 120px = 240 px as the initial diameter. The boundary expansion coefficient k1 corresponding to the scan rate is k1 = 1 + scan rate / 1000 = 1 + 0.25 = 1.25. Therefore, based on the initial diameter d0 and the boundary expansion coefficient k1 corresponding to the scan rate, the electronic device can determine the corrected initial diameter d1 = d0 × k1 = 240 × 1.25 = 300 px.

[0087] As mentioned earlier, the higher the scan rate, the wider the user's field of vision coverage. Based on a scan rate of 2 scans / second, for example, the distribution coefficient k2 corresponding to the scan rate can be determined as k2 = 1 + scan rate / 10 = 1 + 0.2 = 1.2. Based on the corrected initial diameter d1 and the distribution coefficient k2 corresponding to the scan rate, the theoretical value of the major axis length of the elliptical gaze region can be calculated as a = (d1 × k2) / 1.5 = 240px. To facilitate the system's region division, boundary determination, parameter configuration, and logical operations during rendering adaptation, and to reserve redundant range for eye saccades, the electronic device system can take the nearest value of 250px for the theoretical value a = 240px of the major axis length of the elliptical gaze region, and determine the major axis length of the elliptical gaze region as 250px. Based on the aforementioned major axis length of 250px, the electronic device can further calculate the minor axis length of the ellipse's viewing area as b = a × 0.72 = 250 × 0.72 = 180px according to the preset ratio of the minor axis length to the major axis length of the ellipse (such as 0.72 or other values).

[0088] Based on the aforementioned major axis length of 250px, minor axis length of 180px, and the formula for calculating the area of ​​an ellipse, the electronic device can determine the area of ​​the elliptical gaze region, such as S = 35343 square pixels (px²). At this point, the electronic device can combine the user's gaze point position with the coverage area of ​​the elliptical gaze region to determine the user's gaze region on the main interface.

[0089] In other embodiments, the computational model or algorithm deployed in the electronic device may also employ other computational methods that can determine the user's gaze area, and this is not limited thereto. In some embodiments, the above-mentioned saccade features may also be described as eye movement features, and this is not limited thereto.

[0090] Based on this, the aforementioned second data may include data related to gaze duration, gaze distance, gaze area, etc., determined based on the analysis of the second data. The electronic device can also determine the intensity of the user's gaze at the theme interface based on the second data. For example, when the user's gaze intensity at the lock screen interface is high, the electronic device can replace the first theme element in the currently displayed lock screen interface with a second theme element. As another example, when the user's gaze intensity at the lock screen interface is low, the electronic device can display a dynamic effect of switching between the first and second theme elements in the currently displayed lock screen interface.

[0091] In some embodiments, the electronic device may preset a threshold or threshold group for determining whether the user's gaze intensity is high-intensity gaze or low-intensity gaze, and then display different adjustment effects on the subject element for high-intensity gaze or low-intensity gaze respectively.

[0092] For example, if the duration of a user's gaze at the first gaze area is greater than or equal to a corresponding preset time threshold (which can be denoted as the first threshold), the electronic device can determine that the threshold condition for high-intensity gazing is met, and thus can control the display of the second subject element within the first gaze area. If the duration of a user's gaze at the first gaze area is greater than 0 and less than the aforementioned time threshold, the electronic device can determine that the threshold condition for low-intensity gazing is met, and thus can control the display of a dynamic effect of switching between the first subject element and the second subject element within the first gaze area.

[0093] For example, if the distance at which the user gazes at the first gaze area (i.e., the gaze distance) is less than the corresponding set distance threshold (which can be denoted as the second threshold), the electronic device can determine that the threshold condition for high-intensity gaze is met, and thus can control the display of the second subject element in the first gaze area. If the distance at which the user gazes at the first gaze area (i.e., the gaze distance) is greater than or equal to the aforementioned distance threshold, the electronic device can determine that the threshold condition for low-intensity gaze is met, and thus can control the display of a dynamic effect of switching between the first subject element and the second subject element in the first gaze area.

[0094] For example, if the area of ​​the first gaze region is less than a corresponding preset area threshold (which can be denoted as the third threshold), the electronic device can determine that the threshold condition for high-intensity gaze is met, and thus can control the display of the second subject element in the first gaze region. If the area of ​​the first gaze region is greater than or equal to the aforementioned area threshold, the electronic device can determine that the threshold condition for low-intensity gaze is met, and thus can control the display of a dynamic effect of switching between the first subject element and the second subject element in the first gaze region.

[0095] As an example, Figure 2A An embodiment of this application illustrates a schematic diagram of interface changes for changing theme elements implemented by a theme display method.

[0096] like Figure 2A As shown, the lock screen interface 210 displayed by the electronic device 10 may include multiple theme elements, such as the emoji element 102, the first text element 103, and the second text element 104, which may be the aforementioned first theme element. Each theme element can be related to the above... Figure 1The theme elements in the lock screen interface 101 shown are identical. The emoticon element 102 can be a sticker element selected by the user when setting the theme, expressing the user's emotion upon selection. Emoticon element 102 can also be a sticker element generated from an image entered by the user, expressing the style of the image or the expression of a person in the image, etc., without limitation. For example, emoticon element 102 can express excitement. Similarly, the first text element 103 and the second text element 104 can be text elements generated corresponding to the text entered by the user when setting the theme, or text elements generated based on the style of the image entered by the user, without limitation. For example, the text of the first text element 103 could be "The sky is vast, the earth is vast," and the text of the second text element 104 could be "Happiness is the greatest," etc.

[0097] Based on the theme display method provided in this application, the electronic device 10 can adjust one or more theme elements, i.e., second theme elements, displayed on the lock screen interface 210 based on the recognized current emotion of the user when the user is looking at the lock screen interface 210. For example, if the user's gaze area covers the area where the emoji element 102 is located, the electronic device 10 can control the display of another emoji element 211 in the area where the emoji element 102 is located, or display a dynamic effect of switching between emoji element 102 and emoji element 211. As another example, if the user's gaze area covers the area where the first text 103 is located, the electronic device 10 can control the display of a third text element 212 in the area where the first text element 103 is located, or display a dynamic effect of switching between the first text element 103 and the third text element 212. As yet another example, if the user's gaze area covers the area where the second text element 104 is located, the electronic device 10 can control the display of another emoji element 213 in the area where the second text element 104 is located, or display a dynamic effect of switching between the second text element 104 and the fifth text element 215.

[0098] As an example, based on acquired data such as gaze duration and gaze distance, the electronic device 10 can determine that the user's gaze intensity on the lock screen interface 210 is high-intensity. At this time, the electronic device 10 can change one or more theme elements in the lock screen interface 210 according to the user's current emotion. For example, if the user's current emotion is surprise, the electronic device 10 can change the emoji element 102 in the lock screen interface 210 to an emoji element 211 expressing surprise. Alternatively, the electronic device 10 can change the first text element 103 in the lock screen interface 210 to a third text element 212, the text of which can match the user's current surprised emotion, such as "What happened?", etc. In some embodiments, the electronic device 10 can also change the second text element 104 in the lock screen interface 210 to an emoji element 213, which can present another expression of surprise, and this emoji element 213 can cooperate with the aforementioned changed third text element 212 to produce an interactive visual effect.

[0099] As an example, Figure 2B According to an embodiment of this application, a schematic diagram of interface changes is shown, illustrating the dynamic effect of switching between display theme elements implemented by a theme display method.

[0100] like Figure 2B As shown, the lock screen interface 210 displayed by the electronic device 10 may include multiple theme elements such as emoji elements 102, first text elements 103, and second text elements 104. The display styles of each theme element can be referenced above for... Figure 2A The relevant descriptions will not be repeated here.

[0101] Based on the theme display method provided in this application, the electronic device 10 can adjust one or more theme elements displayed on the lock screen interface 210 according to the recognized current emotion of the user when the user is looking at the lock screen interface 210. For example, if the user's gaze area covers the area where the first text element 103 is located, the electronic device 10 can control the display of the fourth text element 214 in the area where the first text element 103 is located, or display a dynamic effect of switching between the first text element 103 and the fourth text element 214. As another example, if the user's gaze area covers the area where the second text element 104 is located, the electronic device 10 can control the display of the fifth text element 215 in the area where the second text element 104 is located, or display a dynamic effect of switching between the second text element 104 and the fifth text element 215.

[0102] It should be noted that the above Figure 2A or Figure 2BThe fixation area shown is only for illustrative purposes, and is not an actually divided and displayed area on the theme interface. The relevant data of the fixation area determined by the electronic device based on the above saccade features and the like can be used to determine the display position of theme elements such as expression elements and text elements when rendering the above theme elements, which is not limited or repeated herein.

[0103] As an example, based on the obtained data such as fixation duration and fixation distance, the electronic device 10 can determine that the user's fixation intensity on the lock screen 210 is low-intensity fixation. At this time, the electronic device 10 can control one or more theme elements in the lock screen 210 to dynamically switch according to the user's current emotion. For example, if the user's current emotion is happiness, the electronic device 10 can display a dynamic effect in the lock screen 210 in which the first text element 103 and the fourth text element 214 matched according to the happy emotion switch with each other. For this dynamic effect, reference can be made to Figure 2B the changing effect of mutual switching between text elements shown as "the world is as big as it is" → "worldly烟火" → "the world is as big as it is" → "worldly烟火" → "the world is as big as it is". For another example, the electronic device 10 can also control the second text element 104 in the lock screen 210 and the text element 215 matching the emotion to switch alternately. For this dynamic effect, reference can be made to Figure 2B the changing effect of mutual switching between text elements shown as "happiness is the greatest" → "joy is supreme" → "happiness is the greatest" → "joy is supreme" → "happiness is the greatest".

[0104] It can be understood that, taking the user's current emotion as happy emotion as an example, this emotion can be the same as the happy emotion expressed by the expression element 102 in the lock screen 210. Therefore, the expression element 102 in the lock screen 210 displayed by the electronic device 10 at this time can remain unchanged.

[0105] In some embodiments, the manner in which the electronic device displays the second theme element in the first fixation area may include displaying the second theme element by covering a partial area of the first theme element with the second theme element, or may include removing the first theme element from the first fixation area and displaying the second theme element in the first fixation area, which is not limited herein.

[0106] In some embodiments, the electronic device may display the second theme element by covering the face area with the second theme element when it detects that the first theme element includes a face area, the size of the face area falls within a first size range, and / or the face orientation angle indicating the face area falls within a first angle range. As an example, the first theme element may include a portrait image element having a face area, and the second theme element may be an expression sticker, which may also be referred to as an expression sticker in some embodiments.

[0107] It is understood that the aforementioned face orientation angle falling within the first angle range can be used to identify whether the face in the character image element is a frontal face. If it is a frontal face, the electronic device can display an emoji sticker representing the user's current emotion on or near the face, even if the user's gaze area covers the area where the character image element is located. In some embodiments, the electronic device can also display an emoji sticker representing the user's current emotion on or near the face when the user is gazing at the aforementioned theme interface with the character image element; this is not limited. If the aforementioned face orientation angle does not fall within the first angle range, the face in the corresponding character image element is not a frontal face, and in this case, the electronic device will not display an emoji sticker even if the user is gazing at the area where the character image element is located.

[0108] In some embodiments, the aforementioned face orientation angle can be the angle between the direction the face is facing and the direction perpendicular to the screen of the electronic device and pointing outwards from the screen. The value of this angle can be between [0°, 90°]. Correspondingly, the aforementioned first angle range can be preset to a range such as [0°, 30°], [0°, 45°], or [0°, 60°], and is not limited here.

[0109] As an example, Figure 3A According to an embodiment of this application, a schematic diagram of interface changes is shown, illustrating a method for determining the display of emoji stickers based on the facial orientation angle of a facial region.

[0110] like Figure 3A As shown, the lock screen interface 310 displayed by the electronic device 10 may include multiple theme elements, such as the person image element 311 and person image element 312, which may be the first theme element mentioned above. For example, person image element 311 may be a selfie, with the face area facing the user. Person image element 312 may be a person running, with the face area facing the running direction and not towards the user; therefore, no emoji stickers are displayed on this person image element 312.

[0111] Based on the theme display method provided in this application, the electronic device 10 can display an emoji sticker 313 on the face area of ​​the character image element 311 when the user's gaze area on the lock screen interface 310 covers the area where the character image element 311 is located, based on the recognized current emotion of the user. For example, if the user's current emotion is excitement or pleasure, the emoji sticker 313 can be an expression expressing excitement or pleasure.

[0112] As another example Figure 3B According to an embodiment of this application, a schematic diagram of interface changes is shown for another method of determining the display of emoji stickers based on the facial orientation angle of the face region.

[0113] like Figure 3B As shown, the lock screen interface 310 displayed by the electronic device 10 may include the aforementioned first theme elements such as the character image element 311 and character image element 312. Based on the theme display method provided in this application, the electronic device 10 can display an expression sticker 314 on the face area of ​​the character image element 311 when the user's gaze area on the lock screen interface 310 covers the area where the character image element 311 is located, based on the recognized current emotion of the user. For example, if the user's current emotion is surprise, the expression on the expression sticker 314 can be an expression of surprise.

[0114] It is understood that in some embodiments, if the first interface displayed by the electronic device (such as a lock screen or desktop) does not include human image elements, or as described above... Figure 3A or Figure 3B The face area of ​​the character image element 312 shown in the first interface includes face areas whose face orientation angle does not fall within the aforementioned first angle range, i.e., face areas that are not facing the user. Based on the theme display method provided in this application, the electronic device can control the display of emoji stickers not to be covered in the face area.

[0115] It is understood that the size of the aforementioned face region falling within the first size range can be used to determine whether the face in the person image element is appropriate, too large, or too small. If the face is appropriate, such as if the size of the face region falls within the first size range, the electronic device can control the display of the emoji sticker (i.e., the second theme element) by covering the face region with an emoji sticker. If the face is too large, such as if the size of the face region exceeds the upper limit of the first size range, the electronic device can control the display of the emoji sticker in the area near the face. In this case, the emoji sticker can be displayed close to the face to create an interactive effect. In some embodiments, the aforementioned display close to the face can be determined, for example, by whether the distance between the center point of the first area covered by the emoji sticker in the person image element and the center point of the face region is less than a relevant threshold (which can be denoted as the fourth threshold). If the face is too small, such as if the size of the face region is less than the lower limit of the first size range, the electronic device can control the display of the emoji sticker, and in this case, the electronic device may not load the relevant theme resources.

[0116] In some embodiments, the electronic device can determine whether a face is of appropriate size, too large, or too small based on a comparison between the size of the rectangle containing the face region and the screen size of the electronic device. For example, if the ratio between the maximum side length of the rectangle containing the face region and the screen width is within a preset ratio range, such as 0.133 to 0.272, 0.133 to 0.511, or other ratio ranges, the electronic device can determine that the size of the face region belongs to a first size range. As another example, if the ratio between the maximum side length of the rectangle containing the face region and the screen width is greater than the upper limit of the preset ratio range, the electronic device can determine that the size of the face region is too large. If the ratio between the maximum side length of the rectangle containing the face region and the screen width is less than the lower limit of the preset ratio range, the electronic device can determine that the size of the face region is too small.

[0117] In some embodiments, the composition parameters of the theme template used by the theme interface (such as the lock screen or desktop) displayed by the electronic device may include core area position parameters. The core area may be the area in the theme interface where the visual effect is best suited for displaying theme elements such as stickers. As an example, taking the position parameters of the upper left corner of the electronic device screen {left:0, top:0, width:0, height:0} as an example, the composition parameters of a theme template where the wallpaper element is a single image may include the composition parameters {left:0.088, top:0.279, width:0.819, height:0.470}. Similarly, the layout parameters of a theme template where the wallpaper element is multiple images may include the position parameters {left:0.022, top:0.279, width:0.9, height:0.589}, and so on. No further limitations are imposed here.

[0118] As an example, Figure 3C According to an embodiment of this application, a schematic diagram of interface changes is shown to determine the display method of emoji stickers based on the size of the face region.

[0119] like Figure 3C As shown, the lock screen interface 320 displayed by the electronic device 10 may include a person image element 321 (an example of a first theme element). Similar to the above... Figure 3A The interface shown is different. Figure 3C The face area of ​​the character image element 321 in the interface shown is relatively large, such as being greater than or equal to the upper limit of the preset first size range. Figure 3A The face area of ​​the character image element 311 in the interface shown belongs to the first size range.

[0120] As mentioned above, the electronic device can also display emoticon stickers representing the user's current emotion on or near the face of the user when the user is looking at the aforementioned theme interface with the human image element 321. As an example, based on the theme display method provided in this application, the electronic device 10 can, when the user is looking at the lock screen interface 320, display emoticon stickers representing the user's current emotion in the area near the face of the human image element 321, based on the recognized current user emotion and the relatively large size of the face area of ​​the human image element 321. Figure 3C The image shows an emoji sticker 322 displayed in the area near the face.

[0121] In some embodiments, if the first interface displayed by the electronic device (such as a lock screen or desktop) does not include a person image element, or if the face of the person image element in the first interface is too small, such as the size of the face area being smaller than the lower limit of the first size range, the electronic device can also control the display of emoji stickers not to be covered in the face area based on the theme display method provided in this application.

[0122] To ensure interactivity between the emoji stickers and the character image elements displayed in the adjusted first interface, in some embodiments, if the character image element does not include a face area, the electronic device may not display emoji stickers related to the user's current expression in the area where the character image element is displayed. This can increase the visual appeal of the lock screen or desktop, thereby improving the user experience.

[0123] In some embodiments, users can swipe across the lock screen or desktop displayed on the electronic device (i.e., the first interface), such as swiping from the top left corner to the bottom right corner, or from the top right corner to the bottom left corner, or switching their gaze across different display areas of the theme elements on the lock screen. In this case, if multiple character images are displayed on the first interface (e.g., the lock screen) and each character image includes a face area, the electronic device can also, based on a set threshold (e.g., 2 or other values), control the display of emoticon stickers representing the user's current emotion in a portion of the face areas (e.g., two face areas) on the first interface, while keeping the other portion of the face areas undisplayed. This avoids the problem of poor user visual experience caused by adding too many theme elements to the first interface.

[0124] As an example, Figure 3D An embodiment of this application illustrates an interface variation diagram for controlling the number of emoji stickers used to represent a user's current emotion.

[0125] like Figure 3DAs shown, the lock screen interface 330 displayed by the electronic device 10 may include multiple first theme elements, such as character image element 331, character image element 332, character image element 333, etc. The orientation of the face in each character image element can be within a preset first angle range, that is, they are all frontal faces. In addition, the size of the face area in each character image element can also be within a preset first size range.

[0126] Based on the theme display method provided in this application, the electronic device 10 can, when the user is looking at the lock screen interface 310, control the display of expression stickers 334 on the face area of ​​the character image element 331 and the display of expression stickers 335 on the face area of ​​the character image element 332, based on the recognized current emotion of the user and the size of the face area of ​​the character image element 331 being within a first size range.

[0127] Continue to refer to Figure 3D Based on a preset quantity threshold (such as 2), the electronic device 10 can control the display of facial stickers on the face area of ​​the character image element 333 when the user is looking at the lock screen interface 310, so as to avoid the problem of displaying too many sticker elements and causing poor visual experience for the user on the lock screen interface 330.

[0128] In some embodiments, the electronic device may, based on the user's gaze intensity on the lock screen area, display the aforementioned emoji stickers and other theme elements fixedly on or near the face area. Alternatively, the electronic device may, based on the user's gaze intensity on the lock screen area, display dynamic effects of having emoji stickers and not having them on the face area.

[0129] It is understandable that if an electronic device detects a change in a user's emotions over a period of time, such as a change in a user's facial expression from laughing to surprised, the corresponding user emotion can change from happiness to surprise. In this case, the electronic device can display a dynamic effect on the face area based on the user's high-intensity gaze, where the emoji sticker representing happiness gradually changes to the emoji sticker representing surprise. Alternatively, based on the user's low-intensity gaze, it can display a dynamic effect of switching between emoji stickers representing happiness and emoji stickers representing surprise.

[0130] As an example, the dynamic effect of the emoji stickers gradually changing from representing happiness to representing surprise could be, for example, the happy emoji stickers gradually fading until they disappear, and the surprised emoji stickers appearing and gradually deepening, or other dynamic effects, without limitation. In some embodiments, while displaying the dynamic effect of the emoji stickers switching between each other, the electronic device can also overlay a sweeping light effect or other dynamic effects to enhance the user's visual experience.

[0131] Based on the above Figures 2A to 2B , Figures 3A to 3D The diagram shown illustrates the interface changes. The following section, in conjunction with the flowchart, provides a detailed explanation of the implementation process of the theme display method provided in this application, along with related figures.

[0132] It should also be stated that the steps in the methods and processes in this application are numbered for ease of reference, not to limit the order of steps. If there is an order between the steps, the textual description shall prevail.

[0133] As an example, Figure 4 An implementation flowchart of a topic display method is shown according to an embodiment of this application.

[0134] It is understood that, in the embodiments of this application, Figure 4 In the process shown, each step is executed by an electronic device, as described above. Figures 2A to 2B , Figures 3A to 3D The electronic devices shown, such as 10, are not limited here. Further details will follow. Figure 4 When describing each step of the process shown, the entity responsible for executing each step will not be described again.

[0135] like Figure 4 As shown, the implementation process may include S401 to S403:

[0136] S401: Display the first interface.

[0137] For example, the first interface mentioned above can be the theme interface applied to the theme resource, such as a lock screen or desktop. Taking the lock screen as an example, the lock screen may include wallpaper elements (such as character image elements), sticker elements (such as emoticons and text), and clock control elements. The first theme element mentioned above can be a wallpaper element or a sticker element. In the embodiments of this application, the first theme element can be configured with interactive attributes. For example, the face area of ​​the character image element can be configured with interactive attributes for sticker attachment, so that it can support both directly overlaying stickers on the face area and placing stickers in the area around the face. As another example, the sticker element can be configured with style switching interactive attributes, which can realize flexible replacement of sticker content and appearance style such as emoticons and text.

[0138] S402: Determine the user's first gaze area on the first interface, where the first theme element is displayed.

[0139] For example, the electronic device may collect gaze perception data while the user is looking at the first interface (i.e., the main interface), including eye movement data and gaze point data obtained based on the user's eye visual information collected by an image acquisition device (such as a camera), as well as feature data such as gaze characteristics and saccade characteristics determined based on gaze perception data analysis, without limitation. In some embodiments, the electronic device may rely on gaze perception technology to acquire the above-mentioned gaze perception data and perceive the user's gaze behavior.

[0140] The principle of gaze sensing technology is based on eye tracking and gaze perception mechanisms. It acquires the user's visual information through image acquisition devices, and then analyzes the data to extract raw data such as the gaze placement point, eye movement state, pupil parameters, and blinking behavior—this is the gaze sensing raw data. On one hand, this gaze sensing raw data can be used to identify gaze behavior formed by stable gaze fixation and to determine gaze characteristics such as gaze duration, gaze area, and gaze distance. On the other hand, the aforementioned gaze sensing data can be used to identify saccade behavior resulting from rapid gaze switching and to analyze saccade characteristics such as saccade rate, saccade amplitude, and saccade frequency. All of the aforementioned gaze characteristics, saccade characteristics, and the gaze sensing raw data on which these characteristics are analyzed can be referred to as gaze sensing data in the embodiments of this application.

[0141] In this embodiment, the aforementioned saccade features can be used to analyze the area of ​​the gaze region, quantify the user's level of attention concentration, and thus quantify the user's gaze intensity. For example, the user's saccade amplitude can determine the distance of a single gaze movement, the saccade rate can reflect the speed of the user's gaze movement, and the saccade frequency can reflect the frequency of the user's gaze switching. The larger the saccade amplitude, the faster the saccade rate, and the higher the saccade frequency, the more dispersed the gaze activity, and the larger the corresponding area of ​​the gaze region. Based on this, the electronic device can calculate the area of ​​the gaze region actually covered by the user's gaze based on the aforementioned saccade features. The specific calculation method can refer to the relevant descriptions and examples of the calculation model or algorithm for the area of ​​the gaze region mentioned above, and will not be elaborated here.

[0142] S403: Based on the first data, display the second subject element in the first gaze area, or display a dynamic effect of switching between the first subject element and the second subject element in the first gaze area.

[0143] For example, the first data used to characterize the user's current emotion may include facial expression data and eye movement data collected by an image acquisition device (such as a camera), as well as emotion scores calculated based on emotion perception data, and is not limited thereto. Correspondingly, the gaze perception data used to determine the first gaze region in S402 above can be referred to as the second data.

[0144] In some embodiments, the electronic device can score the emotional state expressed by collected facial expression data, eye movement data, etc., based on a preset emotion scoring mechanism, and determine the emotion type of the user's current emotion based on the scoring results. In some embodiments, the preset emotion types may include very happy, somewhat happy, calm, somewhat unhappy, very angry (or very unhappy), crying (or very unhappy), etc., and each emotion type may correspond to a scoring range. For example, very happy may correspond to a score of 0, somewhat happy to a score of 1, calm to a score of 2, somewhat unhappy to a score of 3, very angry (or very unhappy) to a score of 4, crying (or very unhappy) to a score of 5, etc. In some embodiments, a score of -1 may indicate that the user's facial expression or emotion type has not been recognized. In other embodiments, the scores corresponding to the above-mentioned emotion types may also be set to other values, which are not limited here.

[0145] As an example, electronic devices can first determine a user's emotion type based on emotion perception data, and then determine matching thematic elements based on that emotion type. One way electronic devices determine a user's emotion type based on emotion perception data is by first determining a rating for the user's current emotion, and then determining the emotion type to which the user's current emotion belongs based on that rating. This rating could be, for example, rating 0, rating 1, rating 2, rating 3, rating 4, rating 5, etc., and the corresponding emotion type could be one of the emotion types exemplified by ratings 0, 1, 2, 3, 4, and 5, respectively. These details will not be elaborated further here.

[0146] In other embodiments, the gaze perception technology upon which the electronic device is based can combine saccade features (such as saccade rate, saccade amplitude, saccade frequency, etc.), gaze features (such as gaze duration, gaze area, gaze distance, etc.), and pupil features (such as pupil size, pupil dilation amplitude, pupil change rate, pupil stability, etc.) determined based on the aforementioned gaze perception data into a multi-dimensional emotion feature vector. This vector is then associated with preset emotion types to identify the user's emotion type, such as anger, tension, sadness, fatigue, curiosity, excitement, anxiety, fear, irritability, calmness, relaxation, pleasure, happiness, disgust, boredom, etc. In some embodiments, the preset emotion types can be labeled using emotion tags, which is not limited here.

[0147] It's understandable that the rate, amplitude, frequency, and fixation point distribution of saccades vary depending on a person's emotions. For example, a person's saccade rate is faster when angry or tense, and slower when sad or tired. Similarly, a person's saccade amplitude is larger when curious or excited, and smaller when anxious or fearful, potentially involving frequent, small-area saccades. Furthermore, a person's saccade rate is higher when tense or irritable, and lower when calm or relaxed. Additionally, longer fixation duration indicates higher concentration or stronger interest, and thus can also indicate pleasure or happiness. Shorter fixation duration indicates lower concentration or less interest, and thus can also indicate aversion or boredom.

[0148] In some embodiments, the electronic device can also determine the user's pupil characteristics based on the aforementioned gaze perception data, such as pupil size, pupil dilation / contraction amplitude, pupil rate of change, pupil offset position, and pupil stability. Here, pupil size refers to the diameter of the pupil. Pupil dilation / contraction amplitude refers to the amount of change in pupil dilation and constriction. Pupil rate of change characterizes how quickly the pupil size changes. Pupil offset position characterizes the amount of deviation of the pupil relative to an intraocular reference point. Pupil stability characterizes the degree of swaying or jittering in the pupil position.

[0149] In some embodiments, the aforementioned gaze perception data can also be used to determine the distribution of the user's gaze points, and then to analyze the user's saccade trajectory. This saccade trajectory can be used to predict the user's next gaze point position, etc. The prediction result can be used to preload thematic resources at the relevant gaze point positions. The details will be described below in conjunction with the relevant figures, and will not be elaborated here.

[0150] In some embodiments, the electronic device, with a gaze-aware interaction function (referred to as the first interaction function) enabled, can display a second theme element matching the emotion perception data within the current user's first gaze area, based on the aforementioned gaze perception data and emotion perception data. Alternatively, it can display a dynamic effect of switching between the first theme element and the second theme element within the first gaze area. For example, the electronic device can acquire gaze perception data when it detects that the first interaction function is enabled. This first interaction function adjusts the interaction function of the theme elements in the first interface based on the perceived result of the user's gaze behavior on the first interface. Furthermore, the electronic device can determine the first gaze area for the first interface based on the aforementioned gaze perception data. Displaying the second theme element or displaying a dynamic effect of switching between the first and second theme elements are two different ways of displaying the second theme element. The electronic device can determine which way to display the second theme element to use based on whether the user's gaze intensity on the first gaze area is high-intensity or low-intensity.

[0151] For example, an electronic device can determine to display a second subject element in the first gaze area based on the user's high-intensity gaze on the first gaze area. Alternatively, an electronic device can determine to display a dynamic effect of switching between a first subject element and a second subject element in the first gaze area based on the user's low-intensity gaze on the first gaze area.

[0152] It is understandable that when an electronic device displays a second theme element in the first viewing area, or displays a dynamic effect of switching between the first and second theme elements, if neither the second theme element nor the initially displayed first theme element in the first viewing area is a wallpaper element (e.g., both the first and second theme elements are sticker elements), the electronic device can remove the first theme element while displaying the second theme element. See the above for details. Figure 2A The diagram shows the interface changes of sticker elements matched with emotion perception data. This avoids the visually unappealing problem caused by overlapping theme elements.

[0153] When an electronic device displays a second theme element or a dynamic effect switching between the first and second theme elements within the first viewing area, if the first theme element originally displayed within the first viewing area is a wallpaper element, such as a portrait image, then the first theme element can remain displayed within the first viewing area while the second theme element is displayed. In this case, the second theme element can be displayed within the first viewing area, overlapping a portion of the first theme element, as described above. Figure 3A , Figure 3B and Figure 3DThe example shown illustrates how emoji stickers are displayed over the face area of ​​a person's image element, or you can refer to the above. Figure 3C The example shown illustrates how emoji stickers are displayed overlaid on the area near the face of a person's image element, where the second theme element can be... Figures 3A to 3D The example shown is an emoji sticker. This allows for the overlaying of emojis, text, and other sticker elements onto wallpaper elements, enhancing the visual appeal of the lock screen or desktop and improving the user experience.

[0154] The following description, in conjunction with the accompanying drawings, exemplarily illustrates the interface of an electronic device that enables gaze-aware interactive functionality based on the subject display method provided in this application.

[0155] As mentioned earlier, electronic devices can acquire the aforementioned gaze perception data and emotion perception data when the gaze perception interaction function is enabled. It is understood that electronic devices can avoid acquiring the user's gaze perception data when the gaze perception interaction function is not enabled, which helps save device power consumption and protect user privacy.

[0156] As an example, Figure 5A An embodiment of this application illustrates a schematic diagram of a theme settings interface when the gaze-aware interaction function is not enabled.

[0157] like Figure 5A As shown, users can enable the gaze-aware interaction function in the theme settings interface 510 displayed on the electronic device 10. As an example, the theme settings interface 510 displayed by the electronic device 10 based on the theme display method provided in this application may include an editing control 511 for adjusting the layout of elements in the theme template, an input control 512 for triggering the input of an image, an input control 513 for setting sticker elements, and a switch control 514 for setting the gaze-aware interaction function. The editing control 511 allows users to customize the layout of theme elements in the selected theme template, such as changing theme elements or adjusting their layout. The input control 512 allows users to select an image to set as wallpaper, which can replace the background image in the theme template selected by the user.

[0158] The input control 513 allows users to customize sticker elements in the selected theme template. For example, after clicking the input control 513, users can select their favorite stickers under the "My" option to display on the lock screen or desktop. The electronic device 10 can display a preview image of the selected sticker on the input control 513. In some embodiments, users can also delete the selected sticker element using the delete control 515 on the input control 513. Furthermore, users can also select their favorite sticker combinations, letter stickers, or symbol stickers under options such as "Combinations," "Letters," or "Symbols," without limitation.

[0159] In some embodiments of this application, the electronic device 10 can also generate personalized stickers that match the user's preferences based on the image input by the user. These personalized stickers can match the style of the image input by the user, or the expression of the person in the image input by the user, etc. In some embodiments, the generated personalized stickers can also be displayed, for example, on... Figure 5A Under the "My" option shown, there are no restrictions here.

[0160] It is understood that the images entered by users are not limited to those containing people, but can also be those that do not contain people, such as landscape images, anime images, paintings or graffiti images, etc. There are no restrictions here.

[0161] It's understandable that if the user-input image contains people, such as a selfie, it can be processed into a person image element on the lock screen or desktop, and this image element can be set as wallpaper. The number of people in such images, the intimacy between them, etc., can reflect relationships to some extent, such as parent-child relationships or romantic relationships. Furthermore, facial expressions can convey emotions. Based on these images, electronic devices can also generate stickers and other themed elements that match the emotions or relationships of the people in the image. If the user-input image does not contain people, such as a landscape photo, it can be processed into other wallpaper elements on the lock screen or desktop. These images can present a specific style, such as artistic, minimalist, punk, or mechanical. Based on these images, electronic devices can also generate stickers and other themed elements that match the style of the image. Thus, the stickers and themed elements generated from user-input images can better suit user preferences, improving the user experience.

[0162] Continue to refer to Figure 5AWhen the switch control 514 for the gaze-aware interaction function is in the "on" state, it can control the gaze-aware interaction function to be enabled. When the switch control 514 for the gaze-aware interaction function is in the "off" state, it can control the gaze-aware interaction function to be disabled. Users can control the gaze perception provided by the theme display method provided in this application by clicking or touching the switch control 514.

[0163] As an example, the on / off switch 514 for the gaze-aware interaction function displayed on the electronic device 10 can be in the off state by default. When the user clicks or touches the switch 514, the electronic device 10 can respond to the user's on / off operation by turning the switch 514 on and displaying... Figure 5B The theme settings interface shown is 510.

[0164] In some embodiments, the theme settings interface 510 may also include related controls for setting the background color of the theme interface, such as template color setting controls (not shown in the figure), etc., which are not limited here.

[0165] Figure 5B An embodiment of this application illustrates a schematic diagram of a theme settings interface after the gaze-aware interaction function is enabled.

[0166] like Figure 5B As shown, in response to a user's instruction to enable gaze-aware interaction, such as the user's operation of turning on the switch control 514 (e.g., clicking or touching), the electronic device 10 can display the switch control 514 in the "on" state. At this time, the user can continue to click or touch the application control 516 within the theme settings interface 510 displayed on the electronic device 10. In response to the user's click or touch operation on the application control 516, the electronic device 10 can apply the enabled gaze-aware interaction to the interface where the theme resources are applied, such as the lock screen or desktop.

[0167] As mentioned above, after enabling the gaze-aware interaction function, the electronic device 10 can acquire gaze-aware data and emotion-aware data based on the theme display method provided in this application, and dynamically adjust the theme elements on the lock screen or desktop, such as displaying the aforementioned... Figure 2A The interface change process shown, or the above display Figure 2B The interface changes shown interact with the user's gaze and emotional changes, thereby enhancing the user experience.

[0168] In other embodiments, based on the theme display method provided in this application, the electronic device can also support users to quickly enable or disable the gaze-aware interaction function through other interfaces. The interfaces that support quickly enabling or disabling the gaze-aware interaction function are hereinafter referred to as quick settings interfaces.

[0169] For example, when an electronic device is displaying a lock screen, it can respond to a user's instruction to enter the theme settings interface, displaying a theme settings interface that includes a gaze-aware interaction function switch, and then respond to the user's click or touch on the gaze-aware interaction function switch on the interface to turn the gaze-aware interaction function on or off.

[0170] As another example Figure 6 According to an embodiment of this application, a schematic diagram of interface changes for quickly enabling or disabling gaze-aware interaction is shown.

[0171] like Figure 6 As shown, when the electronic device 10 displays the lock screen interface 210, it can respond to the user's operation of entering the theme settings interface by pinching two fingers together, and display the unlock interface 610. After the user completes the fingerprint verification or enters the unlock password, the electronic device 10 can enter the quick settings interface 620 for setting the lock screen interface style.

[0172] It is understood that the quick settings interface 620 may include a switch control 621 for the gaze-aware interaction function. When the switch control 621 is in the on state, it can control the gaze-aware interaction function to be enabled. When the switch control 621 is in the off state, it can control the gaze-aware interaction function to be disabled.

[0173] The on / off switch 621 for the gaze-aware interaction function displayed on the electronic device 10 can be in an on or off state. For example, if the user has already used the above... Figure 5A If the switch control 514 in the theme settings interface 510 shown enables the gaze-aware interaction function, then... Figure 6 The toggle switch 621 for the gaze-aware interaction function in the quick settings interface 620 shown can be in the "on" state. For example, if the user has already... Figure 5A If the switch control 514 in the theme settings interface 510 shown disables the gaze-aware interaction function, or if the gaze-aware interaction function of the electronic device 10 is disabled by default, then... Figure 6 The on / off switch 621 for the gaze-aware interaction function in the quick settings interface 620 shown can be in the off state.

[0174] Continue to refer to Figure 6 Taking the switch control 621 in the closed state as an example, when the user clicks or touches the switch control 621, the electronic device 10 can respond to the user's opening operation by turning the switch control 621 into the open state, entering... Figure 6 The lock screen style settings interface shown is 630.

[0175] Continue to refer to Figure 6In the lock screen style settings interface 630, the on / off switch 621 for the gaze-aware interaction function is in the on state.

[0176] It is understood that in some embodiments, users can also switch the switch control 621 to the off state by clicking or touching it in the lock screen style settings interface 630, thereby controlling the disabling of the gaze-aware interaction function. This will not be elaborated here.

[0177] In other embodiments, based on the topic display method provided in this application, the electronic device can also support users to quickly enable or disable the gaze-aware interaction function through other interfaces.

[0178] For example, when an electronic device is displaying a desktop, it can respond to a user's instruction to display a control center interface, which includes a gaze-aware interaction function switch. Then, it can respond to the user's click or touch on the gaze-aware interaction function switch on the interface to turn the gaze-aware interaction function on or off.

[0179] As another example Figure 7 According to an embodiment of this application, a schematic diagram of interface changes for quickly enabling or disabling gaze-aware interaction is shown.

[0180] like Figure 7 As shown, when the electronic device 10 displays the desktop 710, it can display a control center interface 720 in response to an operation such as a user swiping down from the top right side of the desktop 710 to enter the control center. This control center interface 720 may include a switch control 721 for quickly turning gaze-aware interaction functions on or off.

[0181] Continue to refer to Figure 7 Taking the switch control 721 in the closed state as an example, when the user clicks or touches the switch control 721, the electronic device 10 can respond to the user's opening operation by turning the switch control 721 into the open state, entering... Figure 7 The control center interface 730 is shown. In the control center interface 730, the switch control 721 is in the on state, at which time the gaze-aware interaction function can be enabled, which will not be described in detail here.

[0182] As mentioned above, after enabling the gaze-aware interaction function, when the electronic device 10 enters the lock screen interface, the electronic device 10 can acquire gaze-aware data and emotion-aware data during the display of the lock screen interface based on the theme display method provided in this application, and dynamically adjust the theme elements on the lock screen interface or desktop, such as displaying the aforementioned... Figure 2A The interface change process shown, or the above display Figure 2BThe interface changes shown interact with the user's gaze and emotional state, thereby enhancing the user experience. It is understood that the user can put the electronic device 10 into a locked state by pressing the power button or lock screen button. At this time, the electronic device 10 can display the lock screen interface in response to the user picking up the device or pressing the power button or lock screen button; this will not be further limited or elaborated upon here.

[0183] contrast Figure 7 The switch control 721 in the control center interface 720 and the switch control 721 in the control center interface 730 are shown. It can be understood that in some embodiments, the icon style of the switch control 721 in the closed state and the switch control 721 in the open state may be different. For example, the icon style of the switch control 721 in the closed state may be a closed eye graphic, while the icon style of the switch control 721 in the open state may be an open eye graphic, etc.

[0184] Continue to refer to Figure 7 In some embodiments, users can also switch the switch control 721 to the off state by clicking or touching it in the control center interface 730. At this time, the icon style of the switch control can also be changed to... Figure 7 The icon style of the switch control 721 in the control center interface 720 shown here indicates that the gaze-aware interaction function can be turned off again, which will not be elaborated here.

[0185] In some embodiments, the aforementioned first theme element or second theme element may include one or more of the following: visual material elements, style configuration elements, dynamic effect elements, and interactive elements, without limitation. Visual material elements may include lock screen wallpapers, desktop wallpapers, application icons, desktop widgets, lock screen illustrations, decorative patterns, border materials, font resources, and background color schemes. Style configuration elements may include icon layouts, lock screen interface layouts, text styles, control styles, status bar or navigation bar styles, etc. Dynamic effect elements may include sweeping light effects, wallpaper dynamic particle visual effects, lock screen transition animations, unlocking animations, sliding gradient effects, and blurring effects, etc. As an example, the aforementioned sweeping light effect can be implemented by a rendering program (such as a shader) running on the GPU. This rendering program can be responsible for calculating and drawing the color, gradient, movement trajectory, and glowing effect of the screen sweeping light based on the position of the gaze point, thereby presenting a visual sweeping light effect.

[0186] In this embodiment, dynamic effects elements can support dynamic changes triggered by the user's gaze behavior. Interactive elements can include lock screen sliding effects, unlock gesture animations, desktop sliding transition effects, feedback animations for long presses or clicks, etc. In this embodiment, interactive elements can also include elements that present interactive effects matching the user's gaze behavior, without limitation.

[0187] It is understandable that the resources such as the first theme element and the second theme element mentioned above can be managed by the module in the system that performs theme resource management functions. This module can be integrated into the theme application or the settings application.

[0188] The electronic device provided in this application embodiment can run an operating system (OS). An operating system is a set of interconnected system software programs that manage and control the operation of electronic devices, utilize and run hardware and software resources, and provide public services to organize user interaction. In electronic devices, the operating system connects downwards to the physical devices at the hardware layer and provides a runtime environment for application software upwards.

[0189] An operating system typically includes a kernel layer, a middleware layer, and an application layer. The application layer includes applications, which can include system applications and third-party applications. The middleware layer includes a suite of software providing various services to application developers, or frameworks providing services such as databases, multimedia, and graphics, or capabilities such as distributed scheduling and system scaling. For example, the middleware layer may include a framework layer and / or a system service layer. The framework layer provides application programming interfaces (APIs) and programming frameworks for applications in the application layer. The system service layer includes the system's core capabilities, providing services to applications through the framework layer. The kernel layer is the layer between hardware and software. The kernel layer may include hardware drivers and the operating system kernel. In addition to providing hardware drivers, the kernel layer also supports functions such as memory management and system process management.

[0190] The electronic devices we use in our daily lives come in various types and forms, and are applied in a wide range of scenarios. Therefore, based on the different forms and functions of electronic devices, different application scenarios, and different user needs, the operating systems used in these devices may also differ. The basic functions implemented by the electronic device provided in this application can be achieved through a general-purpose operating system or a dedicated operating system.

[0191] To facilitate understanding of the specific implementation process of the topic display method provided in this application, the software structure of the operating system capable of implementing the topic display method provided in this application will be described below in conjunction with the relevant accompanying drawings.

[0192] As an example, Figure 8 A schematic diagram of the software structure of an operating system is shown according to an embodiment of this application.

[0193] It is understandable that the system architecture of the operating system running on an electronic device can be divided into several software structure layers. For example... Figure 8 As shown, the software structure layer of the operating system capable of implementing the theme display method provided in this application embodiment may include an application layer 810, a framework layer 820, a system service layer 830, and a kernel layer 840.

[0194] Application layer 810 may include application packages, such as themes, settings, and gallery applications 811. Some application packages, such as themes or settings applications, may include program data from the functional modules of theme resource management 812, without limitation. Theme resource management 812 can be used to perform management functions for theme-related resources, which may include functional components such as theme components and custom components. Custom components may include pre-set templates, images, stickers, and other resources.

[0195] In some embodiments, the template package can be stored in the template resource library managed by the resource manager 821, and the sticker package can be stored in the element resource library managed by the resource manager 821. The theme resource manager 812 can access the template resource library managed by the resource manager 821 to obtain the template package, and can access the element resource library managed by the resource manager 821 to obtain the sticker package.

[0196] For example, the theme component can support electronic devices in displaying theme menus and accessing theme settings interfaces. Templates in the custom component can support electronic devices in providing customizable theme templates when displaying theme settings interfaces. For example, electronic devices can... Figure 5A and Figure 5B The theme settings interface 510 shown provides theme templates. Users can select a theme template in this interface and customize the layout of theme elements within the selected theme template. Images in the custom component can be provided as wallpaper elements. In this embodiment, the custom component can also support the electronic device providing a trigger entry point for calling the gallery application in a relevant interface; this trigger entry point can refer to the above-described... Figure 5A and Figure 5BThe input control 512 and related descriptions in the theme settings interface 510 shown are not elaborated here. When a user clicks this trigger entry, they can select the image to input from the images managed by the gallery application. The stickers in the custom component can provide various types and styles of sticker elements. In this embodiment, the stickers may also include sticker elements generated based on the user-input image. These sticker elements can match the style of the user-input image or the facial expressions and other features of the people in the user-input image; no limitations are imposed here.

[0197] Gallery 811 includes images and can receive calls from theme resource management 812, providing users with image resources such as albums, cutouts, and collages for selection. Furthermore, Gallery 811 also includes a theme-adding component. This component allows electronic devices to add user-input images to theme-related resources in response to the user's confirmation of uploading selected images, such as saving them as theme template resources or theme element resources. It is understood that the aforementioned theme resources can include theme template resources, theme element resources, animation resources, visual effects resources, etc., without limitation.

[0198] The framework layer 820 may include a resource manager (RM) 821, a theme engine 822, and a data conversion interface based on a domain-specific language (DSL). The resource manager 821 may include functional components such as a preview image rendering component, a resource generation component, and a resource saving component. The preview image rendering component can render and display preview images of resources such as sticker elements generated based on user-input images. In some embodiments, the preview image may include the aforementioned... Figure 5A and Figure 5B The input control 513 in the theme settings interface 510 shows a preview image of the sticker selected by the user, as well as preview images of stickers under options such as "My", "Group", "Letter" or "Symbol", etc., without any restrictions.

[0199] The resource generation component can generate stickers and other theme element resources based on user-input images. The resource saving component can store the generated stickers and other theme element resources into corresponding resource packages. These resource packages can refer to the sticker, image, and template resource packages available in the custom components mentioned above. After completing the resource generation and saving functions, the resource manager 821 can notify the theme engine 822 to update resources, etc.

[0200] The theme engine 822 may include functional components such as a node rendering component, a visual effects update component, a face detection component, a gaze event handling component, an emotion detection component, a resource update component, and a theme element interaction component. The theme element interaction component can refer to the gaze-aware interaction function mentioned above, which allows users to enable or disable it. The resource update component can respond to notifications from the resource manager 821 and can be used to trigger updates to resources within the relevant resource packages managed by the theme resource manager 812. This ensures that users can see the latest theme-related resources in the theme settings interface and other related interfaces.

[0201] The node rendering component can draw and render the nodes corresponding to the thematic elements that need to be displayed on the interface based on the rendering node tree. In this embodiment, the node rendering function provided by the theme engine 822 can support electronic devices to independently render thematic elements within the user's gaze area, thereby enabling the display of thematic elements (such as second thematic elements) that match the emotion perception data within the user's gaze area (such as the first gaze area).

[0202] The visual effects update component can dynamically refresh specific effects such as the sweeping light effect on the relevant theme interface (such as the lock screen or desktop) according to changes in the user's gaze area. In some embodiments, when the theme engine 822 performs the visual effects update function, it can call the visual effects engine of the system service layer 830 to complete the resource loading and rendering processes of the relevant visual effects, which is not limited here.

[0203] The face detection component can detect parameters such as the size and position of the face region in the human image element set by the user as wallpaper. The gaze event handling component can process gaze events triggered by the user's gaze behavior, including determining the gaze area, gaze distance, gaze duration, etc. based on gaze perception data. For the specific processing of gaze perception data, please refer to S402 and related descriptions above, which will not be elaborated here.

[0204] The emotion detection component can detect the type and changes in a user's emotion when they are currently looking at the lock screen or desktop, as well as detect changes in the user's emotion over a period of time. The theme engine 822 executes the emotion detection component's function to detect raw emotion perception data, including the user's facial expression data and eye movement data. The aforementioned raw emotion perception data, as well as the facial expression features and eye movement features determined based on this raw data, can all be referred to as emotion perception data in this embodiment, that is, the first data used to characterize the user's current emotion.

[0205] Based on the domain-specific language (DSL) data conversion interface, user-invisible data processed by the framework layer 820 and system service layer 830 can be converted into user-visible data such as preview images displayed in the application interface using the domain-specific language (DSL).

[0206] The system service layer 830 may include structures such as system ability (SA), visual effects engine, and intelligent recommendation service. The system ability may include gaze perception capability and emotion perception capability. The gaze perception capability can acquire gaze perception data related to the user's gaze behavior (such as gazing at the first interface) based on the raw perception data provided by the kernel layer 840, and can determine the user's gaze area (such as the first gaze area), gaze duration, gaze distance, and analyze the user's gaze intensity based on this perception data. The emotion perception capability can acquire emotion perception data based on the raw perception data provided by the kernel layer 840, and determine the user's current emotion type or emotion tags to indicate the user's current emotion type.

[0207] The visual effects engine provides unified rendering capabilities and node rendering capabilities. It can also provide rendering capabilities for advanced and complex visual effects such as immersive shadows, particle streams, and frosted glass textures, as well as 3D spatial rendering capabilities, which will not be elaborated here.

[0208] The intelligent recommendation service can provide reasoning and analysis capabilities and recommendation functions. For example, the intelligent recommendation service can perform reasoning and analysis on the effect of applying the aforementioned stickers and other theme element resources generated based on user-input images to the lock screen or desktop, and recommend sticker resources with better effects to the resource manager 821, so that the resource manager 821 can perform preview image rendering and display the recommended sticker preview images on the relevant interface. In some embodiments, the aforementioned intelligent recommendation service can be provided by an artificial intelligence (AI) assistant running on the system. This service can call large models deployed in the cloud to provide the aforementioned reasoning and analysis capabilities and recommendation functions, which will not be limited or elaborated here.

[0209] Kernel layer 840 can drive hardware devices such as cameras to collect raw perceptual data. This raw data can include raw perceptual data related to the user's gaze behavior (referred to as gaze perception raw data) and raw perceptual data that can characterize the user's current emotion (referred to as emotion perception raw data). For example, kernel layer 840 can drive a low-power camera (such as a Swing camera) to collect gaze perception raw data and emotion perception raw data. The gaze perception raw data, after preprocessing, image processing, and processing by relevant algorithms provided by gaze perception technology, can form the aforementioned gaze perception data, i.e., the second data. The emotion perception raw data, after preprocessing, image processing, and processing by relevant algorithms provided by emotion perception technology, can form the aforementioned emotion perception data, i.e., the first data.

[0210] Based on the above Figure 8 The software structure shown below, combined with the interaction flowchart within the operating system, will be used to describe in detail the implementation process of the topic display method provided in this application.

[0211] Figure 9 According to an embodiment of this application, an interactive implementation flowchart of a theme display method that enables theme elements to interact with the user's emotions is provided.

[0212] like Figure 9 As shown, this interactive implementation process involves the interaction between the gallery 811, theme resource management 812, resource manager 821, and theme engine 822. The theme components provided by theme resource management 812 can be based on the above... Figure 8 The theme package implementation shown allows users to select theme function menus in the theme application interface or select theme settings options in the settings application interface. The template resource library and element resource library managed by Resource Manager 821 can be accessed in the above... Figure 8 After the resource manager 821 performs resource generation and saving, it updates the template resources or theme element resources in the resource library. Theme engine 822 can provide... Figure 8 The resource update functionality shown can also provide trackers for tracking user gaze behavior, renderers for rendering nodes of the main elements that need to be displayed, etc. The tracker can perform the above... Figure 8 The renderer can perform the functions of the gaze event handling component executed by the theme engine 822, as shown above. Figure 8 The node rendering component, face detection component, emotion detection component, and theme element interaction component in the theme engine 822 are shown.

[0213] Specifically, such as Figure 9 As shown, the interactive implementation process may include S901 to S927:

[0214] S901: Theme Resource Management 812 receives the user's operation to select a theme.

[0215] For example, the theme component of the theme resource management 812 can receive operations from the user selecting the theme function menu in the theme application interface, or selecting theme settings options in the settings application interface.

[0216] S902: Theme Resource Management 812 - Enter Customization.

[0217] For example, the theme component of the theme resource management 812 can respond to the user's operation of selecting a custom theme, call the custom component, enter the custom theme mode, and provide the user with a custom theme settings interface, which can refer to the above. Figure 5A and Figure 5B The theme settings interface 510 and related descriptions shown here will not be elaborated upon here.

[0218] S903: Theme Resource Management 812 receives the user's operation of selecting a theme template.

[0219] For example, the custom component of the theme resource management 812 can support electronic devices displaying at least one theme template for the user to select in a custom theme settings interface. Furthermore, the custom component of the theme resource management 812 can receive and respond to the user's selection of a theme template.

[0220] S904: Theme Resource Management 812 requests template resources from Resource Manager 821.

[0221] For example, a custom component of the theme resource management 812 can request access to the template resource library from the resource manager 821 and request the theme template resource selected by the user.

[0222] Correspondingly, resource manager 821 can respond to the request of theme resource manager 812 and return template resources to theme resource manager 812.

[0223] S905: Theme Resource Management 812 receives the user's operation of selecting theme elements.

[0224] For example, the custom component of the theme resource management 812 can receive the user's operation of selecting a favorite theme element in the theme settings interface, and respond to the operation event corresponding to the operation, and continue to execute S906 as described below.

[0225] It is understood that the theme elements selected by the user can include theme elements recommended by the system, or theme elements generated based on the image input by the user; there are no restrictions here.

[0226] S906: Theme resource management 812 requests theme element resources from resource manager 821.

[0227] For example, the custom component of the theme resource management 812 can respond to the user's operation of selecting a theme element by requesting access to the element resource library from the resource manager 821 and requesting to obtain the element resource corresponding to the theme element selected by the user.

[0228] S907: Resource Manager 821 returns the resource list to Subject Resource Manager 812.

[0229] For example, the resource list may include resources such as sticker previews recommended to the user for selection, which can be displayed in a relevant interface (such as a theme settings interface) for the user to choose from.

[0230] In some embodiments, the resource list containing the theme elements selected by the user returned by the resource manager 821 can be processed such as preview image rendering to become an element preview image that can be displayed on the theme settings interface, and then returned to the theme resource manager 812 as a resource list.

[0231] S908: Theme Resource Management 812 receives the user's input image.

[0232] For example, the custom component of the theme resource management 812 can support electronic devices to display a trigger entry point for users to input images in a custom theme settings interface, as described above. Figure 5A and Figure 5B The input control 512 is shown. Furthermore, the custom component of the theme resource management 812 can receive and respond to user input of images.

[0233] S909: Theme Resource Management 812 requests access to the image library from Image Library 811.

[0234] For example, the custom component of the theme resource management 812 can respond to the above-mentioned user input of an image, request access to the image library, and then display the image library interface for the user to select images managed by the image library 811.

[0235] S910: Image Library 811 returns a list of image resources to Theme Resource Management 812.

[0236] For example, the image resource list may include album images managed by Gallery 811 or all images managed by Gallery 811. In some embodiments, the images in the image resource list may be displayed in the theme settings interface for the user to select.

[0237] S911: Theme Resource Management 812 receives confirmation from the user regarding the selected image.

[0238] For example, a user can select the desired image from the returned image resource list and confirm the input (e.g., by clicking the confirmation button). At this point, the custom component of the theme resource management 812 can retrieve the selected image in response to the user's confirmation.

[0239] S912: Theme Resource Management 812 sends the user-inputted image to Theme Engine 822.

[0240] For example, the custom component of theme resource management 812 can send the image obtained from user input to the resource update component of theme engine 822.

[0241] S913: Theme Resource Management 812 sends a resource generation request to Theme Engine 822.

[0242] For example, the custom component of the theme resource management 812 can request the theme engine 822 to generate some theme element resources, such as sticker elements, based on the image input by the user.

[0243] S914: Theme Resource Management 812 generates element resources based on user-input images, such as sticker elements and other theme element resources.

[0244] In some embodiments, S913 to S914 described above may also be performed after S915 described below, without limitation.

[0245] S915: Theme Resource Management 812 receives user confirmation of the application's custom theme operation.

[0246] For example, after completing custom settings in the theme settings interface displayed on the electronic device, the user can apply the customized theme to the theme interface (such as the lock screen or desktop). At this time, the customization component of the theme resource management 812 can receive the user's confirmation of applying the customized theme settings. As an example, this operation could be performed by the user in the aforementioned... Figure 5B The operation of clicking the application control 516 in the theme settings interface 510 shown.

[0247] In this embodiment of the application, the user-confirmed custom theme settings of the application may include the setting to enable the gaze-aware interaction function.

[0248] S916: Theme Engine 822 generates element resources based on the user's current mood, such as theme element resources like emoji stickers.

[0249] For example, when the above-mentioned gaze-aware interaction function is enabled, the resource update component of the theme engine 822 can generate element resources according to the user's current mood.

[0250] S917: Theme engine 822 returns element resources to theme resource management 812. These element resources may include the element resources extracted from the image based on user input in S914, and may also include the element resources generated based on the user's current mood in S916.

[0251] S918: Theme Resource Management 812 requests Resource Manager 821 to save element resources.

[0252] For example, the resource saving component of the resource manager 821 may, in response to a request from the theme resource management 812, save the element resources extracted from the image based on user input and the element resources generated based on the user's current mood to the managed element resource library.

[0253] S919: The theme engine 822 acquires gaze perception data and tracks the user's gaze behavior.

[0254] For example, refer to the above Figure 8 The tracker of the theme engine 822 can be based on the above. Figure 8 The system service layer 830, as shown, uses its perception system capabilities to acquire gaze perception data. Furthermore, the tracker of the theme engine 822 can execute the functions of the gaze event processing component, processing gaze events generated based on the aforementioned gaze perception data, including determining the gaze area, gaze distance, and gaze duration based on the gaze perception data. Thus, the tracker of the theme engine 822 can track the user's gaze behavior based on the processed gaze characteristics such as gaze area, gaze distance, and gaze duration.

[0255] S920: The theme engine 822 determines the gaze region. This gaze region can be referenced from the gaze region processed in S919 above.

[0256] S921: The tracker of the theme engine 822 sends gaze region data to the renderer. This gaze region data can be, for example, the coordinate data of the gaze region, etc., and is not limited here.

[0257] S922: Theme Engine 822 requests the resource of the theme element to be displayed in the view area from Resource Manager 821.

[0258] For example, the aforementioned gaze area may be the aforementioned first gaze area, the subject element displayed within the aforementioned gaze area may be the aforementioned first subject element, and the subject element resource may be the element resource of the first subject element.

[0259] S923: Resource Manager 821 returns element resources that match the view area to Theme Engine 822, such as the element resources of the first theme element.

[0260] S924: Theme Engine 822 determines the theme elements to be displayed in the gaze area based on the user's current mood.

[0261] For example, the aforementioned theme element determined based on emotion can be the aforementioned second theme element.

[0262] S925: Theme Engine 822 requests the resource manager 821 to obtain theme element resources based on mood determination.

[0263] For example, the aforementioned theme element resources determined based on emotion can be element resources of a second theme element.

[0264] S926: Resource Manager 821 returns element resources that match the user's current mood to Theme Engine 822.

[0265] For example, the element resources that match the user's current mood, i.e., element resources that match the emotion perception data, such as emoji stickers, are not limited here.

[0266] S927: Theme Engine 822 completes element rendering and requests Theme Resource Management 812 to display theme elements that match the user's current mood.

[0267] For example, the renderer of the theme engine 822 can, after completing the rendering process of the relevant theme elements, request the theme component of the theme resource management 812 to display the theme elements that match the user's current mood, based on the elements that match the user's current mood. The way to display the second theme element may include displaying the second theme element in the gaze area (such as the first gaze area), or displaying a dynamic effect of switching between the second theme element and the first theme element originally displayed in the gaze area (such as the first gaze area). For details, please refer to the relevant description in S402 above, which will not be repeated here.

[0268] Based on the above-described interactive implementation process S901 to S927, the theme display method provided by this application can display theme elements representing the user's current emotions in the user's gaze area when the user is looking at the lock screen interface or desktop theme interface, thereby creating interaction between the theme interface and the user and improving the user experience.

[0269] In some embodiments, the theme display method provided in this application can determine the way to display theme elements that represent the user's current emotions in the gaze area based on the intensity of the user's gaze on each area in the theme interface such as the lock screen or desktop. For example, the theme element can be displayed directly in the gaze area when the gaze is intense, or the theme element can be displayed in the gaze area when the gaze is weak, or a dynamic effect can be used to switch between the theme element originally displayed in the gaze area and the theme element displayed in the gaze area.

[0270] In some embodiments, the theme display method provided in this application can determine, based on the intensity of the user's gaze at each area of ​​the theme interface such as the lock screen or desktop, whether to display theme elements with higher rendering complexity (which can be denoted as the third theme element) or theme elements with lower rendering complexity (which can be denoted as the fourth theme element). Rendering complexity refers to the comprehensive quantification of various consumption factors involved in the rendering output of a single frame of interface, scene, or visual element (such as the aforementioned theme element), including the number of rendering nodes, the number of layer levels, the number of effect overlays, the amount of graphics computation, the number of texture sampling times, the shader computation overhead, and the animation refresh frequency. It is used to characterize the computational load and resource consumption of the graphics rendering task.

[0271] As an example, Figure 10 According to an embodiment of this application, an interactive implementation flowchart of a theme display method capable of dynamically adjusting the rendering complexity of theme elements displayed within the gaze area is provided.

[0272] like Figure 10 As shown, this interactive implementation process involves the interaction between the gallery 811, theme resource management 812, resource manager 821, and theme engine 822. The theme resource management 812 can provide... Figure 8 The theme components shown allow users to select theme function menus in the theme application interface or select theme settings options in the settings application interface. The template resource library and element resource library managed by Resource Manager 821 can be found in the above... Figure 8 After the resource manager 821 performs resource generation and saving, it updates the template resources or theme element resources in the resource library. Theme engine 822 can provide... Figure 8 The resource update component shown can also provide a tracker for tracking user gaze behavior, an emotion detection component for detecting user emotions, and a renderer for rendering nodes of the subject elements that need to be displayed. The tracker can perform the above... Figure 8 The function of the gaze event handling component performed by the theme engine 822 is shown.

[0273] With the above Figure 9 The difference between the renderers involved in the interactive implementation process shown is that... Figure 10The renderer involved in the interactive flow shown can perform the above... Figure 8 The functions implemented by the node rendering component, face detection component, and theme element interaction component in the theme engine 822 are shown.

[0274] Specifically, such as Figure 10 As shown, the interactive implementation process may include S1001 to S1023:

[0275] S1001: Theme Resource Management 812 receives the user's selection of an image and instructs the generation of a theme template.

[0276] For example, the operation of the user selecting an image can refer to the operation and related description of the user inputting an image in S908 above, and the operation of the above instruction to generate a theme template can refer to the operation and related description of the user confirming the application of custom theme settings in S915 above, which will not be repeated here.

[0277] S1002: Theme Resource Management 812 requests Resource Manager 821 to save the generated template resources.

[0278] For example, a custom component of the theme resource management 812 may request the resource manager 821 to save the template resource in response to the above-mentioned instruction to generate a theme template.

[0279] S1003: Resource Manager 821 generates a template ID and saves the template resources. The template ID is the identifier for the theme template, used to mark the generated theme template so that it can be used as a calling identifier when loading the template resources of that theme template later.

[0280] For example, the resource saving component of the resource manager 821 can, in response to the request from the theme resource management 812, save the template resources of the generated theme template to the managed template resource library. It can be understood that after the resource manager 821 saves the template resources of the generated theme template, these template resources can be displayed in the customized theme settings interface for the user to select and use. That is, the saved template resources can be reused multiple times.

[0281] S1004: Theme Resource Management 812 received the user's operation of selecting a theme template.

[0282] For example, the above-mentioned operation of selecting a theme template can refer to the operation and related description of the user selecting a theme template in S903 above, and will not be repeated here.

[0283] S1005: Theme Resource Management 812 requests template resources from Resource Manager 821.

[0284] For example, a custom component of the theme resource management 812 can, in response to the user's operation of selecting a theme template, request the template resource of the theme template selected by the user from the resource manager 821.

[0285] S1006: Resource Manager 821 returns template resources to Theme Resource Manager 812.

[0286] For example, resource manager 821 may return template resources to theme resource manager 812 in response to a request from theme resource manager 812.

[0287] S1007: Theme Engine 822 acquires gaze perception data.

[0288] For example, the tracker of the theme engine 822 can acquire gaze-aware data and process gaze events generated based on that data.

[0289] S1008: Theme Engine 822 analyzes gaze characteristics.

[0290] For example, gaze characteristics may include gaze duration, gaze area, gaze distance, etc. The tracker of the topic engine 822 can analyze the above gaze characteristics based on the acquired gaze perception data.

[0291] S1009: Theme Engine 822 determines the intensity of the user's gaze at each gaze region.

[0292] For example, the tracker of the theme engine 822 can further determine the user's gaze intensity on the gaze region based on the gaze characteristics analyzed above. In some embodiments, as the user's gaze moves across different areas of the theme interface, the electronic device can detect multiple gaze regions. In this case, the tracker of the theme engine 822 can determine the user's gaze intensity on each gaze region based on the gaze characteristics analyzed above.

[0293] S1010: The tracker of the topic engine 822 sends gaze features to the emotion detection component.

[0294] For example, the tracker of the theme engine 822 can send the aforementioned gaze features to the emotion detection component to notify the emotion detection component that the user is looking at the screen. In some embodiments, the tracker can also send gaze events to the emotion detection component based on received gaze events to notify the emotion detection component that the user is looking at the screen.

[0295] S1011: Theme Engine 822 acquires emotion perception data.

[0296] For example, the emotion detection component of the theme engine 822 can determine that the user is looking at the screen based on the received gaze features, and then obtain emotion perception data to detect the user's current emotion.

[0297] S1012: Theme Engine 822 analyzes emotion types or detects emotion changes.

[0298] For example, the emotion detection component of the theme engine 822 can analyze the emotion type or detect emotion changes based on the acquired emotion perception data. In some embodiments, the emotion detection component of the theme engine 822 can perform emotion scoring based on the currently acquired emotion perception data according to a preset emotion scoring mechanism, and then analyze the emotion type corresponding to the emotion score. In other embodiments, the emotion detection component of the theme engine 822 can also analyze and determine the emotion type of the user's current emotion based on a multi-dimensional emotion feature vector, as described in the relevant description in S402 above, which will not be repeated here.

[0299] S1013: The emotion detection component of Theme Engine 822 notifies the renderer of the detected emotion type.

[0300] S1014: Theme Engine 822 requests the Resource Manager 821 to retrieve theme elements that match the mood type.

[0301] For example, the renderer of the theme engine 822 can request the theme element that matches the emotion type from the resource manager 821 based on the emotion type reported by the emotion detection component.

[0302] In some embodiments, if the electronic device's sensing system fails to detect the user's facial expression or fails to detect the user's face, it can report an emotion score of -1 to indicate that the user's facial expression or emotion type was not identified. In this case, refer to the above... Figure 8 The theme engine 822 in the framework layer 820 of the operating system of electronic devices may not need to load theme element resources.

[0303] S1015: File Explorer 821 matches theme elements based on mood type.

[0304] For example, the element resource library managed by the resource manager 821 includes theme element resources such as sticker elements that correspond to or match emotion types. Therefore, the resource manager 821 can search for matching theme elements in the element resource library based on the detected emotion type.

[0305] In some embodiments, the resource manager 821 may also search for matching special effects elements, such as sweeping light effects, from the element resource library based on features such as emotion type and changes in the user's gaze area, without limitation.

[0306] S1016: Resource Manager 821 returns element resources that match the mood type to Theme Engine 822.

[0307] For example, the resource manager 821 can return the found element resources that match the mood type to the renderer of the theme engine 822, including theme element resources such as sticker elements and effect elements.

[0308] S1017: Theme Engine 822 completes element rendering and requests Theme Resource Management 812 to display theme elements that match the user's current mood.

[0309] For example, the renderer of the theme engine 822 can complete the rendering process of the theme elements that match the mood type, and request the theme component of the theme resource management 812 to display the rendered theme elements, such as the second theme element mentioned above. For details, please refer to the relevant description in S927 above, which will not be repeated here.

[0310] The aforementioned element resources that match the user's current mood, i.e. element resources that match the emotion perception data, such as emoji stickers, etc.

[0311] S1018: Theme Resource Management 812 refreshes the theme interface display effect.

[0312] For example, the theme component of the theme resource management 812 can display the rendered theme elements on the relevant theme interface based on the request of the renderer of the theme engine 822, and the display effect of the relevant theme interface will change, that is, refresh the display effect of the theme interface.

[0313] It's understandable that the refreshed theme interface mentioned above could be, for example, the lock screen or the home screen. The lock screen interface can be referenced as described above. Figures 2A to 2B ,as well as Figures 3A to 3D The lock screen interface shown.

[0314] S1019: The tracker of Theme Engine 822 sends the user's gaze intensity for each gaze region to the renderer.

[0315] In some embodiments, S1019 may be performed before S1017.

[0316] Thus, after the renderer of theme engine 822 completes the rendering of the theme elements matching the emotion type, it can request the theme resource management 812 to display a second theme element based on the intensity of the user's gaze on each gaze area. For example, for a gaze area with high intensity, theme engine 822 can request the theme component of theme resource management 812 to display a second theme element. At this time, the theme element originally displayed in the gaze area displayed by the second theme element can be removed or covered by the second theme element. As another example, for a gaze area with low intensity, theme engine 822 can request the theme component of theme resource management 812 to display a second theme element and a dynamic effect of switching between it and the theme element originally displayed in the gaze area.

[0317] S1020: Theme Engine 822 requests theme elements from Resource Manager 821 that match the gaze intensity.

[0318] For example, the renderer of the theme engine 822 may request a theme element that matches the gaze intensity from the resource manager 821 based on whether the received gaze intensity indication is high-intensity gaze or low-intensity gaze.

[0319] For example, for areas of intense gaze, the renderer of Theme Engine 822 can request to render more complex theme elements to achieve a better visual effect (such as a more dazzling) for the theme interface. For areas of low-intensity gaze, the renderer of Theme Engine 822 can request to render less complex theme elements to achieve reasonable allocation of rendering resources and improve the rendering efficiency of theme elements and the theme interface.

[0320] S1021: Resource Manager 821 returns element resources that match the gaze intensity to Theme Engine 822.

[0321] S1022: Theme Engine 822 completes element rendering and requests Theme Resource Management 812 to display theme elements that match the gaze intensity.

[0322] S1023: Theme Resource Management 812 refreshes the theme interface display effect.

[0323] Based on the above-described interactive implementation process S1001 to S1023, the theme display method provided by this application can detect changes in the user's emotions when the user is looking at the lock screen interface or desktop and refresh the theme elements in the viewing area according to the user's current emotion type, so that the theme interface and the user can interact, which is beneficial to improving the user experience.

[0324] Furthermore, based on the interactive implementation process of S1001 to S1023 described above, the theme display method provided by this application can also dynamically adjust the rendering complexity of the theme elements displayed in the gaze area according to the changes in the user's gaze intensity. The theme elements with higher rendering complexity are displayed in the gaze area with high intensity gaze, and the theme elements with lower rendering complexity are displayed in the gaze area with low intensity gaze. In this way, while improving the user's visual experience, rendering resources can be reasonably allocated to improve the rendering efficiency of theme elements and theme interface.

[0325] In some embodiments, the theme display method provided in this application can determine whether to display theme elements with higher rendering complexity or theme elements with lower rendering complexity in different areas of the theme interface, based on the intensity of the user's gaze at each area of ​​the theme interface such as the lock screen or desktop. To improve the smoothness of rendering and displaying theme elements with different rendering complexities, the electronic device can preload the theme element resources that will be refreshed and displayed in the theme interface.

[0326] As an example, Figure 11 According to an embodiment of this application, a schematic diagram illustrating the implementation process and principle of a theme display method for preloading theme element resources is provided.

[0327] like Figure 11 As shown, the system architecture of the operating system running on the electronic device used to implement the subject display method provided in this application can be referred to the above. Figure 8 As shown, it includes an application layer 810, a framework layer 820, a system service layer 830, and a kernel layer 840.

[0328] Referring to the above Figure 8 As shown and described, the kernel layer 840 can provide raw perceptual data, which may include raw gaze perception data and raw emotion perception data. The system service layer 830 can provide perceptual system capabilities, including gaze perception capabilities for processing gaze perception data and emotion perception capabilities for processing emotion perception data. Based on the processing results of the above gaze perception data and emotion perception data, the application layer 810 can execute steps S1101 to S1103, and the framework layer 820 can execute steps S1104 to S1106.

[0329] Specifically, such as Figure 11 As shown, the implementation process may include S1101 to S1106:

[0330] S1101: Displays the theme settings interface in response to the user's selection of a theme function.

[0331] For example, the theme settings interface can be referenced as described above. Figure 5A and Figure 5B The example interface.

[0332] Refer to the above Figure 8 The theme resource management 812 in the application layer 810 of the electronic device's operating system can display a theme settings interface in response to the user's selection of the theme function. As mentioned earlier, the user can select the theme function menu in the interface of the theme application running on the electronic device, or select the theme settings option in the interface of the settings application and select custom theme to enter the aforementioned theme settings interface.

[0333] S1102: In response to the user's operation of turning on the switch, the gaze-aware interaction function is enabled.

[0334] For example, refer to the above Figures 5A to 5B Users can turn on the gaze-aware interaction function by turning on the switch control 514.

[0335] Refer to the above Figure 8 The theme resource management 812 in the application layer 810 of the electronic device's operating system can respond to the user's operation of turning the switch control 514 on, i.e., the on operation, and can enable the gaze-aware interaction function.

[0336] S1103: Load template resources and element resources in response to user-defined actions.

[0337] For example, refer to the above Figure 5A or Figure 5B Users can load template resources, image resources, and theme element resources by clicking or touching controls such as editing control 511, input control 512, and input control 513, as well as by clicking or touching controls used to set theme templates (not shown in the figure), to complete the setting of a custom theme. In some embodiments, after the custom theme is stored in the template resources, the template resources may include image resources. That is, if the user uses the theme template of the custom theme and does not enter a new image, there is no need to load image resources.

[0338] Refer to the above Figure 8 The custom component of theme resource management 812 in the application layer 810 of the operating system of electronic devices can complete the above-mentioned processes such as loading template resources and element resources.

[0339] S1104: After applying a custom theme, detect whether the user is currently gazing.

[0340] For example, after applying a custom theme, the electronic device can sense whether the user is currently gazing based on the activation of the aforementioned gaze-aware interaction function.

[0341] Refer to the above Figure 8The theme engine 822 in the framework layer 820 of the operating system of the electronic device can process gaze events and perform emotion detection based on the processing results of gaze perception data and emotion perception data provided by the system service layer 830.

[0342] It is understandable that if the electronic device's sensing system detects that the gaze duration is too short, it may not report the gaze event to frame layer 820. In this case, refer to the above... Figure 8 The theme engine 822 in the framework layer 820 of the operating system of electronic devices may not load theme element resources.

[0343] S1105: Perceive the user's gaze behavior and predict the area where the user's next gaze point will be located.

[0344] For example, an electronic device can track a user's gaze behavior over a period of time, such as tracking the gaze area focused by the user's gaze point, thereby fitting a gaze trajectory. Furthermore, based on this gaze trajectory, the electronic device can predict the area where the user's next gaze point will be located, i.e., predict the next gaze area.

[0345] Refer to the above Figure 9 or Figure 10 The theme engine 822 of the framework layer 820 may include a tracker for tracking the user's gaze behavior. The tracker can fit a gaze trajectory based on the user's gaze behavior over a period of time and predict the area where the user's next gaze point will be located, i.e., predict the next gaze area.

[0346] S1106: Preload subject element resources based on the region where the next fove is located.

[0347] For example, the electronic device can preload the subject element resources displayed in the area where the next gaze point is located, based on the prediction results of the area predicted above.

[0348] Refer to the above Figure 9 or Figure 10 The theme engine 822 of the operating system framework layer 820 of the electronic device may also include a renderer for rendering theme elements. The tracker of the theme engine 822 can send the prediction result of the predicted area where the next foveation point is located to the renderer, and the renderer can request element resources from the resource manager 821 based on the prediction result. The element resources may be element resources of theme elements that will be displayed in the area where the next foveation point is located, such as element resources of theme elements with high rendering complexity.

[0349] Thus, the process of preloading element resources described above can prepare theme elements in advance for the area where the user's next focus will be, which helps to improve the rendering efficiency of theme elements and thus improves the smoothness of the refresh of related theme interfaces.

[0350] As an example, the above Figure 8 The operating system shown can be any operating system used in industry, such as an operating system developed based on OpenHarmony, such as HarmonyOSTM; or other operating systems such as Android™, iOS™ mobile operating systems; it can also be any open source operating system or its derivative operating system, such as Linux OS, and other embedded operating systems; or it can be a new type of operating system in the future, such as an AI operating system based on artificial intelligence.

[0351] The following section uses HarmonyOSTM, an operating system developed based on OpenHarmony, as an example, and describes the system architecture of the operating system for industrial applications to which the subject display method provided in this application is applicable, in conjunction with the accompanying drawings.

[0352] As an example, Figure 12 The present application illustrates a system architecture for HarmonyOSTM. Those skilled in the art can deduce implementations of the present application's embodiments under other specific operating systems, such as Android™.

[0353] like Figure 12 As shown, the software architecture of the HarmonyOSTM system can be divided into several layers. In some embodiments, from bottom to top, these layers are: kernel layer, system service layer, framework layer, and application layer. Layers communicate with each other through software interfaces. System functions can be tailored, added, or combined at the subsystem granularity in deployment scenarios with different device forms. Each subsystem can also be tailored, added, or combined at the functional granularity.

[0354] The kernel layer can include the kernel abstract layer (KAL), kernel subsystems, and driver subsystems. The KAL provides basic kernel capabilities to higher layers by shielding the differences between multiple kernels, including but not limited to process / thread management, memory management, file system management, network management, and peripheral device management.

[0355] Kernel Subsystem: Supports the selection of a suitable OS kernel for different resource-constrained devices, including but not limited to Linux™ kernel, Harmony™ kernel, and Lite OS.

[0356] Driver Subsystem: The driver framework is the foundation for the open system hardware ecosystem, providing unified peripheral access capabilities and a framework for driver development and management. The driver framework includes: display drivers, camera drivers, audio drivers, Bluetooth drivers, sensor drivers, etc.

[0357] The system service layer comprises the core capabilities of the system, providing services to applications through the framework layer. This layer includes, but is not limited to, the following subsystems:

[0358] The system's basic capability subsystem set provides fundamental capabilities for the operation, scheduling, and migration of distributed applications across multiple devices. This set may include distributed soft bus, distributed data management, distributed task scheduling, and Ark multi-language runtime; it may also include multi-modal input subsystem, graphics subsystem, security subsystem, and AI subsystem.

[0359] The AI ​​subsystem provides a unified artificial intelligence (AI) engine framework, enabling rapid plug-in integration of algorithm capabilities. The framework mainly includes modules for plug-in management, module management, and communication management, providing lifecycle management and on-demand deployment of AI algorithm capabilities. In this embodiment, the AI ​​subsystem can provide the above-mentioned... Figure 8 The theme engine 822 shown above; its specific functions can be found in the above description. Figure 8 The descriptions and related information are not repeated here. In some embodiments, the AI ​​subsystem may also provide a face detection function to detect face regions, such as whether wallpaper elements in the theme interface contain face regions, the size of the face regions, and the position parameters of the face regions in the corresponding theme templates, etc., without limitation.

[0360] Basic software service subsystem set: provides public and general software services; the basic software service subsystem set may include event notification subsystem, telephone service subsystem, multimedia subsystem, etc.

[0361] In this embodiment of the application, the basic software service subsystem set can also provide a mobile sensing development platform (MSDP) subsystem. The aforementioned subsystem can be deployed in the MSDP subsystem. Figure 8 The perception system capabilities (SA) in the system service layer 830 shown may include gaze perception capabilities, emotion perception capabilities, etc. The specific functions of these system capabilities can be referred to the above. Figure 8 The descriptions and related information shown will not be repeated here.

[0362] Enhanced software service subsystem suite: Provides differentiated enhanced software services for different devices; the enhanced software service subsystem suite may include smart screen proprietary business subsystem, wearable proprietary business subsystem, IoT proprietary business subsystem, etc.

[0363] Hardware service subsystem set: Provides hardware services; the hardware service subsystem set may include location service subsystem, unified identity and access management (IAM) subsystem, wearable proprietary hardware service subsystem, biometric identification, IoT proprietary hardware service and other subsystems.

[0364] Distributed task scheduling enables distributed service management (discovery, synchronization, registration, and invocation), supporting remote startup, remote invocation, remote connection, and migration of applications across devices.

[0365] Distributed data management enables data synchronization, data storage, data sharing, and data access across all scenarios and devices.

[0366] The distributed soft bus provides communication-related capabilities for seamless interconnection between multiple devices, including: WLAN service capabilities, Bluetooth service capabilities, soft bus, remote procedure call (RPC), and StarFlash communication capabilities.

[0367] Ark Multilingual Runtime is a unified compilation runtime platform designed to support the joint compilation and execution of multiple programming languages ​​and multiple chip platforms.

[0368] The framework layer provides application programming interfaces (APIs) and programming frameworks for applications in the application layer. The framework layer includes: the Ark UI framework (which provides a complete infrastructure for UI development of system applications, including UI functions such as components, layouts, animations, and interactive events, as well as interface preview tools), the user application framework, and the Ability framework (an Ability is a lightweight application; the Ability framework schedules and manages the operation and lifecycle of Abilities). Different devices may have different operating systems, and the APIs they support may also differ.

[0369] Ark UI provides a complete infrastructure for UI development of system applications, including UI functions (such as components, layouts, animations, and interactive events) and interface preview tools. The Ark UI framework provides at least two development paradigms, including a declarative development paradigm based on Ark TS (referred to as "declarative development paradigm") and a JS-compatible web-like development paradigm (referred to as "web-like development paradigm").

[0370] An application framework (AF) is a programming framework that can be an implementation of an application model. Developers can use an application framework to implement the structure and behavior of an application, which is described by the application model. An application model is a model of an application used to represent different aspects of the application, such as application components, process models, thread models, task management, and package management. The application model provides a unified language and architecture for describing various aspects of the application.

[0371] The Ability framework enables the scheduling and management of the operation and lifecycle of Abilities (meta-capabilities). An Ability is a fundamental component of an application, the smallest unit for system-scheduled applications, and a component capable of performing an independent function. An application can contain one or more Abilities.

[0372] Abilities are divided into two types: meta-programs (feature ability, FA) and meta-services (particle ability, PA). An application can consist of one or more FAs or PAs. FAs have a user interface, providing the ability to interact with the user; while PAs do not have a user interface, but provide the ability to run background tasks and a unified data access abstraction. The aforementioned perception system capabilities (SA) can be encapsulated externally as PAs, without any restrictions here.

[0373] The HarmonyOS API is a series of open capabilities provided to support HarmonyOS application development. The HarmonyOS API can be set at the framework layer or independently of the framework layer. The HarmonyOS API includes the Audio API (audio service), Push API (push service), and Account API (account service), etc. In this embodiment, the HarmonyOS API may include the above-mentioned... Figure 8 The data conversion interface based on a domain-specific language (DSL) shown above can be found in the detailed description of its functions. Figure 8 The descriptions and related information shown will not be repeated here.

[0374] Applications may include system applications and extended / third-party applications. System applications may include the desktop, control bar, settings, contacts, phone, camera, etc., while extended / third-party applications may include social applications, travel applications, etc. In this embodiment, system applications may also include applications such as themes and gallery. In other embodiments, extended / third-party applications may also include applications such as themes.

[0375] As mentioned earlier, settings, themes, and other applications can include the above. Figure 8 The topic resource management 812 and other functional modules shown above can be referenced for their specific functions. Figure 8 The descriptions and related information shown will not be repeated here.

[0376] Based on the above Figure 8 and Figure 11 The system architecture shown below, in conjunction with the accompanying drawings, provides a summary explanation of the implementation principle of the topic display method provided in this application.

[0377] As an example, Figure 13 An embodiment of this application illustrates a schematic diagram of the implementation principle of a theme display method in an operating system.

[0378] like Figure 13 As shown, when the aforementioned gaze-aware interaction function is enabled, the user's gaze at the screen can trigger the gaze-aware capability within the perception system to report gaze-aware data. Simultaneously, the user's gaze at the screen can trigger the emotion-aware capability within the perception system to report emotion-aware data.

[0379] The perception system can register gaze events and emotion events with the Mobile Sensing Platform (MSDP), which in turn registers these events with the theme engine. Gaze events can be system events triggered by the user's gaze at the screen (or the primary interface) as perceived by the system's gaze perception capabilities. Emotion events can be system events triggered by the user's current emotion type or emotional changes as perceived by the system's emotion perception capabilities.

[0380] Resource generation components can provide data support for the capabilities of the perception system. (Refer to the above.) Figure 8 The resource generation component can be deployed in the resource manager 821. In some embodiments, a resource preloading component (not shown in the figure) can also be deployed in the resource manager 821 to provide data input for the gaze region prediction module. Refer to the above. Figure 8The aforementioned gaze region prediction module can be deployed in the gaze event processing component of the theme engine 822. It can fit the gaze trajectory based on the gaze perception data reported by the gaze event, and predict the region where the next gaze point is located (i.e. the predicted gaze region), and then issue a resource preloading instruction to the node rendering component or renderer of the theme engine.

[0381] In some embodiments, the gaze region prediction module can also determine the user's gaze region during the gaze of the topic interface based on saccade characteristics such as saccade rate, saccade amplitude, and saccade frequency analyzed from the aforementioned gaze perception data. For example, the gaze region prediction module can analyze the distance of a single gaze movement based on saccade amplitude, the speed of the user's gaze movement based on saccade rate, and the frequency of the user's gaze switching based on saccade frequency. A larger saccade amplitude, a faster saccade rate, and a higher saccade frequency indicate more dispersed gaze activity, and the corresponding gaze region area is larger. Based on this, the electronic device can determine the user's gaze region and calculate the area of ​​the user's gaze region based on the gaze region prediction module.

[0382] When lock screen events and emotion events are triggered, the theme engine can respond to the lock screen event, preload theme element resources, etc., and drive the visual effects engine to execute theme interface rendering and visual effects output, completing the dynamic interactive display between theme elements in the lock screen interface and the user's gaze behavior.

[0383] Based on this, the emotion detection component in the theme engine can also detect the user's current emotion, and thus can be used to... Figure 13 The principle illustrated involves preloading the theme element resources used to represent the user's current emotion, rendering the theme elements, and displaying them. For details, please refer to the above. Figures 9 to 11 The relevant descriptions in the process shown are not repeated here.

[0384] The following description, in conjunction with the accompanying drawings, provides an exemplary description of the electronic device hardware structure to which the subject display method provided in this application applies.

[0385] As an example, Figure 14 A schematic diagram of the hardware structure of an electronic device is shown according to an embodiment of this application.

[0386] In the embodiments of this application, Figure 14 The electronic device 100 shown can be the one described above. Figures 2A to 2B , Figures 3A to 3D , Figures 5A to 5B , Figure 6 as well as Figure 7 The electronic devices 10 shown, such as mobile phones (including foldable screen phones) and tablets (including foldable screen tablets), are not limited here.

[0387] like Figure 14As shown, the electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, an antenna, a communication module 130, a sensor module 140, buttons 150, a camera 160, and a display screen 170, etc. The sensor module 140 may include a pressure sensor 140A, a distance sensor 140B, a touch sensor 140C, etc.

[0388] It is understood that the structures illustrated in the embodiments of this application do not constitute a limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0389] Processor 110 may include one or more processing units, such as application processors (APs), modem processors, graphics processing units (GPUs), image signal processors (ISPs), controllers, video codecs, digital signal processors (DSPs), baseband processors, and / or neural network processing units (NPUs). These different processing units may be independent devices or integrated into one or more processors.

[0390] The controller can generate operation control signals based on the instruction opcode and timing signals to complete the control of instruction fetching and execution.

[0391] The processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can retrieve it directly from the aforementioned memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0392] In this embodiment, the processor 110 of the electronic device 100 can generate operation control signals through the aforementioned controller to complete the operation of the aforementioned... Figure 4 The process shown controls the fetching and execution of instructions for each step in the flowchart, or completes the above-mentioned... Figures 9 to 11The instructions for each step in the process shown are used to control the fetching and execution of instructions in order to implement the subject display method provided in this application.

[0393] It is understood that the interface connection relationships between the modules illustrated in the embodiments of this application are merely illustrative and do not constitute a structural limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may also employ different interface connection methods or combinations of multiple interface connection methods as described in the above embodiments.

[0394] The wireless communication function of electronic device 100 can be implemented through antenna, communication module 130 and baseband processor, etc., to support electronic device 100 to access topic resource database to obtain topic elements, etc., which will not be elaborated here.

[0395] Electronic device 100 implements display functions through a GPU, a display screen 170, and an application processor. The GPU is a microprocessor for image processing, connected to the display screen 170 and the application processor. The GPU performs mathematical and geometric calculations and is used for graphics rendering. Processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information. The display screen 170 is used to display images, videos, etc.

[0396] In this embodiment, the electronic device 100, through the display functions implemented by the GPU, display screen 170, and application processor, can be used to display the aforementioned... Figures 2A to 2B , Figures 3A to 3D , Figures 5A to 5B , Figure 6 as well as Figure 7 The interface shown.

[0397] The electronic device 100 can perform shooting functions through an ISP, camera 160, video codec, GPU, display screen 170, and application processor. In this embodiment, the electronic device 100 can sense the user's gaze behavior through the camera 160 and collect gaze perception data and emotion perception data, etc.

[0398] The external storage interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external storage interface 120 to perform data storage functions. For example, music, video, and other files can be saved on the external memory card.

[0399] Internal memory 121 can be used to store computer executable program code, which includes instructions. Processor 110 performs various functional applications and data processing of electronic device 100 by executing instructions stored in internal memory 121 and / or instructions stored in memory disposed in the processor.

[0400] Pressure sensor 140A is used to sense pressure signals and convert them into electrical signals. In some embodiments, pressure sensor 140A can be disposed on display screen 170. There are many types of pressure sensors 140A, such as resistive pressure sensors, inductive pressure sensors, and capacitive pressure sensors. A capacitive pressure sensor may include at least two parallel plates with conductive material. When force is applied to pressure sensor 140A, the capacitance between the electrodes changes. Electronic device 100 determines the pressure intensity based on the change in capacitance. When a touch operation is applied to display screen 170, electronic device 100 detects the intensity of the touch operation based on pressure sensor 140A. Electronic device 100 can also calculate the touch position based on the detection signal from pressure sensor 140A. In some embodiments, touch operations applied to the same touch position but with different touch operation intensities can correspond to different operation commands.

[0401] Distance sensor 140B is used to measure distance. Electronic device 100 can measure distance using infrared or laser. In this embodiment, electronic device 100 can use distance sensor 140B to measure the user's gaze distance, that is, the distance between the user's eyes and the screen of electronic device 100 when the user gazes at the main interface (i.e., the first interface).

[0402] Touch sensor 140C, also known as a "touch device," can be located on display screen 170. The touch sensor 140C and display screen 170 together form a touchscreen, also known as a "touchscreen." Touch sensor 140C detects touch operations applied to or near it. The touch sensor can transmit the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through display screen 170. In other embodiments, touch sensor 140C may also be located on the surface of electronic device 100, in a different position than display screen 170.

[0403] In this embodiment, the electronic device 100 can detect user clicks or touches on the switch control of the gaze-aware interactive function based on the pressure sensor 140A and touch sensor 140C, and then turn the gaze-aware interactive function on or off in response to the operation.

[0404] Button 150 includes a power button, volume buttons, etc. Button 150 can be a mechanical button or a touch button. Electronic device 100 can receive button input and generate key signal inputs related to user settings and function control of electronic device 100. In this embodiment, button 150 may include the aforementioned power button or screen lock button. In some embodiments, the function of the power button or screen lock button may be integrated into the aforementioned power button; this is not limited here.

[0405] This application also provides an apparatus, including a module for performing the topic display method provided in the specific embodiments described above.

[0406] This application also provides a chip, including a processor coupled to a memory, for executing a computer program or instructions stored in the memory, so that the chip executes the topic display method provided in the above specific embodiments.

[0407] This application also provides a computer-readable storage medium storing instructions that, when executed on a computer, cause the computer to perform the topic display method provided in the above-described specific embodiments.

[0408] This application also provides a computer program product, including a computer program / instruction, which, when executed by a processor, is used to implement the topic display method provided in the specific embodiments described above.

[0409] Various embodiments of the mechanisms disclosed in this application can be implemented in hardware, software, firmware, or combinations of these implementation methods. Embodiments of this application can be implemented as computer program modules or module code executable on a programmable system, the programmable system including at least one processor, a storage system (including volatile and non-volatile memory and / or storage elements), at least one input device, and at least one output device.

[0410] Computer program modules or module code can be applied to input instructions to perform the functions described in this application and generate output information. The output information can be applied to one or more output devices in a known manner. For the purposes of this application, the processing system includes any system having a processor such as, for example, a digital signal processor (DSP), a microcontroller, an application-specific integrated circuit (ASIC), or a microprocessor.

[0411] Module code can be implemented using a high-level modular language or an object-oriented programming language to communicate with the processing system. Assembly language or machine language can also be used to implement module code when needed. The mechanisms described in this application are not limited to any particular programming language. In either case, the language can be a compiled language or an interpreted language.

[0412] In some cases, the disclosed embodiments may be implemented in hardware, firmware, software, or any combination thereof. The disclosed embodiments may also be implemented as instructions carried or stored thereon on one or more temporary or non-temporary machine-readable (e.g., computer-readable) storage media, which may be read and executed by one or more processors. For example, the instructions may be distributed via a network or via other computer-readable storage media. Therefore, machine-readable storage media may include any mechanism for storing or transmitting information in a machine-readable (e.g., computer-readable) form, including but not limited to floppy disks, optical disks, optical discs, magneto-optical disks, read-only memory (ROM), random access memory (RAM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), magnetic cards or optical cards, application-specific integrated circuits (ASICs), flash memory, or tangible machine-readable storage for transmitting information (e.g., carrier waves, infrared signals, digital signals, etc.) using the Internet in the form of electrical, optical, acoustic, or other forms of propagated signals. Therefore, machine-readable storage media includes any type of machine-readable storage medium suitable for storing or transmitting electronic instructions or information in a machine-readable form (e.g., a computer). Furthermore, the computer mentioned in the specification may include a single processor or may be an architecture employing multiple processors for increased computing power.

[0413] In this specification, the reference to "an embodiment" or "an embodiment" means that a specific feature, structure, or characteristic described in connection with the embodiment is included in at least one exemplary implementation or technology disclosed according to an embodiment of this application. The appearance of the phrase "in an embodiment" in various places in the specification does not necessarily refer to the same embodiment.

[0414] Furthermore, the language used in this specification has been primarily chosen for readability and instructional purposes and may not have been chosen to depict or limit the disclosed subject matter. Therefore, the embodiments disclosed herein are intended to illustrate, and not limit, the scope of the concepts discussed herein.

Claims

1. A method for displaying a theme, characterized in that, Applied to electronic devices, the method includes: Display the first screen; Determine the first gaze area of ​​the user on the first interface, wherein a first theme element is displayed within the first gaze area; Based on the first data, a second theme element is displayed in the first gaze area, or a dynamic effect of switching between the first theme element and the second theme element is displayed in the first gaze area, wherein the first data is used to represent the user's emotions, the second theme element is determined based on the first data, and the second theme element is different from the first theme element.

2. The method according to claim 1, characterized in that, The step of displaying a second subject element in the first gaze area based on the first data, or displaying a dynamic effect of switching between the first subject element and the second subject element in the first gaze area, includes: Based on the first data, obtain the second topic element that matches the first data; Display a second subject element in the first gaze area, or display a dynamic effect in the first gaze area that switches between the first subject element and the second subject element.

3. The method according to claim 1 or 2, characterized in that, The first gaze region is determined based on second data, which includes data on the user's gaze at the first interface and the duration of the user's gaze at the first gaze region. The step of displaying a second subject element in the first gaze area, or displaying a dynamic effect in the first gaze area that switches between the first subject element and the second subject element, includes: Based on the user's sustained gaze duration being greater than or equal to a first threshold, the second topic element is displayed in the first gaze area; or... Based on the fact that the duration of the user's gaze on the first gaze area is greater than 0 and less than the first threshold, a dynamic effect of switching between the first theme element and the second theme element is displayed in the first gaze area.

4. The method according to claim 3, characterized in that, The second data includes the distance at which the user gazes at the first gaze region. The step of displaying a second subject element in the first gaze area, or displaying a dynamic effect in the first gaze area that switches between the first subject element and the second subject element, includes: Based on the fact that the distance between the user's gaze and the first gaze region is less than a second threshold, the second topic element is displayed in the first gaze region; or... Based on the fact that the distance between the user's gaze and the first gaze area is greater than or equal to the second threshold, a dynamic effect of switching between the first theme element and the second theme element is displayed in the first gaze area.

5. The method according to claim 4, characterized in that, The distance at which the user gazes at the first gaze area falls within the first distance range.

6. The method according to any one of claims 3 to 5, characterized in that, The second data includes the area related to the first gaze region; The step of displaying a second subject element in the first gaze area, or displaying a dynamic effect in the first gaze area that switches between the first subject element and the second subject element, includes: Based on the fact that the area of ​​the first gaze region is less than a third threshold, the second subject element is displayed within the first gaze region; or... Based on the fact that the area of ​​the first gaze region is greater than or equal to the third threshold, a dynamic effect of switching between the first subject element and the second subject element is displayed within the first gaze region.

7. The method according to claim 6, characterized in that, The area of ​​the first gaze region is determined based on the saccade features corresponding to the second data, wherein the saccade features include one or more of the following: Scan rate, scan amplitude, or scan rate.

8. The method according to any one of claims 1 to 7, characterized in that, The step of displaying the second subject element in the first gaze area includes: If the first theme element is a wallpaper element and includes a face area, then the second theme element is displayed such that it covers a portion of the first theme element; If the first theme element is not a wallpaper element, then the first theme element is removed, and the second theme element is displayed in the first viewing area.

9. The method according to claim 8, characterized in that, Displaying the second theme element such that it overlaps a portion of the first theme element includes any of the following: If the size of the face region is within a first size range and the face orientation angle within the face region is within a first angle range, then the second theme element is displayed such that it covers the face region. If the size of the face region is greater than the upper limit of the first size range, the second theme element is displayed such that the second theme element covers the first region of the first theme element, wherein the distance between the center point of the first region and the center point of the face region is less than the fourth threshold. If the size of the face region is smaller than the lower limit of the first size range, or the face orientation angle is greater than the upper limit of the first angle range, then the second theme element will not be displayed.

10. The method according to any one of claims 3 to 7, characterized in that, Before determining the user's first gaze region on the first interface based on the second data, the method includes: Enable the first interactive function; Determining the user's first gaze region on the first interface based on the second data includes: When the first interactive function is enabled, the second data is acquired; The user's first gaze area on the first interface is determined based on the second data.

11. The method according to claim 10, characterized in that, Enabling the first interactive function includes: The second interface is displayed, wherein the second interface includes a switch control for turning the first interactive function on or off; In response to the user's operation of turning on the switch control, the first interactive function is enabled.

12. The method according to any one of claims 2 to 11, characterized in that, The step of obtaining the second topic element that matches the first data based on the first data includes: Based on the first data, determine the emotion type to which the user's current emotion belongs; Obtain the second theme element that matches the stated emotion type.

13. The method according to claim 12, characterized in that, Determining the emotion type of the user's current emotion based on the first data includes: Determine the user's current mood rating based on the first data; Based on the rating, determine the first emotion type to which the user's current emotion belongs; Obtain the second theme element that matches the first emotion type.

14. The method according to claim 13, characterized in that, The first theme element matches the second emotion type, which is different from the first emotion type.

15. The method according to any one of claims 1 to 14, characterized in that, Before displaying the second subject element in the first gaze area, or before displaying a dynamic effect of switching between the first subject element and the second subject element in the first gaze area, the method further includes: At the first moment, the user's first gaze behavior data on the first interface is acquired, and the position of the user's first gaze point on the first interface is determined based on the first gaze behavior data. At a second moment, the user's second gaze behavior data on the first interface is acquired, and the position of the user's second gaze point on the first interface is determined based on the second gaze behavior data, wherein the second moment is later than the first moment; Based on the first gaze point position and the second gaze point position, predict the position of the third gaze point that the user is about to gaze at; In response to the third gaze point being located within the first gaze region, the second subject element is preloaded.

16. The method according to any one of claims 1 to 15, characterized in that, The first gaze region is determined based on the second data. After displaying the first interface, the method further includes: Based on the second data, a second gaze region of the user on the first interface is determined, and the second gaze region is a different region from the first gaze region. A third subject element is displayed in the first gaze area, and a fourth subject element is displayed in the second gaze area, based on one or more of the following: The duration for which the user gazes at the first gaze region is greater than the duration for which the user gazes at the second gaze region; or, The distance at which the user gazes at the first gaze region is less than the distance at which the user gazes at the second gaze region; or, The area of ​​the first gaze region is smaller than the area of ​​the second gaze region; The rendering complexity of the third theme element is higher than that of the fourth theme element.

17. An electronic device, characterized in that, include: One or more processors; One or more memories; the one or more memories storing one or more programs that, when executed by the one or more processors, cause the electronic device to perform the subject display method according to any one of claims 1 to 16.

18. A computer-readable storage medium, characterized in that, The readable storage medium stores instructions that, when executed on a computer, cause the computer to perform the topic display method according to any one of claims 1 to 16.

19. A computer program product, characterized in that, It includes a computer program / instruction that, when executed by a processor, implements the topic display method according to any one of claims 1 to 16.

Images