Naked eye 3D theme display method, electronic equipment, storage medium and chip

Abstract

The embodiment of the invention provides a naked eye 3D theme display method, electronic equipment, a storage medium and a chip, and relates to the technical field of terminals. The method is applied to terminal equipment and comprises the steps that N interactive elements in a naked eye 3D theme are displayed on a first interface, and N is an integer larger than or equal to 1; in response to the control operation, displaying a naked-eye 3D dynamic effect corresponding to the at least one interactive element on the first interface; wherein the naked eye 3D dynamic effect is generated according to the control operation. Through the technical scheme provided by the embodiment of the invention, the terminal equipment not only can display various elements in the theme in a more three-dimensional and vivid 3D visual effect, but also can control the elements to respond to the control operation of the user and interact with the user, and has relatively strong interestingness and relatively good user experience.

Description

[0001] This application is a divisional application. The application number of the original application is 202411335271.2, the original application date is September 23, 2024, and the name of the invention is "Display method, electronic device, storage medium and chip for naked-eye 3D themes". The entire contents of the original application are incorporated into this application by reference. Technical Field

[0002] The present application relates to the field of terminal technology, and in particular to a display method, electronic device, storage medium and chip for naked-eye three-dimensional (3D) themes. Background Art

[0003] At present, terminal devices provide a variety of themes, such as cartoon themes and landscape themes, and some themes also have animation elements. By setting a theme for the terminal device, the user can change the display style of its operation interface, such as changing the display style of desktop wallpaper, lock screen wallpaper, icons, fonts and other elements, to achieve personalized settings for the terminal device. However, the current theme elements are not visually three-dimensional and vivid, and the animation elements included are usually relatively fixed, less interesting, and the user experience is not good. Summary of the invention

[0004] The present application provides a naked-eye 3D theme display method, electronic device, storage medium and chip, which are used to solve the problem in the prior art that theme elements are not three-dimensional and vivid enough and have poor interest.

[0005] In a first aspect, an embodiment of the present application provides a method for displaying a naked-eye 3D theme, which is applied to a terminal device, comprising: displaying N interactive elements in the naked-eye 3D theme on a first interface, where N ≥ 1 and is an integer; in response to a control operation, displaying a naked-eye 3D animation corresponding to at least one interactive element on the first interface; wherein the naked-eye 3D animation is generated according to the control operation.

[0006] For example, the first interface includes the desktop, lock screen interface or screen-off interface of the terminal device.

[0007] Through the method provided in the embodiment of the present application, the terminal device can not only display the elements in the theme with more three-dimensional and vivid 3D visual effects, but also control these elements to interact with the user, which has strong fun and good user experience.

[0008] In some embodiments, in response to the control operation, displaying the naked eye 3D animation corresponding to the at least one interactive element on the first interface includes: in response to the first air gesture at the first position, selecting the first interactive element from N interactive elements and displaying the first naked eye 3D animation of the first interactive element; in response to the first air gesture at the second position, deselecting the first interactive element from the N interactive elements, selecting the second interactive element, and displaying the naked eye 3D animation of the second interactive element. The first position is different from the second position, and the first interactive element is different from the second interactive element.

[0009] Through the method provided in the embodiment of the present application, the terminal device can select the corresponding interactive element according to the position of the user's air gesture, and control the interactive element to interact with the user, thereby improving the user experience.

[0010] In some embodiments, in response to a control operation, a naked-eye 3D motion effect corresponding to an interactive element is displayed on a first interface, including: in response to a first air gesture, a first interactive element is randomly selected from N interactive elements, and a first naked-eye 3D motion effect of the first interactive element is displayed.

[0011] In some embodiments, the second air gesture includes a gesture of changing the distance from the first interactive element, and the second naked-eye 3D animation includes an animation related to the physical size of the first interactive element or an animation related to the rotation speed of the first element.

[0012] For example, as the second air gesture gradually approaches the first interactive element, the physical size of the first interactive element gradually increases; as the second air gesture gradually moves away from the first interactive element, the physical size of the first interactive element gradually decreases. Alternatively, as the second air gesture gradually approaches the first interactive element, the rotation speed of the first interactive element gradually increases; as the second air gesture gradually moves away from the first interactive element, the rotation speed of the first interactive element gradually decreases.

[0013] Through the method provided in the embodiment of the present application, after selecting an interactive element, the terminal device can control the size of the interactive element by the distance of the gesture, and the control process is quite interesting.

[0014] In some embodiments, when the first interactive element is selected, the method further includes: in response to the second air gesture, displaying a second naked-eye 3D animation effect of the first interactive element on the first interface. That is, after selecting an interactive element, the terminal device can further control the interactive element and interact with the user.

[0015] In some embodiments, the first air gesture is the same as the second air gesture, that is, the user can control the interactive element to display different naked-eye 3D motion effects through the same air control gesture.

[0016] In some embodiments, in response to a second air gesture, a second naked-eye 3D motion effect of the first interactive element is displayed on the first interface, including: determining the gesture strength of the second air gesture; and displaying the second naked-eye 3D motion effect of the first interactive element on the first interface according to the gesture strength.

[0017] Through the method provided in the embodiment of the present application, the user and the interactive element can interact more closely with the real scene, with a better user experience. Taking the first interactive element as an example, the stronger the gesture of the user when slapping the basketball, the farther the basketball moves. Or, taking the first interactive element as an example, the stronger the gesture of the user when slapping the Emoji expression, the angrier the Emoji expression, etc.

[0018] In some embodiments, the method also includes: in the second naked-eye 3D motion effect, if there is interaction between the first interactive element and the third interactive element, then displaying a third naked-eye 3D motion effect corresponding to the third interactive element, and the first naked-eye 3D motion effect is different from the third naked-eye 3D motion effect.

[0019] Through the method provided in the embodiment of the present application, the associated interactive elements can be linked. For example, during the shooting process, when the dynamic effect of the basketball is displayed, if the basketball is thrown into the basket, the dynamic effect corresponding to the net is displayed synchronously.

[0020] In some embodiments, the method also includes: in response to the first air gesture, displaying a fourth naked-eye 3D motion effect of other interactive elements except the first interactive element on the first interface, the first naked-eye 3D motion effect is different from the fourth naked-eye 3D motion effect; wherein, among the other interactive elements, the naked-eye 3D motion effects of at least two interactive elements are the same.

[0021] Through the method provided in the embodiment of the present application, when the user is controlling the interactive elements, the unselected interactive elements can also respond to the user's control behavior and interact with the user, thereby providing a better user experience.

[0022] In some embodiments, in response to the control operation, displaying the naked-eye 3D motion effect corresponding to the at least one interactive element on the first interface includes: in response to an air gesture moving toward a first direction, controlling the interactive element to rotate toward the first direction; in response to an air gesture moving toward a second direction, controlling the interactive element to rotate toward the second direction; wherein the first direction and the second direction are different. That is, the user controls the rotation direction of the interactive element by the movement direction of the gesture.

[0023] In some embodiments, in response to the control operation, the naked-eye 3D animation corresponding to the interactive element is displayed on the first interface, including: in response to the user looking at the display screen of the terminal device from a first perspective, the interactive element in the first interface rotates according to the first perspective; and in response to the user looking at the display screen of the terminal device from a second perspective, the interactive element in the first interface rotates according to the second perspective; wherein the first perspective is different from the second perspective. That is, the user controls the rotation of the interactive element by rotating his own perspective, keeping the interactive element always facing the user, and the control process is more interesting.

[0024] For example, in response to a user looking at the display screen of a terminal device at a first perspective, the eyes of the interactive element look at the user along the first perspective; and, in response to the user looking at the display screen of the terminal device at a second perspective, the eyes of the naked-eye 3D animation effect look at the user along the second perspective; wherein the first perspective is different from the second perspective.

[0025] In some embodiments, in response to a control operation, displaying a naked-eye 3D motion effect corresponding to an interactive element on a first interface includes: in response to the terminal device tilting in a first direction under the control operation, controlling the interactive element to move in a first direction on the first interface. The posture of the terminal device tilting in the first direction is determined by data collected by a gyroscope sensor. That is, the terminal device can simulate a scene in which an interactive element rolls on a desktop. As the desktop tilts downward, the interactive element rolls in the tilted direction, and the control process is more interesting.

[0026] In some embodiments, in response to a control operation, a naked-eye 3D motion effect corresponding to the interactive element is displayed on the first interface, including: displaying the naked-eye 3D motion effect corresponding to the interactive element according to the shaking force of the terminal device; wherein the shaking force is determined according to the acceleration data collected by the acceleration sensor, and there is a mapping relationship between the shaking force and the naked-eye 3D motion effect.

[0027] In some embodiments, the interactive element in the naked eye 3D theme is an expression element, and the corresponding naked eye 3D dynamic effect of the interactive element is displayed according to the shaking strength of the terminal device, including: displaying the naked eye 3D dynamic effect of the expression element corresponding to the emotion according to the shaking strength; wherein there is a preset mapping relationship between the shaking strength and the emotion. For example, as the shaking strength gradually increases, the emotion of the Emoji expression gradually progresses, such as gradually becoming angry.

[0028] In some embodiments, in response to the control operation, displaying the naked-eye 3D animation corresponding to the interactive element on the first interface includes: receiving user data collected by the second electronic device; and displaying the naked-eye 3D animation corresponding to the interactive element according to the user data. Exemplarily, the second electronic device can be a wearable device, such as a smart bracelet.

[0029] In some embodiments, the interactive elements in the naked-eye 3D theme are expression elements, and the user data includes the user's heart rate data; according to the user data, the corresponding naked-eye 3D animation of the interactive element is displayed, including: according to the user's heart rate data, the naked-eye 3D animation corresponding to the expression element is displayed; wherein the heart rate data is related to the emotion of the expression element. For example, the faster the heart rate, the more excited the user is, and the emotion of the expression element can be happy; the slower the heart rate, the more stable the user is, and the emotion of the expression element can be expressionless. Through the method provided in the embodiments of the present application, the terminal device can collect more abundant user data with the help of other electronic devices, so as to display expression animations that are more in line with the user's mood.

[0030] In some embodiments, in response to a control operation, a naked-eye 3D motion effect corresponding to an interactive element is displayed on a first interface, including: in response to a touch operation on a target interactive element, controlling the target element to display a first naked-eye 3D motion effect, and controlling other interactive elements except the target element to display a second naked-eye 3D motion effect, the first naked-eye 3D motion effect is different from the second naked-eye 3D motion effect.

[0031] That is, when the user is controlling the interactive elements, the interactive elements that are not selected can also respond to the user's control behavior and interact with the user, which provides a good user experience.

[0032] In some embodiments, the terminal device is a folding device, and the method further includes: in response to the operation of unfolding the folding device, switching from the first screen to the second screen to display the naked-eye 3D animation of the interactive element. The first screen is the display screen used by the folding device in the folded state, and the second screen is the display screen used by the folding device in the unfolded state. The method provided in this embodiment can control the interactive element to interact with the user during the process of unfolding the folding device, making the unfolding process of the folding device more interesting.

[0033] In a second aspect, an embodiment of the present application provides a display device, which includes a first display module and a second display module.

[0034] The first display module is used to display N interactive elements in the naked-eye 3D theme on a first interface, where N≥1 and is an integer.

[0035] The second display module is used to display a naked-eye 3D motion effect corresponding to at least one interactive element on the first interface in response to a control operation; wherein the naked-eye 3D motion effect is generated according to the control operation.

[0036] In a third aspect, an embodiment of the present application provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and when the processor executes the computer program, it implements the method shown in the first aspect above.

[0037] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the method shown in the first aspect above.

[0038] In a fifth aspect, an embodiment of the present application provides a chip, which includes a processor and a memory, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the method shown in the first aspect above is implemented.

[0039] In a sixth aspect, an embodiment of the present application provides a computer program product, which includes a computer program. When the computer program is executed by an electronic device, the electronic device implements the method shown in the first aspect above.

[0040] It can be understood that the beneficial effects of the second to sixth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 is a schematic diagram of a two-dimensional theme provided in an embodiment of the present application;

[0042] Figure 2 It is a schematic diagram of the hardware structure of a terminal device to which the method for displaying a naked-eye 3D theme provided in an embodiment of the present application is applicable;

[0043] Figure 3 is a schematic diagram of a software architecture involved in a method for displaying a naked-eye 3D theme provided in an embodiment of the present application;

[0044] Figure 4 It is a schematic diagram of the process of setting the theme provided in the embodiment of the present application;

[0045] Figure 5 is a flow chart of a method for displaying naked-eye 3D themes provided by one embodiment of the application;

[0046] Figure 6 It is a schematic diagram of displaying Emoji expressions when a terminal device is shaken, provided by an embodiment of the application;

[0047] Figure 7 Schematic diagram of Emoji expressions with different emotions provided in the embodiments of the present application;

[0048] Figure 8 is a schematic diagram of displaying an Emoji expression when a terminal device is shaken, provided in another embodiment of the application;

[0049] Fig. 9 is a flow chart of a method for displaying naked-eye 3D themes provided by an embodiment of the present application;

[0050] Fig.10 is a schematic diagram of a sensor coordinate system provided by an embodiment of the present application;

[0051] Fig.11 This is a schematic diagram of a user controlling an Emoji expression through air gestures provided by an embodiment of the present application;

[0052] Fig.12 This is a schematic diagram of display transformation of a screen-off interface provided by an embodiment of the present application;

[0053] Fig.13 is a schematic diagram of a user controlling a basketball shot through an air gesture provided by an embodiment of the present application;

[0054] Fig.14 This is a schematic diagram of the dynamic effect of a basketball net during a shooting process provided by an embodiment of the present application;

[0055] Fig.15 is a schematic diagram of a user controlling an Emoji expression through an air gesture provided by another embodiment of the present application;

[0056] Fig.16 is a flow chart of a method for displaying a naked-eye 3D theme in a shaking scene provided by an embodiment of the present application;

[0057] Fig.17 is a schematic diagram of a lock screen interface provided by an embodiment of the present application;

[0058] Fig.18 This is a schematic diagram of a display interface of a naked-eye 3D theme in a line-of-sight tracking scenario provided by an embodiment of the present application;

[0059] Fig.19 is a schematic flow chart of displaying a naked-eye 3D theme in a gaze tracking scenario provided by an embodiment of the present application;

[0060] Fig. 20 This is a schematic diagram of a display interface of a naked-eye 3D theme in a touch scenario provided by an embodiment of the present application;

[0061] Figure 21-22 It is a schematic diagram of display transformation of the screen-off interface provided in different embodiments of the present application;

[0062] Fig.23 This is a schematic diagram of the dynamic effect of a foldable mobile phone during the unfolding process provided by an embodiment of the present application;

[0063] Fig.24 This is a schematic diagram of the dynamic effect of a foldable mobile phone during the unfolding process provided by an embodiment of the present application;

[0064] Fig.25is a schematic diagram of an information sharing device provided by an embodiment of the present application;

[0065] Fig.26 It is a schematic diagram of the structure of a chip provided by an embodiment of the present application. DETAILED DESCRIPTION

[0066] It should be understood that in the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in this article is merely a way to describe the association relationship of associated objects, indicating that three relationships may exist, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone.

[0067] In this embodiment, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of this embodiment, unless otherwise specified, "plurality" means two or more.

[0068] Before introducing the technical solutions provided by the embodiments of the present application, the application scenarios involved in the present application are first introduced.

[0069] In order to beautify the interface display effect of the terminal device, the terminal device provides a variety of themes, such as cartoon themes, landscape themes, mechanical themes, text themes, etc. By setting the theme for the terminal device, the user can change the display style of the operation interface in the terminal device, for example, the display style of elements such as the desktop wallpaper, lock screen wallpaper, off-screen wallpaper, icons, fonts, etc., and the sound effects and even the playing effect of ringtones can be changed to realize personalized settings of the terminal device.

[0070] As terminal devices gradually develop towards intelligence and user needs increase, the themes of terminal devices are also developing and changing step by step. For example, themes are gradually changing from static to dynamic. For another example, some interesting animation plug-ins are gradually appearing in themes, making the themes more vivid and flexible. For another example, themes are gradually supporting users to edit or customize.

[0071] However, current themes are generally 2-dimensional (2D) visuals (see Figure 1), not three-dimensional and vivid enough. In addition, the animation content in current themes is usually fixed. For example, the transition animation of the terminal device from off screen to lock screen and then to desktop, or the animation displayed by the terminal device in response to the user's click operation, the animation content is preset and fixed. In short, the current themes not only have poor visual effects, but also cannot convert user behavior into animation feedback on the display screen, have poor interactivity with users, are not interesting, and have a poor user experience.

[0072] To this end, an embodiment of the present application provides a method for displaying a naked-eye 3D theme. Through this method, a terminal device can not only display a naked-eye 3D visual effect theme, but also control each naked-eye 3D element in the theme to respond to user behavior, thereby providing the user with an entertaining, interactive and controllable theme.

[0073] The method for displaying naked-eye 3D themes provided in the embodiments of the present application is described in detail below with reference to the accompanying drawings.

[0074] For the convenience of description, the basic concepts and terms involved in the embodiments of the present application are first introduced. It should be noted that the following introduction is intended to explain and illustrate, rather than to limit.

[0075] (1) User Interface

[0076] The user interface (UI) is an interface for interaction and information exchange between applications (Apps) and operating systems and users, and is used to achieve the conversion between the internal form of information and the form acceptable to users. In other words, the user interface is the interface source code written in a specific computer language such as Java and extensible markup language (XML). The interface source code is parsed and rendered on the electronic device and finally presented as content that the user can recognize. In short, the existence of the user interface makes the interaction between users and electronic devices more convenient.

[0077] (2) Theme

[0078] A theme is a combination of visual and interactive elements customized for a terminal device, including but not limited to wallpapers, icons, lock screen styles, fonts, sound effects, etc. By setting a theme for a terminal device, a user can change the display style of the terminal device's operating interface and personalize the terminal device.

[0079] (3) Glasses-free 3D technology

[0080] Naked-eye 3D technology refers to a display technology that allows users to directly perceive two-dimensional computer images as three-dimensional images without the need for external auxiliary equipment such as special glasses. This technology mainly relies on visual illusions and uses specific display methods to make the human brain produce a sense of three-dimensionality. For example, the terminal device can control the color grayscale of each element in the two-dimensional image to create a visual illusion, thereby achieving naked-eye 3D display.

[0081] (4) Screen status

[0082] In this embodiment, the screen status of the terminal device includes but is not limited to a screen-on status, a screen-off status, a screen-off status (always on display, AOD) and a screen-lock status, etc. The details are as follows.

[0083] The screen-on state refers to the state in which the screen of the terminal device is fully lit and displays various user interfaces. For example, the user interface may be a desktop, an application interface, a lock screen interface, and the like.

[0084] The screen-off state refers to the state where the terminal device completely turns off the screen and the phone does not display any information. In the screen-off state, although the terminal device interface does not display content, the APP can continue to run in the background.

[0085] The lock screen state refers to the state in which the terminal device locks the screen, making it impossible for the user to operate most functions. For example, the user cannot control the terminal device to display the desktop, cannot use desktop applications, and cannot set up the terminal device. However, the terminal device still retains some functions available, such as unlocking function, emergency call function, camera function, etc. The lock screen state can prevent users from accidentally touching the screen and prevent others from using the terminal device at will, ensuring the safety of the terminal device. When the user does not operate the terminal device for a long time, or the user presses the power button on the terminal device, or the user touches the one-key lock screen icon in the display interface, the terminal device will enter the lock screen state. When the user wants to use the terminal device again, he can unlock the terminal device by fingerprint unlocking, gesture unlocking, or face unlocking.

[0086] The screen-off state usually refers to the state when the terminal device is in the locked screen state, without lighting up the entire screen, but displaying time, phone calls, messages, battery information, push messages, themes, animations, wallpapers and other content in a partial area of ​​the screen. In the screen-off state, users can intuitively see the required information, which can reduce the steps for users to operate the terminal device and bring convenience to users. It should be noted that in the screen-off state, the terminal device's APP can continue to run in the background.

[0087] In some embodiments, in the screen-off state, the terminal device can display the content to be presented to the user in full screen, for example, displaying the naked-eye 3D animation in the theme in full screen.

[0088] (5) Lock screen and off screen

[0089] The lock screen interface refers to the interface in which the entire screen is lit up to display information such as time, date, and wallpaper when the terminal device is in lock screen mode.

[0090] The off-screen interface, also known as the AOD interface, refers to the interface displayed when the terminal device is in the off-screen state. Depending on the off-screen display content set by the terminal device lock, the off-screen interface may be partially lit or fully lit.

[0091] After introducing the basic concepts and terms involved in this application, the hardware structure and related software architecture of the terminal device involved in the embodiments of this application are introduced next.

[0092] In the embodiments of the present application, the terminal device may also be referred to as a terminal, a user equipment (UE), a mobile station (MS), a mobile terminal (MT), etc. The terminal device may be a mobile phone, a smart TV, a wearable device, a tablet computer (Pad), a computer with wireless transceiver function, a virtual reality (VR) terminal device, an augmented reality (AR) terminal device, a wireless terminal in industrial control, a wireless terminal in self-driving, a wireless terminal in remote medical surgery, a wireless terminal in smart grid, a wireless terminal in transportation safety, a wireless terminal in smart city, a wireless terminal in smart home, and a vehicle-mounted device, etc. The embodiments of the present application do not limit the specific technology and specific device form adopted by the terminal device.

[0093] Figure 21 is a schematic diagram of the hardware structure of a terminal device applicable to the display method of naked-eye 3D themes provided in an embodiment of the present application. The terminal device may include a processor 210, an external memory interface 220, an internal memory 221, a universal serial bus (USB) interface 230, a charging management module 240, a power management module 241, a battery 242, an antenna 1, an antenna 2, a mobile communication module 250, a wireless communication module 260, an audio module 270, a speaker 270A, a receiver 270B, a microphone 270C, an earphone interface 270D, a sensor module 280, a button 290, a motor 291, an indicator 292, a camera 293, a display screen 294, and a subscriber identification module (SIM) card interface 295, etc.

[0094] It is to be understood that the structure illustrated in the embodiments of the present application does not constitute a specific limitation on the terminal device. In other embodiments of the present application, the terminal device may include more or fewer components than shown in the figure, or combine certain components, or split certain components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0095] For example, when the terminal device is a mobile phone or a tablet computer, it may include all the components shown in the figure, or may include only some of the components shown in the figure.

[0096] The processor 210 may include one or more processing units, for example, the processor 210 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0097] The USB interface 230 is an interface that complies with USB standard specifications, and specifically can be a Mini USB interface, a Micro USB interface, a USB Type C interface, etc. The USB interface 230 can be used to connect a charger to charge the terminal device, and can also be used to transfer data between the terminal device and peripheral devices. It can also be used to connect headphones to play audio through the headphones.

[0098] It is understandable that the interface connection relationship between the modules illustrated in the embodiment of the present application is only a schematic illustration and does not constitute a structural limitation on the terminal device. In other embodiments of the present application, the terminal device may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0099] The charging management module 240 is used to receive charging input from a charger. The charger may be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 240 may receive charging input from a wired charger through the USB interface 230. In some wireless charging embodiments, the charging management module 240 may receive wireless charging input through a wireless charging coil of a terminal device. While the charging management module 240 is charging the battery 242, it may also power the electronic device through the power management module 241.

[0100] The power management module 241 is used to connect the battery 242, the charging management module 240 and the processor 210. The power management module 241 receives input from the battery 242 and / or the charging management module 240, and supplies power to the processor 210, the internal memory 221, the external memory, the display screen 294, the camera 293, and the wireless communication module 260. The power management module 241 can also be used to monitor parameters such as battery capacity, battery cycle number, and battery health status (leakage, impedance).

[0101] The wireless communication function of the terminal device can be realized through antenna 1, antenna 2, mobile communication module 250, wireless communication module 260, modem processor and baseband processor.

[0102] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals.

[0103] The mobile communication module 250 can provide wireless communication solutions including 2G / 3G / 4G / 5G etc. applied on the terminal equipment.

[0104] The wireless communication module 260 can provide a wireless communication solution applied to the terminal device.

[0105] In some embodiments, antenna 1 of the terminal device is coupled to mobile communication module 250, and antenna 2 is coupled to wireless communication module 260, so that the terminal device can communicate with the network and other devices through wireless communication technology.

[0106] The terminal device implements the display function through a GPU, a display screen 294, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 294 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 210 may include one or more GPUs, which execute program instructions to generate or change display information.

[0107] The display screen 294 is used to display images, videos, etc. For example, the lock screen interface, AOD interface, desktop, etc. in the embodiment of the present application.

[0108] The terminal device can realize the shooting function through the ISP, the camera 293, the video codec, the GPU, the display screen 294 and the application processor.

[0109] The ISP is used to process the data fed back by the camera 293. For example, when taking a photo, the shutter is opened, and the light is transmitted to the camera photosensitive element through the lens. The light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to the ISP for processing and converts it into an image visible to the naked eye. The ISP can also perform algorithm optimization on the noise, brightness, and skin color of the image. The ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, the ISP can be set in the camera 293.

[0110] The camera 293 is used to capture static images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element converts the optical signal into an electrical signal, which is then transmitted to the ISP for conversion into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the terminal device may include 1 or N cameras 293, where N is a positive integer greater than 1.

[0111] The digital signal processor is used to process digital signals. In addition to processing digital image signals, it can also process other digital signals. For example, when the terminal device is selecting a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

[0112] Video codecs are used to compress or decompress digital videos. Terminal devices can support one or more video codecs. In this way, terminal devices can play or record videos in multiple coding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0113] NPU is a neural network (NN) computing processor. It can quickly process input information and continuously self-learn by drawing on the structure of biological neural networks, such as the transmission mode between neurons in the human brain. NPU can realize applications such as intelligent cognition of terminal devices, such as image recognition, face recognition, voice recognition, text understanding, etc.

[0114] In an embodiment of the present application, the NPU or other processors may perform operations such as analyzing and processing images in the video stored in the terminal device.

[0115] The external memory interface 220 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the terminal device. The external memory card communicates with the processor 210 through the external memory interface 220 to implement a data storage function. For example, files such as music and videos can be stored in the external memory card.

[0116] The internal memory 221 can be used to store computer executable program codes, and the executable program codes include instructions. The processor 210 executes various functional applications and data processing of the terminal device by running the instructions stored in the internal memory 221. The internal memory 221 may include a program storage area and a data storage area. Among them, the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.). The data storage area can store data created during the use of the terminal device (such as audio data, a phone book, etc.).

[0117] In addition, the internal memory 221 may include a high-speed random access memory and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0118] The terminal device can implement audio functions through the audio module 270, the speaker 270A, the receiver 270B, the microphone 270C, the headphone interface 270D, and the application processor.

[0119] The audio module 270 is used to convert digital audio signals into analog audio signals for output, and is also used to convert analog audio inputs into digital audio signals. The audio module 270 can also be used to encode and decode audio signals. In some embodiments, the audio module 270 can be arranged in the processor 210, or some functional modules of the audio module 270 can be arranged in the processor 210.

[0120] The speaker 270A, also called a "speaker", is used to convert audio electrical signals into sound signals. The terminal device can listen to music or listen to hands-free calls through the speaker 270A. For example, the speaker can play the comparison analysis results provided in the embodiment of the present application.

[0121] The receiver 270B, also called a "handset", is used to convert audio electrical signals into sound signals. When the terminal device receives a call or voice message, the voice can be heard by placing the receiver 270B close to the ear.

[0122] Microphone 270C, also called "microphone" or "microphone", is used to convert sound signals into electrical signals.

[0123] The earphone jack 270D is used to connect a wired earphone.

[0124] The sensor module 280 may include a pressure sensor 280A, a gyroscope sensor 280B, an air pressure sensor 280C, a magnetic sensor 280D, an acceleration sensor 280E, a distance sensor 280F, a proximity light sensor 280G, a fingerprint sensor 280H, a temperature sensor 280J, a touch sensor 280K, an ambient light sensor 280L, a bone conduction sensor 280M, etc.

[0125] The pressure sensor 280A is used to sense the pressure signal and can convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor 280A can be disposed on the display screen 294 .

[0126] The gyroscope sensor 280B can be used to determine the motion posture of the terminal device. In some embodiments, the angular velocity of the terminal device around three axes (i.e., x, y, and z axes) can be determined by the gyroscope sensor 280B. The gyroscope sensor 280B can be used for shooting anti-shake, navigation, and somatosensory game scenes.

[0127] The air pressure sensor 280C is used to measure air pressure. In some embodiments, the terminal device calculates the altitude through the air pressure value measured by the air pressure sensor 280C to assist positioning and navigation.

[0128] The magnetic sensor 280D includes a Hall sensor, and the terminal device can use the magnetic sensor 280D to detect the opening and closing of the flip leather case.

[0129] The acceleration sensor 280E can detect the magnitude of the acceleration of the terminal device in all directions (generally three axes). When the terminal device is stationary, it can detect the magnitude and direction of gravity. It can also be used to identify the posture of the electronic device and is applied to applications such as horizontal and vertical screen switching and pedometers.

[0130] The distance sensor 280F is used to measure the distance. The terminal device can measure the distance by infrared or laser. In some embodiments, when shooting a scene, the terminal device can use the distance sensor 280F to measure the distance to achieve fast focusing.

[0131] The proximity light sensor 280G may include, for example, a light emitting diode (LED) and a light detector, such as a photodiode. The terminal device may use the proximity light sensor 280G to detect when a user holds the terminal device close to their ear to make a call, so as to automatically turn off the screen to save power. The proximity light sensor 280G may also be used to automatically unlock and lock the screen in the leather case mode and pocket mode.

[0132] The ambient light sensor 280L is used to sense the ambient light brightness. The terminal device can adaptively adjust the brightness of the display screen 294 according to the perceived ambient light brightness. The ambient light sensor 280L can also be used to automatically adjust the white balance when taking pictures. The ambient light sensor 280L can also cooperate with the proximity light sensor 280G to detect whether the terminal device is in a pocket to prevent accidental touch.

[0133] The fingerprint sensor 280H is used to collect fingerprints. The terminal device can use the collected fingerprint characteristics to achieve fingerprint unlocking, access application locks, fingerprint photography, fingerprint answering calls, etc.

[0134] The temperature sensor 280J is used to detect the temperature.

[0135] The touch sensor 280K is also called a "touch panel". The touch sensor 280K can be set on the display screen 294, and the touch sensor 280K and the display screen 294 form a touch screen, also called a "touch screen". The touch sensor 280K is used to detect touch operations acting on or near it. The touch sensor can pass the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 294. In other embodiments, the touch sensor 280K can also be set on the surface of the terminal device, which is different from the position of the display screen 294.

[0136] The bone conduction sensor 280M can obtain vibration signals. In some embodiments, the bone conduction sensor 280M can obtain vibration signals of vibrating bones of the human body. The bone conduction sensor 280M can also contact the human body's pulse to receive blood pressure beating signals.

[0137] The key 290 includes a power key, a volume key, etc. The key 290 may be a mechanical key or a touch key. The terminal device may receive key input and generate key signal input related to user settings and function control of the terminal device.

[0138] Motor 291 can generate vibration prompts. Motor 291 can be used for incoming call vibration prompts, and can also be used for touch vibration feedback. For example, touch operations acting on different applications (such as taking pictures, audio playback, etc.) can correspond to different vibration feedback effects. Motor 291 can also correspond to different vibration feedback effects for touch operations acting on different areas of the display screen 294.

[0139] Indicator 292 may be an indicator light, which may be used to indicate charging status, power changes, messages, missed calls, notifications, etc.

[0140] The SIM card interface 295 is used to connect a SIM card. The SIM card can be connected to or disconnected from the terminal device by inserting the SIM card interface 295 or pulling the SIM card interface 295 out. The terminal device can support 1 or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 295 can support Nano SIM card, Micro SIM card, SIM card, etc.

[0141] Figure 3 Schematic diagram of the software architecture involved in the method for displaying naked-eye 3D themes provided in the embodiment of the present application. Figure 3 As shown, the software architecture includes a theme package module 310 , a unified desktop module 320 and a theme engine 330 .

[0142] The theme package module 310 is used to store theme packages of various themes, including preset themes, personalized themes, and online themes. Preset themes refer to themes pre-configured in the operating system of the terminal device, and the user does not have the authority to add, delete, modify, and edit the preset themes. Personalized themes include various themes edited and set by users according to their personal preferences. Online themes include themes obtained by the terminal device from online theme applications.

[0143] The unified desktop module 320 can control and display elements such as the lock screen page, the Home page, and desktop components. In addition, the desktop module 320 also includes a theme editing unit, which is used to customize the theme obtained by the theme engine 330 from the theme package module 310 on the desktop according to user operations.

[0144] The theme engine 330 includes a domain specific language (DSL) parsing and execution module 331 , an atomic capability module 332 , an engine custom view component 333 , a parameter management module 334 , and a calculation and execution module 335 .

[0145] The DSL parsing and execution module 331 is used to parse the theme package, map the theme package into various atomic capabilities in the theme engine, such as variables, views, etc., and is responsible for the scheduling of various capabilities of the theme engine 330.

[0146] In some embodiments, the DSL parsing and execution module 331 includes a DSL script parsing unit, a DSL instruction registration unit, a DSL instruction cache, a scene management unit, a page drawing unit, and a motion effect and interaction control unit.

[0147] The DSL script parsing unit is used to find the theme script file from the theme directory according to the rules when applying the theme after the theme engine is started, and parse the theme script file into a data node tree.

[0148] The DSL instruction registration unit is used to register various atomic capabilities when the theme engine is started. Atomic capabilities can be understood as instructions for implementing specific functions.

[0149] The DSL instruction table is used for the DSL script parsing unit to parse the subject script into a data node tree, and then quickly match the corresponding atomic capabilities from the DSL instruction table according to the node type. For example, if the data node type is a custom variable, the corresponding atomic capability is: register the custom variable to the parameter management module; if the data node type is a naked eye 3D node, the corresponding atomic capability is: form a view tree of a naked eye 3D image based on relevant data.

[0150] The scene management unit is used to classify and manage the matched atomic capabilities according to scenes. For example, the scenes can be different theme scenes, such as free fall scenes, throwing scenes, gesture control scenes, and sight tracking scenes of theme elements.

[0151] The page drawing unit is used to draw the first frame after the theme is loaded successfully, and to refresh the corresponding visual content after the elements in the theme are refreshed.

[0152] The motion effect and interaction control unit is used to respond to external changes and display theme information. For example, for naked-eye 3D themes, it responds to external changes such as the user shaking the phone, moving the line of sight, or making gestures in the air, collects relevant data, converts the data into attribute values ​​of visual elements, such as the angle and position of the object, and draws it into a graph.

[0153] The atomic capability module 332 is a collection of all atomic capabilities supported by the theme engine 330, such as a collection of atomic capabilities such as variable nodes and text box nodes. The atomic capability module 332 includes atomic capabilities related to functions such as 2D basic view, 2D basic animation, 3D basic view, and personalized customized visual effects. Among them, personalized customized visual effects are atomic capabilities dedicated to naked eye 3D themes, which include material display, lighting display, etc.

[0154] The engine custom view component 333 is a component set formed by combining and encapsulating the underlying UI components to facilitate designers to create themes. For example, for the custom editing capability of the theme engine, various specific components are encapsulated, such as texture selection, color selection, and Emoji selection components used for custom editing of naked-eye 3D themes.

[0155] The parameter management module 334 is responsible for the management of parameter variables in the system. In this embodiment, parameter variables are divided into two categories, namely, custom variables and system variables. Custom variables refer to variables defined in the DSL script, which can be fixed values, expressions, or references to other variables. System variables refer to the only variables obtained from the system of the terminal device, such as the current power, user steps, system time, etc. The user interaction data (shaking, human eyes, gestures, etc.) required for the naked eye 3D theme are all system variables.

[0156] The calculation and execution module 335 is an auxiliary module of the motion effect and interaction control module unit. It supports a large number of complex calculation functions and the combination and nesting of these functions. It calculates a large number of complex expressions parsed from the DSL script with the acquired variables to obtain specific values, and passes them to the motion effect and interaction control module for drawing the positions of all visual elements.

[0157] After the terminal device successfully loads the theme from the theme package module 310, the theme engine 330 first parses the theme package. Subsequently, the theme engine 330 draws the first frame according to the script file of the target theme, and displays the drawn first frame interface through the unified desktop module 320. Subsequently, the theme engine 330 collects and processes the user interaction data, draws the animation in real time according to the processing results, and updates the display interface. The unified desktop module 320 displays the updated interface in real time to enable the elements in the theme to respond to the user's control operations.

[0158] Based on the hardware structure and software architecture of the terminal device provided in the above-mentioned embodiment, the display method of the naked-eye 3D theme provided in the embodiment of the present application is specifically described below.

[0159] First, the process of a user setting a naked-eye 3D theme for a terminal device is exemplarily introduced.

[0160] See also Figure 4 As shown in (a) in FIG. 1 , various applications are installed in the terminal device, such as clock, calendar, gallery, memo, file management, theme application, etc. In response to the user's selection operation on the theme application, the terminal device displays the theme application interface.

[0161] See also Figure 4 As shown in (b) of FIG. 1 , the theme application interface includes various 2D themes and naked-eye 3D themes for users to choose from. Among them, the naked-eye 3D theme usually includes one or more naked-eye 3D elements, such as background elements and interactive elements of naked-eye 3D visual effects. Among them, the interactive elements can respond to the user's behavior and interact with the user.

[0162] Exemplarily, the naked eye 3D theme includes the illustrated Emoji theme and sphere theme, etc. Among them, Emoji is an emoticon, composed of graphics and / or symbols, used to convey emotions, express ideas or increase the interest of the text in the text. The sphere can be a tennis ball, basketball, volleyball, football, billiards, etc., and the present embodiment does not limit the type of sphere. In addition, although not shown in the figure, the naked eye 3D theme can also be a person, an animal, a landscape, a cartoon character, a star, a coin, a vehicle, a building, etc., and the present embodiment of the application does not limit this.

[0163] In response to the user's selection of the target theme, the terminal device can apply the theme and display the corresponding theme elements on the desktop, lock screen interface, and off screen interface. For example, in response to the user's selection of the Emoji theme, the terminal device uses the Emoji theme and displays the corresponding theme elements on the desktop, lock screen interface, and off screen interface. Figure 4 The Emoji expression is displayed in the (c) of FIG. 4 and the off-screen interface (see (d) of FIG. 4 ). The Emoji expression is the interactive element in the theme.

[0164] Next, the process of displaying naked-eye 3D themes on a terminal device and interacting with a user is described in detail.

[0165] Figure 5 FIG. 1 is a flow chart of a method for displaying a naked-eye 3D theme provided by an embodiment of the application. Figure 5 As shown, the method includes the following steps S501-S502.

[0166] S501, the terminal device displays N interactive elements in the naked-eye 3D theme on the first interface, where N≥1 and is an integer.

[0167] Exemplarily, the first interface is a desktop, a lock screen interface, or a screen-off interface of the terminal device, which is not limited in this embodiment.

[0168] S502, the terminal device displays a naked-eye 3D animation corresponding to at least one interactive element on a first interface in response to a control operation; wherein the naked-eye 3D animation is generated according to the control operation.

[0169] Exemplarily, the control operation may be that the user shakes the terminal device, the user controls the terminal device through gestures, the user controls the terminal device through sight, the user controls the terminal device through touch operations, or the user unfolds the terminal device in a folded state. This embodiment does not specifically limit the type of control operation.

[0170] Through the method provided in the embodiment of the present application, the terminal device can not only display the elements in the theme with more three-dimensional and vivid 3D visual effects, but also control these elements to respond to user behavior and interact with the user, thereby providing a better user experience.

[0171] Depending on the naked eye 3D theme, the interactive elements in the theme usually interact with the user in different ways. According to the different interactive methods, the interactive scenes include multiple types, such as (i) shaking scenes, (ii) gesture control scenes, (iii) sight tracking scenes, (iv) touch operation scenes, and (v) folding device unfolding scenes. The following is a specific description of the method provided in the embodiment of the present application in combination with different interactive scenes.

[0172] (I) Shaking scene

[0173] In the shaking scene, the terminal device can respond to the user shaking the terminal device and control the interactive elements therein to display the corresponding interactive effects. For ease of understanding, the embodiment of the present application takes the naked eye 3D theme with circular interactive elements, such as Emoji theme, sphere theme, star theme, coin theme, etc., as an example to introduce the shaking scene. Of course, the shaking scene can also be applied to any interactive element, taking the circular interactive element as an example.

[0174] In this embodiment, the shaking scene includes the user tilting the terminal device and shaking the terminal device, as shown below.

[0175] (1) Tilt terminal equipment

[0176] When the user tilts the terminal device, the terminal device can simulate the force of each element, move in the direction of the terminal device tilt, display collision and interesting visual effects, and enhance the user's sense of use and experience. Exemplarily, the force on the interactive element includes gravity and support force. For example, in the process of simulating a ball rolling on the desktop, the force on the ball includes gravity and the support force of the desktop.

[0177] In some embodiments, in response to the terminal device tilting in the first direction, the terminal device controls the interactive element in the display interface to move in the first direction. The interactive element may be a circular or other shaped interactive element. The display interface may be a desktop, a lock screen interface, or an off screen interface. This embodiment does not limit this.

[0178] Example 1-1:

[0179] The terminal device currently uses the naked eye 3D visual effect Emoji theme, and the interactive elements in the Emoji theme are naked eye 3D visual effect Emoji expressions. When the user tilts the terminal device, the specific animation effects of each Emoji expression are as follows.

[0180] See also Figure 6 As shown in (a) to (b) in the figure, the lock screen interface of the terminal device displays a storage space (such as a box) with naked-eye 3D visual effects, and the bottom of the storage space displays a plurality of randomly arranged Emoji expressions with naked-eye 3D visual effects. In response to the user controlling the terminal device to tilt to the right, each Emoji expression in the storage space simulates the state of bearing gravity and / or supporting force, and scrolls to the right in the storage space. If the terminal device remains in a tilted posture to the right, these Emoji expressions will gather on the right side of the storage space (i.e., near the right side of the terminal device).

[0181] In addition, although not shown in the figure, when the user controls the terminal device to tilt to the left, the Emoji expressions in the accommodation space simulate the state of bearing gravity and support force and roll to the left in the accommodation space. If the terminal device remains tilted to the left, these Emoji expressions will gather on the left side of the accommodation space (i.e., near the right side of the terminal device).

[0182] Alternatively, when the user controls the top edge of the terminal device to tilt downward, the Emoji expressions in the accommodation space simulate the state of bearing gravity and support force and roll toward the top of the accommodation space. If the terminal device keeps the top tilted downward, these Emoji expressions will gather at the top of the accommodation space (i.e., near the top edge of the terminal device).

[0183] Example 1-2:

[0184] The naked-eye 3D theme currently used by the terminal device is a billiard theme, and the display interface of the terminal device (such as the off-screen interface or the lock screen interface) includes: a billiard table with naked-eye 3D visual effects, and billiard holes at the four corners and the middle of each side of the billiard table. In response to the user's operation of tilting the terminal device, each billiard ball rolls on the billiard table in the tilted direction. If its rolling path passes through the billiard hole, it will visually fall into the hole. For example, when the user tilts the terminal device to the left, a billiard ball rolls to the left, and its rolling path passes through the hole in the middle of the left side, then the billiard ball will roll into the hole.

[0185] (2) Shake the terminal device

[0186] Compared with tilting the terminal device, the user's movements frequency and amplitude are usually greater when shaking the terminal device. In other words, shaking the terminal device can be interpreted as tilting the terminal device more quickly and violently.

[0187] When the user shakes the terminal device, the terminal device can not only simulate the movement of each interactive element in the direction of the terminal device's tilt due to gravity and support force, but also determine the dynamic effect of each interactive element according to the strength of the user shaking the terminal device. There is a preset mapping relationship between the shaking strength and the dynamic effect of the interactive element.

[0188] Taking the Emoji theme as an example, the mapping relationship between the shaking intensity and the animation effect of the Emoji expression is as follows: Figure 7 As shown, as the user shakes the terminal device more and more forcefully, the emotion of the Emoji expression gradually becomes angry. Exemplarily, the shaking force can be divided into multiple intervals, and different intervals correspond to Emoji expressions with different emotions. For example, the shaking force is divided into ranges 1 to 6 from small to large, and ranges 1 to 6 correspond to Emoji expressions from "naughty", "questionable", "unhappy", "tired", "dizzy" to "angry" in turn.

[0189] Based on this, see Figure 8 As shown in (a) to (c) in the figure, a plurality of randomly arranged Emoji expressions are displayed at the bottom of the lock screen interface of the terminal device. In response to the user shaking the terminal device within the range 5, and the range 5 corresponds to the expression of "dizziness", the terminal device displays all the Emoji expressions on the lock screen interface as the Emoji expression of "dizziness".

[0190] Optionally, the theme can also be preset with Easter egg effects (such as Easter egg Emoji expressions), and the Easter egg effects cannot be selected and edited on the custom editing page, that is, the user cannot perceive the Easter egg effects on the custom editing page. The Easter egg effects allow the theme of the terminal device to have a lightweight hidden mechanism for users to explore, which can bring greater surprises to users. Exemplarily, when the terminal device detects that the shaking force is in a specific range, the terminal device displays the Easter egg effect. In the process of displaying the Easter egg effects, the terminal device can also be accompanied by ritual scenes such as vibration, sound effects, or fireworks celebrations to enhance the playability of the theme.

[0191] After introducing the user operation and interface display process in the terminal device shaking scenario, the control method for the terminal device to implement the above process is specifically described below.

[0192] Fig. 9 This is a flow chart of a method for displaying a naked-eye 3D theme provided by an embodiment of the present application. The method is applied to a terminal device provided with a naked-eye 3D theme, and specifically includes the following steps S901 to S904.

[0193] S901, the terminal device displays at least one interactive element of naked-eye 3D visual effects on a display interface.

[0194] Exemplarily, the interactive element may be a circular or other shaped interactive element. The display interface may be a desktop, a lock screen interface, or an off screen interface. This embodiment does not limit this.

[0195] S902, the terminal device detects its own posture information and acceleration information.

[0196] In some embodiments, the terminal device uses a gyroscope sensor 280B to detect its own posture information, and uses an acceleration sensor 280E to detect acceleration information of the terminal device.

[0197] The sensors of the terminal device measure the corresponding parameters based on the sensor coordinate system. The sensor coordinate system is also called the device coordinate system or the screen coordinate system. Fig.10 As shown, for most terminal devices, the origin of the sensor coordinate system is the center of the terminal device's screen, the X-axis is perpendicular to the left side of the terminal device and points to the right side, the Y-axis is perpendicular to the bottom side of the terminal device and points to the top side, and the Z-axis is perpendicular to the terminal device's screen and points upward. It should be noted that the sensor coordinate system is a three-axis coordinate system used by the terminal device sensor framework to display sensor data. The three axis directions exist objectively and are set based on the natural direction of the terminal device. They will not change due to changes in the device position or angle.

[0198] Based on this, the attitude information of the terminal device can be expressed by the direction angle α, the pitch angle β, and the roll angle γ, that is, (α, β, γ). Among them, the direction angle α, the pitch angle β, and the roll angle γ represent the rotation angles of the terminal device around the Z axis, X axis, and Y axis of the sensor coordinate system, respectively. The acceleration information of the terminal device detected by the acceleration sensor can be expressed by (x, y, z), where x, y, and z are the three-axis accelerations along the X axis, Y axis, and Z axis in the sensor coordinate system, respectively.

[0199] S903, the terminal device simulates the acceleration of the interactive element after being subjected to external force according to its own posture information and acceleration information.

[0200] In this embodiment, the terminal device obtains posture information from the gyroscope sensor 280B and acceleration information from the acceleration sensor 280E through the parameter management module 334 of the theme engine 330. Subsequently, the motion effect and interaction control unit in the DSL parsing and execution module 331 simulates the simulated acceleration of the interactive element after being subjected to external force based on the posture information and acceleration information.

[0201] In the shaking scenario, the external forces simulated by the terminal device for the interactive elements include gravity, support force, and the force applied to the interactive elements by the user when shaking the device. According to Newton's second law F=ma, when the external force on the interactive element is unbalanced, the interactive element will move according to the simulated acceleration a=F / m, where F is the resultant force on the interactive element and m is the simulated mass of the interactive element. Therefore, the parameter that affects the movement of the interactive element is actually the simulated acceleration.

[0202] In the embodiment of the present application, the terminal device can simulate and determine the simulated acceleration of the interactive element after being subjected to an external force based on the device posture and the acceleration information detected by the acceleration sensor. For example, the acceleration is superimposed on the component of the gravity acceleration along the tilt direction of the terminal device screen to obtain the simulated acceleration of the interactive element.

[0203] S904, the terminal device displays the naked-eye 3D motion effect of the interactive element according to the simulated acceleration.

[0204] In this embodiment, the DSL parsing and execution module 331 of the terminal device generates the dynamic effect of the interactive element according to the simulated acceleration, and the unified desktop module 320 displays the corresponding dynamic effect.

[0205] For example, if the sum of the accelerations of the interactive elements is along the positive direction of the X axis, then see Figure 6As shown in (b), the terminal device controls the interactive elements of the display interface to move along the positive direction of the X axis, that is, to the right side of the terminal device. If the sum of the accelerations of the interactive elements is in the opposite direction of the Y axis, then the terminal device controls the interactive elements of the display interface to move in the opposite direction of the Y axis, that is, to the bottom side of the terminal device. The terminal device can use the theme engine shown above to display the scene of the interactive element scrolling and collision according to the simulated acceleration, and display it in the interface.

[0206] Optionally, the terminal device can control the movement direction of the interactive element according to the magnitude of the simulated acceleration, and determine the shaking strength of the terminal device by simulating the frequency of changes in the acceleration direction, wherein the higher the frequency of changes in the acceleration direction, the higher the shaking frequency of the terminal device and the greater the shaking strength. Exemplarily, the terminal device can display different emotional expressions according to the shaking strength of the user.

[0207] It should be noted that the terminal device will cyclically execute the above steps S902 to S904 until the terminal device exits the current interface. This cyclic processing process can control the interactive elements to respond to the user's operation and improve the user experience.

[0208] (II) Gesture control scenarios

[0209] In the gesture control scenario, the terminal device can respond to the user's air gestures and control the interactive elements in the display interface to display corresponding interactive effects. For example, the target interactive element is selected according to the air gesture, or the target interactive element is controlled according to the air gesture.

[0210] In some embodiments, in response to a first air gesture of a user at a first position, the terminal device selects a first interactive element from multiple interactive elements of the current display interface and displays a first naked-eye 3D animation of the first interactive element; and in response to a first air gesture of a user at a second position, the terminal device cancels the selection of the first interactive element, selects a second interactive element from multiple interactive elements of the current display interface, and displays a naked-eye 3D animation of the second interactive element. The first position is different from the second position, and the first interactive element is also different from the second interactive element.

[0211] In other words, when the user inputs the first air gesture, the terminal device triggers the element selection scene. When the user keeps moving the first air gesture toward the screen, the terminal device switches the selected interactive element according to the position of the gesture movement.

[0212] In other embodiments, in response to the user's first air gesture, the terminal device randomly selects a first interactive element from the current display interface and displays a first naked-eye 3D animation of the first interactive element; in response to the user's second air gesture, the terminal device displays a second naked-eye 3D animation of the first interactive element.

[0213] In some other embodiments, during the movement of the first interactive element, if there is interaction between the first interactive element and the third interactive element, then during the movement of the first interactive element, a third naked-eye 3D motion effect of the third interactive element is displayed.

[0214] In some other embodiments, after selecting the first interactive element, the terminal device may further display a fourth naked-eye 3D animation effect of other interactive elements except the first interactive element.

[0215] In some other embodiments, the terminal device may display the second naked-eye 3D animation effect of the first interactive element according to the gesture strength of the second air gesture, wherein there is a preset mapping relationship between the gesture strength and the second naked-eye 3D animation effect.

[0216] Example 2-1:

[0217] For example, if the naked eye 3D theme currently used by the terminal device is the Emoji theme, the terminal device's off-screen interface displays multiple Emoji expressions arranged in an array, each Emoji expression has a different style, but the size and sight direction are the same. The terminal device can select and switch the target Emoji expression and interact with the target Emoji expression through the following process.

[0218] See also Fig.11 As shown in (a) in the figure, in response to the user making an open palm gesture (i.e., the first air gesture) toward the lower half of the screen, the terminal device selects the Emoji expression in the sixth row and second column according to the position corresponding to the user's palm. After the terminal device selects the Emoji expression, it floats and enlarges the Emoji expression, and controls the eyes of other Emoji expressions to look at the Emoji expression.

[0219] See also Fig.11 As shown in (b) in the figure, in response to the user making an open palm gesture facing the middle of the screen (i.e., the first air gesture), the terminal device selects the Emoji expression in the fourth row and second column according to the position corresponding to the user's palm. After the terminal device selects the expression, it floats and enlarges the Emoji expression, and controls the eyes of other Emoji expressions to look at the Emoji expression.

[0220] It can be seen that as the user's first air gesture gradually moves upward, the Emoji expression selected by the terminal device also gradually switches upward. Of course, although not shown in the figure, as the user's first air gesture gradually moves downward / left / right, the Emoji expression selected by the terminal device also gradually switches downward / left / right.

[0221] See also Fig.11As shown in (c), based on the user having selected the Emoji expression in the second column of the fourth row, in response to the user making a palm-down slapping gesture (i.e., the second air gesture), the terminal device switches the Emoji expression to another preset expression, such as an angry Emoji expression. Alternatively, the terminal device can control the Emoji expression to rotate, jump, or play dynamic expressions.

[0222] Optionally, for the Emoji theme, the terminal device can detect the strength of the second air gesture and switch the selected Emoji expression to the Emoji expression corresponding to the strength. Exemplarily, the terminal device can divide the control strength of the second air gesture into three levels: low, medium, and high. Each level of strength corresponds to two expressions. For example, the low level corresponds to the expressions of "naughty" and "questionable", the medium level corresponds to the expressions of "unhappy" and "tired", and the high level corresponds to the expressions of "dizzy" to "angry". For the two expressions in each level, the terminal device can randomly select one to display.

[0223] Example 2-2:

[0224] Take the current naked eye 3D theme of basketball used by the terminal device as an example, see Fig.12 As shown in (a) of FIG. 1 , a plurality of randomly distributed basketballs are displayed at the bottom of the terminal device's off-screen interface, and no basket is displayed. Fig.12 As shown in (b) of FIG. 1 , in response to the user picking up the terminal device, or in response to the user clicking the screen of the terminal device, or in response to the terminal device detecting the user's face information or gesture information, the terminal device displays a full-screen off-screen interface, which includes multiple basketballs and baskets. In the off-screen interface, the terminal device can control the basketball to interact with the user through the following process.

[0225] See also Fig.13 As shown in (a), in response to a gesture of the user's open palm facing the terminal device (i.e., the first air gesture), the terminal device randomly selects a basketball from multiple basketballs at the bottom of the display and controls it to float in the air.

[0226] See also Fig.13 As shown in (b), in response to the user's palm slapping gesture (ie, the second air gesture), the terminal device controls the selected basketball to shoot.

[0227] Optionally, during the movement of the first interactive element, if there is interaction between the first interactive element and the third interactive element, then during the movement of the first interactive element, the third naked-eye 3D animation of the third interactive element is displayed. For example, when the basketball enters the basket and falls, the terminal device can control the net to perform corresponding feedback animation. For example, control the net to shake after the basketball falls; or control the net to be deformed by the falling basketball and then rebound; or, see Fig.14 As shown in (a) to (b) in the figure, the net is controlled to be stretched by the falling basketball and then rebound, so as to make the dynamic effect of the shooting scene more vivid.

[0228] Optionally, the terminal device can also determine the distance of the basketball throw according to the force of the user's palm swinging downward (i.e., the gesture force of the second air gesture). For example, when the force is large or small, the user cannot throw the basketball into the basket; and when the shooting force is within a preset range, the user can throw the basketball into the basket.

[0229] Optionally, the terminal device can also simulate gravity for the motion elements to make the motion process closer to the real scene. For example, the terminal device can detect the direction of gravity through the acceleration sensor 280E and control the basketball to fall along the direction of gravity after shooting. Through throwing scenes such as basketball, gravity simulation can be presented in a more interesting motion way, with a better user experience.

[0230] Optionally, the terminal device may also set shooting easter eggs for shooting scenes, for example, after each successful shot by the user, or after multiple consecutive successful shots, the terminal device vibrates and / or displays corresponding cheers to improve the user experience.

[0231] Through the above control process, the ball throwing process can be made smoother, which can enhance the user's satisfaction during use.

[0232] It should be noted that the above only takes the shooting scene as an example to illustrate the process of the user controlling the dynamic effect display through air gestures. Of course, in addition to the shooting scene, the above method is also applicable to other game interactive scenes such as playing badminton, playing tennis, and playing volleyball.

[0233] Example 2-3:

[0234] In some embodiments, in response to an air gesture moving toward a first direction, the terminal device controls the interactive elements in the display interface to rotate toward the first direction; in response to an air gesture moving toward a second direction, the terminal device controls the interactive elements in the display interface to rotate toward the second direction; wherein the first direction and the second direction are different.

[0235] For example, for a naked-eye 3D theme including eye elements, in response to a user's air gesture moving toward a first direction, the eyes of multiple naked-eye 3D elements in the naked-eye 3D theme look toward the first direction. In other words, the user can control the sight direction of the naked-eye 3D elements through the gesture direction, and the control process is quite interesting.

[0236] The terminal device's off-screen interface displays an array of Emoji expressions, each with a different style but the same size and sight direction. Fig.15 As shown in (a) to (b), when the user's palm is facing the screen, the Emoji expression does not rotate. In response to the gesture of the user's palm swinging to the left, each Emoji expression in the screen-off interface looks to the left of the illustrated direction. In addition, although not shown in the figure, in response to the gesture of the user's palm swinging to the right, each Emoji expression in the screen-off interface looks to the right of the illustrated direction. In response to the gesture of the user's palm swinging upward, each Emoji expression in the screen-off interface looks above the user. In response to the gesture of the user's palm swinging downward, each Emoji expression in the screen-off interface looks downward in the illustrated direction.

[0237] Example 2-4:

[0238] When the user selects the first interactive element and controls the first interactive element to display the second naked-eye 3D motion effect through the second air gesture, the terminal device can also control the physical size (such as length, height, width, radius, etc.) or rotation speed of the first interactive element according to the distance of the second air gesture. The second air gesture is the same as or different from the first air gesture. The second air gesture includes a gesture of changing the distance from the first interactive element, and the second naked-eye 3D motion effect includes a motion related to the physical size of the first interactive element or a motion related to the rotation speed of the first element.

[0239] In some embodiments, as the second air gesture gradually approaches the first interactive element, the physical size of the first interactive element gradually increases; as the second air gesture gradually moves away from the first interactive element, the physical size of the first interactive element gradually decreases. For example, after the user selects an Emoji expression through the first air gesture, as the user's palm gradually approaches the Emoji expression (i.e., gradually approaches the screen), the radius of the Emoji expression gradually increases. And as the user's palm gradually moves away from the Emoji expression, the radius of the Emoji expression gradually decreases.

[0240] In other embodiments, as the second air gesture gradually approaches the first interactive element, the rotation speed of the first interactive element gradually increases; as the second air gesture gradually moves away from the first interactive element, the rotation speed of the first interactive element gradually decreases.

[0241] In some embodiments, the terminal device receives user data collected by other electronic devices (such as a second electronic device) such as a smart bracelet and a heart rate monitor; and displays the naked-eye 3D animation corresponding to the interactive element based on the user data. For example, when the interactive element in the naked-eye 3D theme is an expression element, the terminal device can detect the user's heart rate data through the smart bracelet; and display the naked-eye 3D animation corresponding to the expression element based on the user's heart rate data; wherein the heart rate data is related to the emotion of the expression element. For example, the faster the heart rate, the more excited the user is, and the emotion of the expression element can be happy; the slower the heart rate, the more stable the user is, and the emotion of the expression element can be expressionless.

[0242] Through the method provided in the embodiment of the present application, the terminal device can collect richer user data with the help of other electronic devices, so as to display expression effects that are more in line with the user's mood.

[0243] After introducing the user operation and interface display process in the gesture control scenario of the terminal device, the control method for the terminal device to implement the above process is specifically described below.

[0244] Fig.16 This is a flow chart of a method for displaying a naked-eye 3D theme in a shaking scene provided by an embodiment of the present application. The method is applied to a terminal device provided with a naked-eye 3D theme, and specifically includes the following steps S1601 to S1604.

[0245] S1601: The terminal device displays at least one interactive element of naked-eye 3D visual effects on a display interface.

[0246] Exemplarily, the interactive element may be a circular or other shaped interactive element. The display interface may be a desktop, a lock screen interface, or an off screen interface. This embodiment does not limit this.

[0247] S1602: The terminal device collects image information through a camera.

[0248] S1603, the terminal device identifies the collected image information to determine the relevant information of the air gesture.

[0249] In this embodiment, the terminal device obtains image information from the camera module through the parameter management module 334 of the theme engine 330. Then, the calculation and execution module 335 determines the position, type, strength and other related information of the air gesture according to the image information.

[0250] In this embodiment, the terminal device can use the Swing algorithm to identify the collected image information to determine the relevant information of the air gesture. Exemplarily, the relevant information of the air gesture includes at least one of the type, strength and position of the air gesture. Depending on the theme applied by the terminal device, the relevant information that the terminal device needs to determine is different. For example, in the Emoji expression scenario provided in the above example 2-1, the terminal device needs to determine the type, strength and position of the air gesture. In the shooting scenario shown in the above example 2-2, the terminal device only needs to determine the type and strength of the air gesture, without paying attention to the position of the gesture. In the Emoji expression scenario shown in the above example 2-3, the terminal device only needs to determine the type of air gesture, without paying attention to the position and strength of the gesture.

[0251] Regarding the determination of the air gesture type:

[0252] In this embodiment, the terminal device uses an artificial intelligence (AI) model to identify the type of user's air gesture from the image information collected by the terminal device. The AI ​​model is trained based on palm images of different sizes, user ages, skin colors, shapes, and gestures, which helps to improve the accuracy of gesture recognition by the AI ​​model.

[0253] Regarding the determination of the strength of the air gesture:

[0254] During the gesture movement, the background of the palm can be considered as unchanged, and this part does not need to be paid special attention to. Therefore, in this embodiment, the terminal device uses the palm as the analysis subject to identify the position of the gesture.

[0255] In some embodiments, the terminal device uses a background difference method to detect the image information used for collection, removes the unchanged background in the image, and leaves the changing object (i.e., the palm) in the scene. The terminal device extracts the target object (i.e., the palm) in the first frame image through algorithm elements, and then selects the area of ​​the target object and records the position. The above process is repeated in the next frame of the picture, and finally the positions of different areas of the same object in the two frames are compared. The frames are used as units and the pixels are used as the moving distance, which can be converted into the corresponding acceleration and speed of the object. Different accelerations and speeds correspond to different gesture strengths, which can produce different interactive effects on the interactive elements.

[0256] Regarding the determination of the position of the air gesture:

[0257] In this embodiment, the position where the camera can identify the user's palm is taken as the plane, and the 9 points where the user's palm is located directly above the device screen are collected respectively, including the four vertices of the device screen, the midpoint of the screen edge and the center point of the screen. Based on these points, a rectangular plane corresponding to the interactive elements on the screen (such as the Emoji expression array) can be constructed, and the mapping relationship between the position of the gesture and the interactive elements in the interface can be obtained.

[0258] S1604, the terminal device detects the direction of gravity.

[0259] In this embodiment, the terminal device obtains acceleration information from the acceleration sensor 280E through the parameter management module 334 of the theme engine 330, and determines the gravity direction according to the acceleration information.

[0260] It should be noted that S1604 is an optional step, that is, the terminal device needs to detect the direction of gravity only in a specific scenario. For example, the terminal device needs to detect the direction of gravity in a basketball shooting scenario.

[0261] S1605, the terminal device displays the naked-eye 3D motion effects of the interactive elements according to the relevant information of the air gesture and the direction of gravity.

[0262] In this embodiment, the dynamic effect and interaction control unit in the DSL parsing and execution module 331 of the terminal device generates dynamic effects of interactive elements according to the relevant information of the air gesture and the direction of gravity, and the unified desktop module 320 displays the corresponding dynamic effects.

[0263] For example, the terminal device first determines the dynamic effects of selecting and throwing a ball based on the type, strength, and gravity direction of the air gesture. Then, the terminal device uses the theme engine to render and display the dynamic effects of selecting and throwing a ball during the pitching phase. Finally, the terminal device uses the theme engine to render and display the dynamic effects of the ball falling during the ball falling phase.

[0264] It should be noted that if the terminal device does not execute S1604, then in S1605, the terminal device only needs to display the naked-eye 3D animation of the interactive element based on the relevant information of the air gesture.

[0265] 3. Eye tracking scenarios

[0266] In an embodiment of the present application, for interactive elements with eyes, such as Emoji expressions, cartoon characters, pets and people, the terminal device can detect the angle at which the user is looking at the screen (i.e., viewing angle) when displaying these interactive elements, and control the eyes of the interactive elements to rotate with the user's viewing angle, that is, the line of sight of the eyes of the interactive elements is always towards the eyes of the user, or the eyes of the interactive elements are always looking at the user, thereby interacting with the user and increasing the entertainment and fun of the user's use of the terminal device.

[0267] In some embodiments, in response to a user gazing at a display screen of a terminal device at a first perspective, the terminal device controls interactive elements in a first interface to rotate according to the first perspective; and, in response to a user gazing at a display screen of a terminal device at a second perspective, the terminal device controls interactive elements in the first interface to rotate according to the second perspective; wherein the first perspective is different from the second perspective.

[0268] Taking the case where the naked-eye 3D theme applied by the terminal device is the Emoji theme and the display interface is the lock screen interface as an example, when the user is within the shooting range of the camera of the terminal device, the terminal device can display the naked-eye 3D animation through the following process.

[0269] First, see Fig.17 As shown, the terminal device displays multiple Emoji expressions arranged in an array on the lock screen interface according to user operations.

[0270] In some embodiments, when the terminal device is in the screen-off state or the screen-off state, in response to the user pressing the power button, the terminal device first displays the screen-off interface, and previews the naked-eye 3D elements on the screen-off interface, such as Emoji expressions arranged in an array in a partial area of ​​the display interface. Subsequently, in response to the user clicking the screen, the terminal device displays the lock screen interface, and displays the corresponding naked-eye 3D elements on the lock screen interface, such as Emoji expressions arranged in an array in the entire area of ​​the lock screen interface.

[0271] In other embodiments, when the terminal device is in the screen-off state or the screen-off state, in response to the user standing up the terminal device from a horizontal state, or in response to the user looking at the screen of the terminal device, or in response to the user pressing the power button of the terminal device, the terminal device enters the lock screen interface and displays the corresponding naked-eye 3D elements on the lock screen interface, for example, Emoji expressions arranged in an array over the entire area of ​​the lock screen interface.

[0272] Then, the user moves closer to the terminal device screen so that the distance between the face and the terminal device is within a preset range. Exemplarily, the preset range may be within 50 cm, within 30 cm, etc.

[0273] Then, the terminal device displays the perspective of the Emoji expression according to the user's perspective. For example, see Fig.18 As shown in (a) in FIG, when the user looks at the screen from the bottom of the terminal device, the Emoji expression's gaze turns to the bottom of the terminal device and looks at the user. Fig.18 As described in (b) of , when the user looks at the screen from the top of the terminal device, the Emoji expression's gaze turns to the top of the terminal device and looks at the user. Fig.18As shown in (c) in FIG. 1 , when the user looks at the screen from the left side of the terminal device, the Emoji expression's gaze turns to the left side of the terminal device and looks at the user. Fig.18 As shown in (d) in the figure, when the user looks at the screen from the left side of the terminal device, the Emoji expression's gaze turns to the right side of the terminal device and looks at the user.

[0274] In the above process, the user can keep the posture of the terminal device unchanged and adjust the viewing angle of the screen by changing his own posture; or keep his own position unchanged and adjust the viewing angle of the screen by changing the posture of the terminal device; of course, the user can also change the posture of himself and the terminal device at the same time to adjust the viewing angle of the screen.

[0275] In summary, when the user is within the shooting range of the terminal device's camera, the terminal device can detect the user's perspective and control the perspective of the Emoji expression to gradually move with the user's perspective, so that the Emoji expression always looks at the user's eyes, that is, always looks at the user, which is highly interactive and interesting.

[0276] In addition, when the user exits the camera shooting range of the terminal device, the terminal device cannot detect the user's perspective and cannot interact with the user based on the user's perspective. In this case, the terminal device displays the naked eye 3D animation effect on the lock screen interface with a preset perspective. For example, the terminal device displays the naked eye 3D animation effect with a vertical screen visual effect.

[0277] After introducing the user operation and interface display process in the terminal device shaking scenario, the control method for the terminal device to implement the above process is specifically described below.

[0278] Fig.19 This is a schematic flow chart of displaying naked-eye 3D themes in a line-of-sight tracking scenario provided by an embodiment of the present application. The method includes the following steps S1901 to S1905.

[0279] S1901, the terminal device displays at least one interactive element of naked-eye 3D visual effects on the display interface.

[0280] Exemplarily, the interactive element may be a circular or other shaped interactive element. The display interface may be a desktop, a lock screen interface, or an off screen interface. This embodiment does not limit this.

[0281] S1902: The terminal device collects image information through a camera.

[0282] S1903: The terminal device determines the viewing angle of the user looking at the terminal device screen based on the collected image information.

[0283] In this embodiment, the terminal device obtains image information from the camera module through the parameter management module 334 of the theme engine 330. Then, the calculation and execution module 33 determines the viewing angle of the user looking at the terminal device screen according to the image information.

[0284] In this embodiment, the terminal device detects the user's viewing angle, including the following two parts: (1) eye area detection, and (2) determining the user's viewing angle based on the eye area. The details are as follows.

[0285] (1) Eye area detection:

[0286] First, the terminal device performs a histogram transformation on the collected user image to obtain a processed grayscale image. Exemplarily, the terminal device can determine a neighborhood (a rectangular area) in the user image, and then gradually perform a histogram transformation on the user image while moving the center of the neighborhood. For example, the center of the neighborhood is moved one pixel at a time from left to right and from top to bottom along the image, and at each pixel position, a histogram transformation is performed on the neighborhood of the pixel point, and finally the grayscale processing is performed on the center point of the neighborhood to obtain a processed grayscale image. This method can reduce the impact of different lighting environments on image recognition and restore some dark details in the original image that cannot be seen due to lighting factors.

[0287] Then, the terminal device extracts facial features from the grayscale image processed by the active shape model (ASM) local sensitive histogram to determine the area where the human eye is located. It should be noted that after the terminal device recognizes the human eye, the terminal device marks the eye features in the image and puts the marked image into the training set as a new shape for training.

[0288] Through the above process, the terminal device can accurately identify the eye area even in situations where the light is uncontrolled and the head moves freely.

[0289] (2) Determining the user’s viewing angle based on the eye area

[0290] In this embodiment, the terminal device estimates the sight direction through the iris center and the inner eye corner in the detected eye area. Because the inner eye corner is insensitive to changes in facial expressions and eye states, the inner eye corner is usually regarded as a reference point for sight estimation, and the sight direction determined based on the inner eye corner is more accurate.

[0291] In some embodiments, the terminal device may detect the eye area for iris detection. For example, the terminal device first estimates the iris radius and then determines the position of the iris center by combining the image energy intensity and edge information.

[0292] In some embodiments, the terminal device identifies the inner canthus of the eye through a trained neural network model.

[0293] The terminal device determines the vector from the center of the iris to the inner corner of the eye, and maps the vector and the distance from the face to the terminal device in the sensor coordinate system. By taking the center of the sensor coordinate system as the observation object and receiving the light reflected by the eye, the eye-screen perspective conversion can be realized to determine the perspective of the human eye looking at the terminal device screen.

[0294] S1904, the terminal device displays the naked-eye 3D motion effects of the interactive elements according to the viewing angle of the user looking at the terminal device screen.

[0295] Exemplarily, the terminal device uses a theme engine to render and display various interactive elements in the lock screen interface according to the perspective of the user looking at the terminal device screen.

[0296] To sum up, the display method of the naked-eye 3D theme provided in this embodiment can detect the viewing angle of the user looking at the screen, and control the eyes of the interactive elements in the theme to always look at the user's eyes, that is, always look at the user, which has strong interactivity and fun.

[0297] (IV) Touch control scenario

[0298] In some embodiments, in response to a user's touch operation on a target interactive element, the terminal device displays a first naked-eye 3D animation effect of the interactive element, and controls other elements except the target element to display a second naked-eye 3D animation effect, wherein the first naked-eye 3D animation effect is different from the second naked-eye 3D animation effect.

[0299] Taking the naked-eye 3D theme currently used by the terminal device as the Emoji theme as an example, the terminal device can control the Emoji expressions to interact with the user through the following process.

[0300] See also Fig. 20 As shown in (a) in the figure, the terminal device's off-screen interface displays Emoji expressions arranged in an array. The styles of each Emoji expression are different, but their sizes and sight directions are the same.

[0301] See also Fig. 20 As shown in (b), in response to the user clicking on the Emoji expression in the second row and second column, the terminal device switches the Emoji expression to a preset Emoji expression (such as an angry Emoji expression). At the same time, the styles of other Emoji expressions except this expression remain unchanged, but the eyes all look at the Emoji expression.

[0302] See also Fig. 20As shown in (c), in response to the user clicking on the Emoji expression in the first column of the third row, the terminal device switches the Emoji expression to a preset Emoji expression (such as an angry Emoji expression). At the same time, the styles of other Emoji expressions except this expression remain unchanged, but the eyes all look at the Emoji expression.

[0303] See also Fig. 20 As shown in (d), in response to the user clicking on the Emoji expression in the third column of the fourth row, the terminal device switches the Emoji expression to a preset Emoji expression (such as an angry Emoji expression). At the same time, the styles of other Emoji expressions except this expression remain unchanged, but the eyes all look at the Emoji expression.

[0304] Through the method provided in the embodiment of the present application, the user can interact with each element in the naked-eye 3D theme in an interesting way through touch operation, which provides a good user experience.

[0305] In some embodiments, the terminal device can also display interactive elements according to user operations in the off-screen interface. Fig.21 As shown, the off-screen interface of the terminal device is partially lit and displays multiple Emoji expressions. In response to the user touching the off-screen interface, or in response to the user picking up the terminal device, the off-screen interface of the terminal device is completely lit and displays these Emoji expressions. Alternatively, see Fig. 22 As shown, the screen-off interface of the terminal device is fully lit, and multiple basketballs are displayed on the left and right sides. When the user does not operate the terminal device for a long time, or when the user presses the power button, the upper half of the screen-off interface gradually goes out, and the basketballs on the left and right sides gradually fall to the bottom of the screen-off interface.

[0306] (V) Folding device expansion scenario

[0307] In some embodiments, when the display method of the naked-eye 3D theme provided in this embodiment is applied to a folding device, when the folding device is in a folded state, its first screen is used to display the user interface. In this case, in response to the operation of unfolding the folding device, the terminal device switches from the first screen to the second screen to display the naked-eye 3D dynamic effect of the interactive element. The first screen is the display screen used by the terminal device in the folded state, and the second screen is the display screen used by the terminal device in the unfolded state.

[0308] The foldable device can display corresponding naked-eye 3D motion effects on the unfolded screen according to its unfolding type.

[0309] For example, see Fig.23As shown in (a) to (e) in the figure, for a mobile phone that is folded up and down, when the screen of the mobile phone is folded to half-expanded, the interactive elements (such as Emoji expressions, basketballs, etc.) gradually fall from the top of the screen to the folded part. When it is from the half-expanded state to the fully expanded state, the interactive elements gradually fall from the folded part to the bottom of the screen.

[0310] For example, see Fig.24 As shown in (a) to (c) in the figure, for a left-to-right foldable mobile phone, in the process of the mobile phone screen being folded and unfolded, interactive elements (such as Emoji expressions, basketballs, etc.) gradually collide from the upper areas of the two sides of the mobile phone, collide with each other on the screen and fall to the bottom of the screen.

[0311] For another example, for a multi-fold (such as a triple-fold) terminal device, when the folding screen is unfolded, interactive elements (such as Emoji expressions, basketballs, etc.) can gradually collide from the left and right sides of the screen, collide with each other on the screen and fall to the bottom of the screen.

[0312] The method provided in this embodiment can control the interactive elements to interact with the user during the process of unfolding the foldable device, making the unfolding process of the foldable device more interesting.

[0313] Based on the same concept, as an implementation of the above method, an embodiment of the present application provides a display device, and the device embodiment corresponds to the above method embodiment. For ease of reading, the present device embodiment will no longer repeat the details of the above method embodiment one by one, but it should be clear that the device in this embodiment can correspond to and implement all the contents of the above method embodiment.

[0314] Fig.25 25 is a schematic diagram of an information sharing device provided by an embodiment of the present application. The display device 2500 is applied to a terminal device, and the terminal device applies a naked eye 3D theme. The display device 2500 includes a first display module 2501 and a second display module 2502. The functions of each module are specifically as follows.

[0315] The first display module 2501 is used to display N interactive elements in the naked-eye 3D theme on the first interface, where N≥1 and is an integer.

[0316] The second display module 2502 is used to display, in response to the control operation, a naked-eye 3D animation effect corresponding to at least one interactive element on the first interface; wherein the naked-eye 3D animation effect is generated according to the control operation. The process of displaying, in response to the control operation, a naked-eye 3D animation effect corresponding to at least one interactive element on the first interface is described above and will not be repeated here.

[0317] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0318] An embodiment of the present application further provides an electronic device, which includes a display method configured to execute the naked-eye 3D theme shown in the above-mentioned embodiments.

[0319] The present application also provides a chip, see Fig.26 As shown, the chip includes a processor and a memory, in which a computer program is stored. When the computer program is executed by the processor, the display method of the naked-eye 3D theme in the above-mentioned embodiments is implemented.

[0320] An embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the display method of the naked-eye 3D theme provided in the above embodiments is implemented.

[0321] An embodiment of the present application also provides a computer program product, which includes a computer program. When the computer program is executed by an electronic device, the electronic device implements the display method of the naked-eye 3D theme provided in the above embodiments.

[0322] It should be understood that the processor mentioned in the embodiments of the present application may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0323] It should also be understood that the memory mentioned in the embodiments of the present application may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (doubledatarate SDRAM, DDR SDRAM), enhanced synchronous dynamic random access memory (enhanced SDRAM, ESDRAM), synchronous link dynamic random access memory (synchlink DRAM, SLDRAM), and direct rambus RAM (DR RAM).

[0324] In the embodiments provided in the present application, the division of each framework or module is only a logical function division. There may be other division methods in actual implementation, for example, multiple frameworks or modules can be combined or integrated into another system, or some features can be ignored or not executed. In addition, each functional module in each embodiment of the present application can be integrated into a processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The above-mentioned integrated modules can be implemented in the form of hardware or in the form of software functional modules.

[0325] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0326] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0327] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A method for displaying naked-eye 3D themes, characterized in that: Applied to a terminal device, the method comprises: Displaying N interactive elements in the naked-eye 3D theme on a first interface, where N≥1 and is an integer, and the first interface includes a desktop, a lock screen interface, or a screen-off interface of the terminal device; In response to a control operation, a naked-eye 3D motion effect corresponding to at least one of the interactive elements is displayed on the first interface; wherein the naked-eye 3D motion effect is generated according to the control operation.

2. The method according to claim 1, characterized in that In response to the control operation, displaying, on the first interface, at least one naked-eye 3D motion effect corresponding to the interactive element includes: In response to a first air gesture at a first position, selecting a first interactive element from the N interactive elements, and displaying a first naked-eye 3D motion effect of the first interactive element; In response to the first air gesture at the second position, deselecting the first interactive element from the N interactive elements, selecting a second interactive element, and displaying a naked-eye 3D motion effect of the second interactive element; The first position is different from the second position, and the first interactive element is different from the second interactive element.

3. The method according to claim 1, characterized in that In response to the control operation, displaying the naked-eye 3D motion effect corresponding to the interactive element on the first interface includes: In response to the first air gesture, a first interactive element is randomly selected from the N interactive elements, and a first naked-eye 3D motion effect of the first interactive element is displayed.

4. The method according to claim 2 or 3, characterized in that When the first interactive element is selected, the method further includes: In response to the second air gesture, a second naked-eye 3D animation effect of the first interactive element is displayed on the first interface.

5. The method according to claim 4, characterized in that The second air gesture includes a gesture of changing the distance from the first interactive element, and the second naked-eye 3D animation includes an animation related to the physical size of the first interactive element or an animation related to the rotation speed of the first element.

6. The method according to claim 5, characterized in that In response to the second air gesture, displaying the second naked-eye 3D animation effect of the first interactive element on the first interface includes: determining a gesture strength of the second air gesture; According to the gesture strength, a second naked-eye 3D motion effect of the first interactive element is displayed on the first interface.

7. The method according to any one of claims 4 to 6, characterized in that: The first air gesture is the same as the second air gesture.

8. The method according to any one of claims 4 to 7, characterized in that: The method further comprises: In the second naked-eye 3D motion effect, if there is interaction between the first interactive element and the third interactive element, the third naked-eye 3D motion effect corresponding to the third interactive element is displayed, and the first naked-eye 3D motion effect is different from the third naked-eye 3D motion effect.

9. The method according to any one of claims 2 to 8, characterized in that: The method further comprises: In response to the first air gesture, displaying a fourth naked-eye 3D motion effect of other interactive elements except the first interactive element on the first interface, wherein the first naked-eye 3D motion effect is different from the fourth naked-eye 3D motion effect; Among the other interactive elements, at least two interactive elements have the same naked-eye 3D motion effects.

10. The method according to claim 1, characterized in that In response to the control operation, displaying the naked-eye 3D motion effect corresponding to the at least one interactive element on the first interface includes: In response to an air gesture moving toward a first direction, controlling the interactive element to rotate toward the first direction; In response to an air gesture moving toward a second direction, controlling the interactive element to rotate toward the second direction; The first direction and the second direction are different.

11. The method according to claim 1, characterized in that: In response to the control operation, displaying the naked-eye 3D motion effect corresponding to the interactive element on the first interface includes: In response to the user gazing at the display screen of the terminal device from a first viewing angle, the interactive element rotates according to the first viewing angle; and In response to the user gazing at the display screen of the terminal device from a second viewing angle, the interactive element rotates according to the second viewing angle; wherein the first viewing angle is different from the second viewing angle.

12. The method according to claim 1, characterized in that In response to the control operation, displaying the naked-eye 3D motion effect corresponding to the interactive element on the first interface includes: In response to the terminal device being tilted toward a first direction under the control operation, controlling the interactive element to move toward the first direction on the first interface; The tilting posture of the terminal device toward the first direction is determined by data collected by a gyroscope sensor.

13. The method according to claim 1, characterized in that In response to the control operation, displaying the naked-eye 3D motion effect corresponding to the interactive element on the first interface includes: According to the shaking strength of the terminal device, displaying the naked-eye 3D motion effect corresponding to the interactive element; Among them, the shaking force is determined according to the acceleration data collected by the acceleration sensor, and there is a mapping relationship between the shaking force and the naked-eye 3D motion effect.

14. The method according to claim 13, characterized in that The interactive element in the naked eye 3D theme is an expression element, and the displaying of the corresponding naked eye 3D dynamic effect of the interactive element according to the shaking strength of the terminal device includes: According to the shaking force, the naked-eye 3D motion effect of the expression element corresponding to the emotion is displayed; wherein there is a preset mapping relationship between the shaking force and the emotion.

15. The method according to claim 1, characterized in that In response to the control operation, displaying the naked-eye 3D motion effect corresponding to the interactive element on the first interface includes: Receiving user data collected by the second electronic device; According to the user data, the corresponding naked-eye 3D animation effect of the interactive element is displayed.

16. The method according to claim 15, characterized in that The interactive element in the naked-eye 3D theme is an expression element, and the user data includes the user's heart rate data; and displaying the corresponding naked-eye 3D dynamic effect of the interactive element according to the user data includes: According to the heart rate data, the naked-eye 3D animation corresponding to the expression element is displayed; wherein the heart rate data is related to the emotion of the expression element.

17. The method according to claim 1, characterized in that In response to the control operation, displaying the naked-eye 3D motion effect corresponding to the interactive element on the first interface includes: In response to a touch operation on a target interactive element, the target element is controlled to display a first naked-eye 3D motion effect, and other interactive elements except the target element are controlled to display a second naked-eye 3D motion effect, where the first naked-eye 3D motion effect is different from the second naked-eye 3D motion effect.

18. The method according to claim 1, characterized in that The terminal device is a foldable device, and the method further includes: In response to the operation of unfolding the folding device, the naked-eye 3D motion effect of the interactive element is displayed by switching from the first screen to the second screen.

19. An electronic device, characterized in that: The method comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method according to any one of claims 1 to 18 when executing the computer program.

20. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 18 is implemented.

21. A chip, characterized in that: The chip includes a processor and a memory, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the method according to any one of claims 1 to 18 is implemented.

Images