Display method for autostereoscopic 3D theme, and electronic device, storage medium and chip

Abstract

The embodiments of the present application relate to the technical field of terminals. Provided are a display method for an autostereoscopic 3D theme, and an electronic device, a storage medium and a chip. The method is applied to a terminal device, and comprises: displaying, on a first interface, N interactive elements in an autostereoscopic 3D theme, wherein N≥1 and N is an integer; and in response to a control operation, displaying, on the first interface, an autostereoscopic 3D motion effect corresponding to at least one interactive element, wherein the autostereoscopic 3D motion effect is generated on the basis of the control operation. By means of the technical solution provided in the embodiments of the present application, a terminal device can not only display various elements in themes with more stereoscopic and vivid 3D visual effects, but can also control the elements to interact with users in response to a control operation of the users, which provides stronger appeal and a better user experience.

Description

Display method of naked-eye 3D theme, electronic device, storage medium and chip

[0001] The present application claims priority from the Chinese patent application No. 202411335271.2 filed on September 23, 2024, and entitled "Display method of naked-eye 3D theme, electronic device, storage medium and chip", the whole content of which is incorporated herein by reference. TECHNICAL FIELD

[0002] The present application relates to the technical field of terminal, and in particular to a display method of naked-eye 3-dimensional (3D) theme, an electronic device, a storage medium and a chip. BACKGROUND

[0003] At present, terminal devices provide various themes, such as cartoon themes and landscape themes, and some themes also have animation elements. Users can change the display style of the operation interface of the terminal device by setting the theme, such as changing the display style of elements such as desktop wallpaper, lock screen wallpaper, icons, fonts, etc., to achieve personalized settings of the terminal device. However, the current theme elements are not stereoscopic and lively enough in vision, and the animation elements included therein are usually fixed and have poor interest, resulting in poor user experience. SUMMARY

[0004] The present application provides a display method of naked-eye 3D theme, an electronic device, a storage medium and a chip, to solve the problem of insufficient stereoscopic and lively theme elements and poor interest in the prior art.

[0005] In a first aspect, the present application provides a display method of naked-eye 3D theme, which is applied to a terminal device, and includes: displaying N interactive elements in a naked-eye 3D theme on a first interface, where N is an integer greater than or equal to 1; and in response to a control operation, displaying a naked-eye 3D dynamic effect corresponding to at least one interactive element on the first interface, wherein the naked-eye 3D dynamic effect is generated according to the control operation.

[0006] For example, the first interface includes a desktop, a lock screen interface or an off-screen interface of the terminal device, etc.

[0007] Through the method provided by the present application, the terminal device can not only display the elements in the theme with more stereoscopic and lively 3D visual effects, but also control the interaction between these elements and the user, which has strong interest and good user experience.

[0008] In some embodiments, in response to the control operation, displaying the naked-eye 3D effect corresponding to the at least one interactive element on the first interface comprises: in response to the first hovering gesture at the first position, selecting a first interactive element from the N interactive elements and displaying a first naked-eye 3D effect of the first interactive element; and in response to the first hovering gesture at the second position, deselecting the first interactive element, selecting a second interactive element, and displaying a naked-eye 3D effect of the second interactive element. The first position is different from the second position, and the first interactive element is different from the second interactive element.

[0009] By the method provided in the embodiments of the present application, the terminal device can select the corresponding interactive element according to the position of the hovering gesture of the user, and control the interactive element to interact with the user, thereby improving the user experience.

[0010] In some embodiments, in response to the control operation, displaying the naked-eye 3D effect corresponding to the interactive element on the first interface comprises: in response to the first hovering gesture, randomly selecting a first interactive element from the N interactive elements and displaying a first naked-eye 3D effect of the first interactive element.

[0011] In some embodiments, the second hovering gesture comprises a gesture of changing the distance from the first interactive element, and the second naked-eye 3D effect comprises an effect related to the physical size of the first interactive element or an effect related to the rotation speed of the first interactive element.

[0012] For example, in the process that the second hovering gesture gradually approaches the first interactive element, the physical size of the first interactive element gradually increases; in the process that the second hovering gesture gradually moves away from the first interactive element, the physical size of the first interactive element gradually decreases. Alternatively, in the process that the second hovering gesture gradually approaches the first interactive element, the rotation speed of the first interactive element gradually increases; in the process that the second hovering gesture gradually moves away from the first interactive element, the rotation speed of the first interactive element gradually decreases.

[0013] By the method provided in the embodiments of the present application, after selecting one interactive element, the terminal device can control the size of the interactive element by the distance of the gesture, and the control process is more interesting.

[0014] In some embodiments, in the case of selecting the first interactive element, the method further comprises: in response to the second hovering gesture, displaying a second naked-eye 3D effect of the first interactive element on the first interface. That is, after selecting one interactive element, the terminal device can further control the interactive element and interact with the user.

[0015] In some embodiments, the first hovering gesture is the same as the second hovering gesture. That is, the user can control the interactive element to display different naked-eye 3D effects by the same hovering control gesture.

[0016] In some embodiments, in response to the second mid-air gesture, displaying the second naked-eye 3D dynamic effect of the first interactive element on the first interface includes: determining a gesture strength of the second mid-air gesture; and displaying the second naked-eye 3D dynamic effect of the first interactive element on the first interface according to the gesture strength.

[0017] By the method provided in the embodiments of the present application, the user can interact with the interactive element more closely to the real scene, and the user experience is better. For example, when the first interactive element is a basketball, the greater the gesture strength of the user in patting the basketball, the farther the basketball moves. Or, for example, when the first interactive element is an Emoji expression, the greater the gesture strength of the user in patting the Emoji expression, the more angry the Emoji expression is.

[0018] In some embodiments, the method further includes: in the second naked-eye 3D dynamic effect, if there is an interaction between the first interactive element and a third interactive element, displaying a third naked-eye 3D dynamic effect corresponding to the third interactive element, the first naked-eye 3D dynamic effect being different from the third naked-eye 3D dynamic effect.

[0019] By the method provided in the embodiments of the present application, the associated interactive elements can be linked. For example, in the process of shooting, in the process of displaying the dynamic effect of the basketball, if the basketball is shot into the basket, the dynamic effect corresponding to the net is displayed synchronously.

[0020] In some embodiments, the method further includes: in response to the first mid-air gesture, displaying a fourth naked-eye 3D dynamic effect of other interactive elements except the first interactive element on the first interface, the first naked-eye 3D dynamic effect being different from the fourth naked-eye 3D dynamic effect; and wherein, in the other interactive elements, the naked-eye 3D dynamic effects of at least two interactive elements are the same.

[0021] By the method provided in the embodiments of the present application, in the process of controlling the interactive element, the unselected interactive element can also respond to the control behavior of the user and interact with the user, and the user experience is better.

[0022] In some embodiments, in response to the control operation, displaying the naked-eye 3D dynamic effect corresponding to the at least one interactive element on the first interface includes: in response to a mid-air gesture moving towards a first direction, controlling the interactive element to rotate towards the first direction; and in response to a mid-air gesture moving towards a second direction, controlling the interactive element to rotate towards the second direction; wherein the first direction and the second direction are different. That is, the user controls the rotation direction of the interactive element through the movement direction of the gesture.

[0023] In some embodiments, in response to the control operation, displaying the naked-eye 3D effect corresponding to the interactive element in the first interface includes: in response to the user gazing at the display screen of the terminal device at a first visual angle, the interactive element in the first interface rotates according to the first visual angle; and in response to the user gazing at the display screen of the terminal device at a second visual angle, the interactive element in the first interface rotates according to the second visual angle; wherein the first visual angle is different from the second visual angle. That is, the user controls the rotation of the interactive element by rotating the visual angle of the user, so that the interactive element always faces the user, and the control process is more interesting.

[0024] For example, in response to the user gazing at the display screen of the terminal device at a first visual angle, the eyes of the interactive element gaze at the user along the first visual angle; and in response to the user gazing at the display screen of the terminal device at a second visual angle, the eyes of the naked-eye 3D effect gaze at the user along the second visual angle; wherein the first visual angle is different from the second visual angle.

[0025] In some embodiments, in response to the control operation, displaying the naked-eye 3D effect corresponding to the interactive element in the first interface includes: in response to the terminal device tilting in a first direction under the control operation, controlling the interactive element to move in the first interface in the first direction. Wherein the posture of the terminal device tilting in the first direction is determined by the data collected by the gyroscope sensor. That is, the terminal device can simulate the scene of the interactive element scrolling on the desktop, and as the desktop tilts downward, the interactive element scrolls in the tilting direction, and the control process is more interesting.

[0026] In some embodiments, in response to the control operation, displaying the naked-eye 3D effect corresponding to the interactive element in the first interface includes: displaying the naked-eye 3D effect corresponding to the interactive element according to the shaking intensity of the terminal device; wherein the shaking intensity is determined according to the acceleration data collected by the acceleration sensor, and there is a mapping relationship between the shaking intensity and the naked-eye 3D effect.

[0027] In some embodiments, the interactive element in the naked-eye 3D theme is an expression element, and displaying the naked-eye 3D effect corresponding to the interactive element according to the shaking intensity of the terminal device includes: displaying the naked-eye 3D effect corresponding to the emotion of the expression element according to the shaking intensity; wherein there is a preset mapping relationship between the shaking intensity and the emotion. For example, as the shaking intensity gradually increases, the emotion of the Emoji expression gradually progresses, for example, gradually becomes angry.

[0028] In some embodiments, in response to the control operation, displaying the naked-eye 3D effect corresponding to the interactive element in the first interface includes: receiving user data collected by a second electronic device; and displaying the naked-eye 3D effect corresponding to the interactive element according to the user data. For example, the second electronic device can be a wearable device, such as a smart bracelet.

[0029] In some embodiments, the interactive element in the naked-eye 3D theme is an expression element, the user data comprises heart rate data of the user; and the corresponding naked-eye 3D dynamic effect of the interactive element is displayed according to the user data, which comprises: the corresponding naked-eye 3D dynamic effect of the expression element is displayed according to the heart rate data of the user; and the heart rate data is related to the emotion of the expression element. For example, the faster the heart rate, the more excited the user is, and the emotion of the expression element can be happy; the slower the heart rate, the more stable the user is, and the emotion of the expression element can be expressionless. Through the method provided in the embodiments of the present application, the terminal device can collect more abundant user data with the aid of other electronic devices, so as to display expression dynamic effects that are more in line with the user's mood.

[0030] In some embodiments, in response to the control operation, the corresponding naked-eye 3D dynamic effect of the interactive element is displayed on the first interface, which comprises: in response to the touch operation on the target interactive element, the target element is controlled to display a first naked-eye 3D dynamic effect, and other interactive elements except the target element are controlled to display a second naked-eye 3D dynamic effect, the first naked-eye 3D dynamic effect being different from the second naked-eye 3D dynamic effect.

[0031] That is, in the process of controlling the interactive element, the unselected interactive element can also interact with the user in response to the control behavior of the user, and the user experience is good.

[0032] In some embodiments, the terminal device is a folding device, and the method further comprises: in response to an operation of unfolding the folding device, switching from the first screen to the second screen to display the naked-eye 3D dynamic effect of the interactive element. The first screen is a display screen used by the folding device in a folded state, and the second screen is a display screen used by the folding device in an unfolded state. The method provided in the embodiment can control the interactive element to interact with the user in the process of unfolding the folding device, so that the unfolding process of the folding device is more interesting.

[0033] In a second aspect, the embodiments of the present application provide a display device, which comprises a first display module and a second display module.

[0034] The first display module is configured to display N interactive elements in a naked-eye 3D theme on a first interface, N being an integer greater than or equal to 1.

[0035] The second display module is configured to display a corresponding naked-eye 3D dynamic effect of at least one interactive element on the first interface in response to a control operation; and the naked-eye 3D dynamic effect is generated according to the control operation.

[0036] In a third aspect, the embodiments of the present application provide an electronic device, which comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the method shown in the first aspect when executing the computer program.

[0037] In a fourth aspect, an embodiment of the present application provides a computer readable storage medium, which stores a computer program. The computer program, when executed by a processor, implements the method shown in the first aspect.

[0038] In a fifth aspect, an embodiment of the present application provides a chip. The chip includes a processor and a memory. The memory stores a computer program. The computer program, when executed by the processor, implements the method shown in the first aspect.

[0039] In a sixth aspect, an embodiment of the present application provides a computer program product. The computer program product includes a computer program. When the computer program is run on an electronic device, the electronic device implements the method shown in the first aspect.

[0040] It can be understood that the beneficial effects of the second aspect to the sixth aspect can be referred to the related description in the first aspect, and will not be repeated here. BRIEF DESCRIPTION OF DRAWINGS

[0041] FIG. 1 is a schematic diagram of a two-dimensional theme according to an embodiment of the present application;

[0042] FIG. 2 is a schematic diagram of a hardware structure of a terminal device to which a display method of a naked-eye 3D theme is applied according to an embodiment of the present application;

[0043] FIG. 3 is a schematic diagram of a software architecture related to the display method of the naked-eye 3D theme according to an embodiment of the present application;

[0044] FIG. 4 is a schematic diagram of a process of setting a theme according to an embodiment of the present application;

[0045] FIG. 5 is a flowchart of a display method of a naked-eye 3D theme according to an embodiment of the present application;

[0046] FIG. 6 is a schematic diagram of a display of an Emoji expression when a terminal device shakes according to an embodiment of the present application;

[0047] FIG. 7 is a schematic diagram of Emoji expressions of different emotions according to an embodiment of the present application;

[0048] FIG. 8 is a schematic diagram of a display of an Emoji expression when a terminal device shakes according to another embodiment of the present application;

[0049] FIG. 9 is a flowchart of a display method of a naked-eye 3D theme according to an embodiment of the present application;

[0050] FIG. 10 is a schematic diagram of a sensor coordinate system according to an embodiment of the present application;

[0051] FIG. 11 is a schematic diagram of a user controlling an emoji expression through a midair gesture according to an embodiment of the present application;

[0052] FIG. 12 is a schematic diagram of a display transformation of a screen-off interface according to an embodiment of the present application;

[0053] FIG. 13 is a schematic diagram of a user controlling a shot through a midair gesture according to an embodiment of the present application;

[0054] FIG. 14 is a schematic diagram of a net animation during a shot according to an embodiment of the present application;

[0055] FIG. 15 is a schematic diagram of a user controlling an emoji expression through a midair gesture according to another embodiment of the present application;

[0056] FIG. 16 is a flowchart of a display method of a naked-eye 3D theme in a shaking scenario according to an embodiment of the present application;

[0057] FIG. 17 is a schematic diagram of a lock screen interface according to an embodiment of the present application;

[0058] FIG. 18 is a schematic diagram of a display interface of a naked-eye 3D theme in a gaze tracking scenario according to an embodiment of the present application;

[0059] FIG. 19 is a schematic flowchart of displaying a naked-eye 3D theme in a gaze tracking scenario according to an embodiment of the present application;

[0060] FIG. 20 is a schematic diagram of a display interface of a naked-eye 3D theme in a touch scenario according to an embodiment of the present application;

[0061] FIGS. 21-22 are schematic diagrams of display transformations of a screen-off interface according to different embodiments of the present application;

[0062] FIG. 23 is a schematic diagram of an animation during a folding mobile phone unfolding process according to an embodiment of the present application;

[0063] FIG. 24 is a schematic diagram of an animation during a folding mobile phone unfolding process according to an embodiment of the present application;

[0064] FIG. 25 is a schematic diagram of an information sharing device according to an embodiment of the present application;

[0065] FIG. 26 is a schematic diagram of a structure of a chip according to an embodiment of the present application. DETAILED DESCRIPTION

[0066] It should be understood that, in the description of the embodiments of the present application, unless otherwise specified, " / " represents the meaning of or, for example, A / B can represent A or B; "and / or" herein is only a description of the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B can represent: A exists alone, A and B exist together, and B exists alone.

[0067] In the embodiments, the terms "first", "second" are used only for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined with "first", "second" can explicitly or implicitly include one or more of the features. In the description of the embodiments, unless otherwise specified, the meaning of "multiple" is two or more.

[0068] Before introducing the technical solutions provided by the embodiments of the present application, the application scenarios involved in the present application are first introduced.

[0069] In order to beautify the interface display effect of the terminal device, the terminal device provides various themes, such as cartoon theme, landscape theme, mechanical theme, text theme, etc. The user can change the display style of the operation interface in the terminal device by setting the theme for the terminal device, such as changing the display style of elements such as desktop wallpaper, lock screen wallpaper, screen-off wallpaper, icon, font, etc., changing the playing effect of sound effects, ring tones, etc., realizing the personalized setting of the terminal device.

[0070] With the gradual development of intelligent terminal devices and the improvement of user demand, the theme of the terminal device is also developing and changing step by step. For example, the theme gradually changes from static to dynamic. For another example, some interesting animation plug-ins gradually appear in the theme, making the theme more lively and flexible. For another example, the theme gradually supports user editing or personalized customization.

[0071] However, the current theme is generally two-dimensional (2-dimensional, 2D) visual effect (see FIG. 1), which is not stereoscopic and lively enough. In addition, the animation content in the current theme is usually fixed. For example, the transition animation of the terminal device from screen-off to lock screen to desktop, or the animation displayed by the terminal device in response to the user's click operation, the animation content is pre-set and fixed. In summary, the current theme not only has poor visual effect, but also cannot convert user behavior into animation feedback to the display screen, has poor interactivity with users, is not interesting, and has poor user experience.

[0072] To this end, an embodiment of the present application provides a display method of a naked-eye 3D theme. Through the method, the terminal device can not only display a naked-eye 3D visual effect theme, but also control each naked-eye 3D element in the theme to respond to user behavior, thereby providing the user with an entertaining, interactive and controllable theme.

[0073] The display method of the naked-eye 3D theme provided by the embodiment of the present application will be described in detail below with reference to the accompanying drawings.

[0074] For ease of description, first, the basic concepts and terms related to the embodiment of the present application are introduced. It should be noted that the following introduction is intended to explain and illustrate, not to limit.

[0075] (1) User interface

[0076] The user interface (user interface, UI) is an interface for interaction and information exchange between an application (application, App) and an operating system and a user, used to realize the conversion between the internal existence form of information and the form acceptable to the user. Or, the user interface is an interface source code written in a specific computer language such as java, extensible markup language (extensible markup language, XML), which is parsed and rendered on an electronic device, and finally presented as content that can be recognized by the user. In short, the existence of the user interface makes the interaction between the user and the electronic device more convenient.

[0077] (2) Theme

[0078] The theme is a combination of visual and interactive elements customized for the terminal device, including but not limited to wallpaper, icon, lock screen style, font, sound effect, etc. The user can change the display style of the operation interface of the terminal device by setting the theme for the terminal device, and make personalized settings for the terminal device.

[0079] (3) Naked-eye three-dimensional 3D technology

[0080] The naked-eye 3D technology refers to a display technology in which the user can directly perceive a two-dimensional computer image as a three-dimensional image without the need for special glasses and other external auxiliary devices. This technology mainly relies on visual illusion, and makes the human brain produce a stereoscopic effect through a specific display method. For example, the terminal device can control the color gray scale of each element in the two-dimensional image to make the human brain produce a visual illusion, thereby realizing naked-eye 3D display.

[0081] (4) Screen state

[0082] In this embodiment, the screen state of the terminal device includes, but is not limited to, a screen-on state, a screen-off state, an always-on display (AOD) state, and a lock screen state, and the like. Details are shown as follows.

[0083] The screen-on state refers to a state in which the screen of the terminal device is fully lit up and displays various user interfaces. For example, the user interface can be a desktop, an application interface, a lock screen interface, and the like.

[0084] The screen-off state refers to a state in which the terminal device completely turns off the screen and the phone does not display any information. In the screen-off state, although the interface of the terminal device does not display content, the APP can continue to run in the background.

[0085] The lock screen state refers to a state in which the terminal device locks the screen so that the user cannot operate most functions. For example, the user cannot control the terminal device to display the desktop, cannot use the desktop application, cannot set the terminal device, and the like. However, the terminal device still retains some functions available, such as the unlocking function, the emergency call function, the camera function, and the like. The lock screen state can prevent the user from accidentally touching the screen and prevent others from using the terminal device at will, thereby ensuring the safe use of the terminal device. When the user does not operate the terminal device for a long time, or the user presses the power button on the terminal device, or the user touches a one-key lock screen icon in the display interface, the terminal device enters the lock screen state. When the user wants to use the terminal device again, the terminal device can be unlocked by fingerprint unlocking, gesture unlocking, or face unlocking, and the like.

[0086] The AOD state generally refers to a state in which, when the terminal device is in the lock screen state, the entire screen does not need to be lit up, but only a part of the screen displays time, phone, message, battery information, push message, theme, animation, wallpaper, and the like. In the AOD state, the user can directly see the required information, which can reduce the steps of operating the terminal device by the user and bring convenience to the user. It should be noted that in the AOD state, the APP of the terminal device can continue to run in the background.

[0087] In some embodiments, in the AOD state, the terminal device can display the content to be displayed to the user in full screen. For example, a naked-eye 3D animation in the theme is displayed in full screen.

[0088] (5) Lock screen interface and AOD interface

[0089] The lock screen interface refers to an interface in which the terminal device lights up the entire screen to display time, date, wallpaper, and the like in the lock screen state.

[0090] The AOD interface, also known as the AOD interface, refers to an interface displayed by the terminal device in the AOD state. Depending on the AOD display content set by the terminal device, the AOD interface can be partially lit up or fully lit up.

[0091] After introducing the basic concepts and terms related to the present application, the hardware structure of the terminal device and the related software architecture related to the embodiments of the present application are introduced.

[0092] In the embodiments of the present application, the terminal device can also be referred to as a terminal, a user equipment (UE), a mobile station (MS), a mobile terminal (MT), etc. The terminal device can be a mobile phone, a smart television, a wearable device, a tablet computer (Pad), a computer with wireless transceiver function, a virtual reality (VR) terminal device, an augmented reality (AR) terminal device, a wireless terminal in industrial control, a wireless terminal in self-driving, a wireless terminal in remote medical surgery, a wireless terminal in smart grid, a wireless terminal in transportation safety, a wireless terminal in smart city, a wireless terminal in smart home, a vehicle-mounted device, etc. The embodiments of the present application do not limit the specific technology and specific device form of the terminal device.

[0093] FIG. 2 is a schematic diagram of the hardware structure of a terminal device to which the display method of the naked eye 3D theme provided by the embodiments of the present application is applied. The terminal device can include a processor 210, an external memory interface 220, an internal memory 221, a universal serial bus (USB) interface 230, a charging management module 240, a power management module 241, a battery 242, an antenna 1, an antenna 2, a mobile communication module 250, a wireless communication module 260, an audio module 270, a speaker 270A, a receiver 270B, a microphone 270C, a headset interface 270D, a sensor module 280, a key 290, a motor 291, an indicator 292, a camera 293, a display screen 294, and a subscriber identification module (SIM) card interface 295, etc.

[0094] It can be understood that the structure illustrated in the embodiments of the present application does not constitute a specific limitation on the terminal device. In some other embodiments of the present application, the terminal device can include more or fewer components than the illustration, or combine certain components, or split certain components, or different component arrangements. The illustrated components can be implemented in hardware, software, or a combination of software and hardware.

[0095] As an example, when the terminal device is a mobile phone or a tablet computer, it can include all the components illustrated, or only include part of the components illustrated.

[0096] The processor 210 can include one or more processing units, for example: the processor 210 can include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Among them, different processing units can be independent devices, or can be integrated in one or more processors.

[0097] The USB interface 230 is an interface conforming to the USB standard specification, and can be a Mini USB interface, a Micro USB interface, a USB Type C interface, etc. The USB interface 230 can be used to connect a charger to charge the terminal device, or to transmit data between the terminal device and a peripheral device. It can also be used to connect a headset to play audio through the headset.

[0098] It can be understood that the interface connection relationship between the modules illustrated in the embodiments of the present application is only illustrative and does not constitute a structural limitation on the terminal device. In some other embodiments of the present application, the terminal device can also use different interface connection methods or a combination of multiple interface connection methods in the above embodiments.

[0099] The charging management module 240 is used to receive charging input from a charger. The charger can be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 240 can receive charging input from a wired charger through the USB interface 230. In some wireless charging embodiments, the charging management module 240 can receive wireless charging input through the wireless charging coil of the terminal device. The charging management module 240 can charge the battery 242 while also providing power to the electronic device through the power management module 241.

[0100] The power management module 241 is configured to connect the battery 242 and the charging management module 240 to the processor 210. The power management module 241 receives input from the battery 242 and / or the charging management module 240 to power the processor 210, the internal memory 221, the external memory, the display screen 294, the camera 293, and the wireless communication module 260, etc. The power management module 241 can also be configured to monitor parameters such as battery capacity, battery cycle count, battery health status (leakage, impedance), etc.

[0101] The wireless communication function of the terminal device can be implemented by the antenna 1, the antenna 2, the mobile communication module 250, the wireless communication module 260, the modem processor, and the baseband processor, etc.

[0102] The antenna 1 and the antenna 2 are configured to transmit and receive electromagnetic wave signals.

[0103] The mobile communication module 250 can provide a solution for wireless communication including 2G / 3G / 4G / 5G, etc. applied to the terminal device.

[0104] The wireless communication module 260 can provide a solution for wireless communication applied to the terminal device.

[0105] In some embodiments, the antenna 1 and the mobile communication module 250 of the terminal device are coupled, and the antenna 2 and the wireless communication module 260 are coupled, so that the terminal device can communicate with the network and other devices through wireless communication technology.

[0106] The terminal device can realize the display function through the GPU, the display screen 294, and the application processor, etc. The GPU is a microprocessor for image processing, connected to the display screen 294 and the application processor. The GPU is configured to perform mathematical and geometric calculations for graphics rendering. The processor 210 can include one or more GPUs that execute program instructions to generate or change display information.

[0107] The display screen 294 is configured to display images, videos, etc. For example, the lock screen interface, the AOD interface, the desktop, etc. in the embodiments of the present application.

[0108] The terminal device can realize the shooting function through the ISP, the camera 293, the video codec, the GPU, the display screen 294, and the application processor, etc.

[0109] ISP is used to process the data feedback by camera 293. For example, when taking a photo, the shutter is opened, the light is transmitted to the camera photosensitive element through the lens, and the optical signal is converted into an electrical signal. The camera photosensitive element transmits the electrical signal to the ISP for processing and converts it into an image visible to the naked eye. The ISP can also optimize the noise and brightness of the image through algorithm. The ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, the ISP can be arranged in the camera 293.

[0110] Camera 293 is used to capture still images or videos. Objects generate optical images through lenses and project them onto photosensitive elements. The photosensitive elements convert the optical signals into electrical signals, which are then transmitted to the ISP for conversion into digital image signals. The ISP outputs the digital image signals to the DSP for processing. The DSP converts the digital image signals into image signals in standard RGB, YUV, etc. formats. In some embodiments, the terminal device can include one or N cameras 293, where N is a positive integer greater than 1.

[0111] Digital signal processor is used to process digital signals, in addition to processing digital image signals, it can also process other digital signals. For example, when the terminal device selects a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy, etc.

[0112] Video codec is used for digital video compression or decompression. The terminal device can support one or more video codecs. In this way, the terminal device can play or record videos in multiple encoding formats, such as moving picture experts group (MPEG) 1, MPEG 2, MPEG 3, MPEG 4, etc.

[0113] NPU is a neural-network (NN) calculation processor, which can quickly process input information by borrowing from the structure of biological neural networks, such as borrowing from the transmission mode between human brain neurons, and can also continuously self-learn. Through NPU, the terminal device can realize intelligent cognition and other applications, such as image recognition, face recognition, voice recognition, text understanding, etc.

[0114] In the embodiments of the present application, the NPU or other processors can analyze and process images in the videos stored by the terminal device, etc.

[0115] The external memory interface 220 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the terminal device. The external memory card communicates with the processor 210 through the external memory interface 220 to realize data storage functions. For example, music, video and other files can be saved in the external memory card.

[0116] The internal memory 221 can be used to store computer executable program codes, the executable program codes including instructions. The processor 210 performs various functional applications and data processing of the terminal device by running the instructions stored in the internal memory 221. The internal memory 221 can include a program storage area and a data storage area. The program storage area can store an operating system and at least one application program (such as a sound playing function, an image playing function, etc.) required by a function. The data storage area can store data (such as audio data, a phone book, etc.) created during use of the terminal device.

[0117] In addition, the internal memory 221 can include a high-speed random access memory, and can also include a non-volatile memory, for example, at least one magnetic disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0118] The terminal device can realize an audio function through an audio module 270, a speaker 270A, a receiver 270B, a microphone 270C, an earphone interface 270D, and an application processor, etc.

[0119] The audio module 270 is used to convert a digital audio signal into an analog audio signal output, and is also used to convert an analog audio input into a digital audio signal. The audio module 270 can also be used to encode and decode an audio signal. In some embodiments, the audio module 270 can be arranged in the processor 210, or part of the functions of the audio module 270 can be arranged in the processor 210.

[0120] The speaker 270A, also called a “loudspeaker”, is used to convert an audio electrical signal into a sound signal. The terminal device can listen to music or listen to a hands-free call through the speaker 270A, for example, the speaker can play the comparison and analysis result provided by the embodiments of the present application.

[0121] The receiver 270B, also called an “earpiece”, is used to convert an audio electrical signal into a sound signal. When the terminal device answers a call or a voice message, the receiver 270B can be held close to the ear to listen to the voice.

[0122] The microphone 270C, also called a “microphone”, “sound transducer”, is used to convert a sound signal into an electrical signal.

[0123] The earphone interface 270D is used to connect a wired earphone.

[0124] The sensor module 280 can include a pressure sensor 280A, a gyro sensor 280B, a barometric pressure sensor 280C, a magnetic sensor 280D, an acceleration sensor 280E, a distance sensor 280F, a proximity light sensor 280G, a fingerprint sensor 280H, a temperature sensor 280J, a touch sensor 280K, an ambient light sensor 280L, a bone conduction sensor 280M, etc.

[0125] The pressure sensor 280A is configured to sense a pressure signal and convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor 280A can be disposed on the display screen 294.

[0126] The gyro sensor 280B can be configured to determine a motion posture of the terminal device. In some embodiments, the gyro sensor 280B can be configured to determine an angular velocity of the terminal device about three axes (i.e., x, y, and z axes). The gyro sensor 280B can be used in a camera anti-shake, navigation, and motion sensing game scenario.

[0127] The barometric pressure sensor 280C is configured to measure air pressure. In some embodiments, the terminal device can calculate an altitude, assist positioning and navigation based on an air pressure value measured by the barometric pressure sensor 280C.

[0128] The magnetic sensor 280D includes a Hall sensor. The terminal device can use the magnetic sensor 280D to detect opening and closing of a flip cover.

[0129] The acceleration sensor 280E can detect an acceleration of the terminal device in each direction (typically three axes). When the terminal device is stationary, the acceleration sensor 280E can detect a magnitude and direction of gravity. The acceleration sensor 280E can also be used to identify an electronic device posture, and can be used in a landscape / portrait screen switching application, a pedometer application, etc.

[0130] The distance sensor 280F is configured to measure a distance. The terminal device can measure a distance using infrared or laser. In some embodiments, the terminal device can use the distance sensor 280F to measure a distance in a photographing scenario to achieve fast focusing.

[0131] The proximity light sensor 280G can include, for example, a light emitting diode (LED) and a light detector, such as a photodiode. The terminal device can use the proximity light sensor 280G to detect that a user is holding the terminal device close to an ear for a call, so as to automatically turn off the screen to achieve power saving. The proximity light sensor 280G can also be used for automatic unlocking and locking in a cover mode and a pocket mode.

[0132] Ambient light sensor 280L is used to sense ambient light brightness. The terminal device can adaptively adjust display screen 294 brightness according to the sensed ambient light brightness. Ambient light sensor 280L can also be used to automatically adjust white balance when taking photos. Ambient light sensor 280L can also cooperate with proximity light sensor 280G to detect whether the terminal device is in a pocket to prevent accidental touch.

[0133] Fingerprint sensor 280H is used to collect fingerprints. The terminal device can use the collected fingerprint characteristics to realize fingerprint unlocking, access application lock, take photos with fingerprint, answer incoming calls with fingerprint, etc.

[0134] Temperature sensor 280J is used to detect temperature.

[0135] Touch sensor 280K, also known as "touch panel". Touch sensor 280K can be disposed on display screen 294, and touch screen, also known as "touch panel", is composed of touch sensor 280K and display screen 294. Touch sensor 280K is used to detect touch operations acting on or near it. Touch sensor can pass the detected touch operation to the application processor to determine the touch event type. Visual output related to the touch operation can be provided through display screen 294. In other embodiments, touch sensor 280K can also be disposed on the surface of the terminal device, which is different from the position where display screen 294 is located.

[0136] Bone conduction sensor 280M can obtain vibration signals. In some embodiments, bone conduction sensor 280M can obtain vibration signals of human body sound part vibration bone block. Bone conduction sensor 280M can also contact human body pulse to receive blood pressure pulsation signals.

[0137] Keys 290 include power-on key, volume key, etc. Keys 290 can be mechanical keys. They can also be touch keys. The terminal device can receive key input and generate key signal input related to user settings and function control of the terminal device.

[0138] Motor 291 can generate vibration prompts. Motor 291 can be used for incoming call vibration prompts, and also for touch vibration feedback. For example, touch operations acting on different applications (such as taking photos, playing audio, etc.) can correspond to different vibration feedback effects. Touch operations acting on different regions of display screen 294 can also correspond to different vibration feedback effects.

[0139] Indicator 292 can be an indicator light, which can be used to indicate charging status, power change, and also to indicate messages, missed calls, notifications, etc.

[0140] The SIM card interface 295 is configured to connect a SIM card. The SIM card can be inserted into or pulled out of the SIM card interface 295 to realize contact and separation with the terminal device. The terminal device can support one or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 295 can support a Nano SIM card, a Micro SIM card, a SIM card, and the like.

[0141] FIG. 3 is a schematic diagram of a software architecture involved in a display method of a naked-eye 3D theme according to an embodiment of the present application. As shown in FIG. 3, the software architecture includes a theme package module 310, a unified desktop module 320, and a theme engine 330.

[0142] The theme package module 310 is configured to store theme packages of various themes, including preset themes, personalized themes, and online themes, and the like. The preset themes refer to themes preconfigured in the operating system of the terminal device, and the user has no permission to add, delete, modify, or edit the preset themes. The personalized themes include various themes edited and set by the user according to personal preferences. The online themes include themes obtained by the terminal device from online theme applications.

[0143] The unified desktop module 320 is capable of controlling and displaying elements such as a lock screen page, a Home page, and a desktop component. In addition, the desktop module 320 further includes a theme editing unit configured to customize and edit themes obtained by the theme engine 330 from the theme package module 310 on the desktop according to user operations.

[0144] The theme engine 330 includes a domain specific language (DSL) parsing and execution module 331, an atomic capability module 332, an engine customized view component 333, a parameter management module 334, and an operation and execution module 335.

[0145] The DSL parsing and execution module 331 is configured to parse a theme package, map the theme package into various atomic capabilities in the theme engine, such as variables, views, and the like, and is responsible for scheduling various capabilities of the theme engine 330.

[0146] In some embodiments, the DSL parsing and execution module 331 includes a DSL script parsing unit, a DSL instruction registration unit, a DSL instruction fast table, a scene management unit, a page drawing unit, and an animation and interaction control unit.

[0147] The DSL script parsing unit is configured to, after the theme engine is started, find a theme script file from a theme directory according to a rule, and parse the theme script file into a data node tree.

[0148] The DSL instruction registration unit is configured to register atomic capabilities when the theme engine is started. The atomic capability can be understood as an instruction for implementing a specific function.

[0149] The DSL instruction fast table is configured to quickly match the corresponding atomic capability from the DSL instruction fast table according to the node type after the DSL script parsing unit parses the theme script into a data node tree. For example, if the type of the data node is a custom variable, the corresponding atomic capability is to register the custom variable to the parameter management module; if the type of the data node is a naked 3D node, the corresponding atomic capability is to form a view tree of a naked 3D image according to related data.

[0150] The scene management unit is configured to classify and manage the matched atomic capabilities according to scenes. For example, the scene can be different theme scenes, such as a free fall scene, a throwing scene, a gesture control scene, a line-of-sight tracking scene, and the like of a theme element.

[0151] The page drawing unit is configured to draw the first frame after the theme is successfully loaded, and to refresh the corresponding visual content after the element in the theme is refreshed.

[0152] The dynamic effect and interaction control unit is configured to display theme information in response to external changes. For example, for a naked 3D theme, in response to external changes such as user shaking of a mobile phone, line-of-sight movement, or a gesture in the air, related data is collected, the data is converted into attribute values of visual elements, such as angles and positions of objects, and a graph is drawn.

[0153] The atomic capability module 332 is a collection of all atomic capabilities supported by the theme engine 330, such as a collection of atomic capabilities such as variable nodes and text box nodes. The atomic capability module 332 includes atomic capabilities related to functions such as 2D basic views, 2D basic dynamic effects, 3D basic views, and personalized customized visual effects. The personalized customized visual effect is a special atomic capability for a naked 3D theme, which includes material display and light display.

[0154] The engine custom view component 333 is a component collection formed by combining and encapsulating underlying UI components to facilitate designers to make themes. For example, for the custom editing capability of the theme engine, various specific components are encapsulated, such as a map selection component, a color selection component, and an Emoji selection component used for custom editing of a naked 3D theme.

[0155] The parameter management module 334 is responsible for the management of parameter variables in the system. In the embodiment, the parameter variables are divided into two categories, namely, self-defined variables and system variables. The self-defined variables are variables defined in the DSL script, which can be fixed values, expressions, or references to other variables. The system variables are unique variables obtained from the system of the terminal device, such as the current power, user steps, system time, and user interaction data (shaking, human eyes, gestures, etc.) required by the naked 3D theme.

[0156] The operation and execution module 335 is an auxiliary module of the motion effect and interaction control module unit, which supports a large number of complex calculation functions and combinations and nesting of these functions. It calculates the specific values of a large number of complex expressions parsed in the DSL script in combination with the obtained variables, and transmits the values to the motion effect and interaction control module for drawing the positions of all visible elements.

[0157] After the terminal device successfully loads the theme from the theme package module 310, the theme engine 330 first parses the theme package. Then, the theme engine 330 performs first-frame drawing according to the script file of the target theme, and displays the drawn first-frame interface through the unity desktop module 320. Then, the theme engine 330 collects and processes user interaction data, and draws animations in real time according to the processing results, and updates the display interface. The unity desktop module 320 displays the updated interface in real time to realize that the elements in the theme respond to the control operations of the user.

[0158] Based on the hardware structure and software architecture of the terminal device provided in the above embodiment, the display method of the naked 3D theme provided in the embodiment of the application is specifically described below.

[0159] First, the process of setting the naked 3D theme for the terminal device by the user is exemplarily introduced.

[0160] Referring to (a) in FIG. 4, various application programs are installed in the terminal device, such as a clock, a calendar, a gallery, a memo, a file management, and a theme application. In response to the selection operation of the user on the theme application, the terminal device displays a theme application interface.

[0161] Referring to (b) in FIG. 4, the theme application interface includes various 2D themes and naked 3D themes that can be selected by the user. The naked 3D theme usually includes one or more naked 3D elements, such as a background element of a naked 3D visual effect and an interactive element. The interactive element can respond to the behavior of the user and interact with the user.

[0162] Exemplarily, the naked-eye 3D theme includes an emoji theme and a ball theme, etc. The emoji is an expression symbol composed of graphics and / or symbols, used to convey emotions, express ideas or add interest to text. The ball can be a tennis ball, a basketball, a volleyball, a football, a table tennis ball, etc. The type of the ball is not limited in the embodiment. In addition, although not shown in the figure, the naked-eye 3D theme can also be a person, an animal, a landscape, a cartoon image, a star, a coin, a vehicle, a building, etc. The type of the naked-eye 3D theme is not limited in the embodiment.

[0163] In response to the selection operation of the target theme by the user, the terminal device can apply the theme, and display the corresponding theme element on the interface such as the desktop, the lock screen interface and the screen-off interface. For example, in response to the selection operation of the emoji theme by the user, the terminal device uses the emoji theme, and displays the emoji expression on the desktop, the lock screen interface (see (c) in FIG. 4) and the screen-off interface (see (d) in FIG. 4). The emoji expression is the interactive element in the theme.

[0164] Next, the process that the terminal device displays the naked-eye 3D theme and interacts with the user is specifically described.

[0165] FIG. 5 is a flowchart of a method for displaying a naked-eye 3D theme according to an embodiment of the application. As shown in FIG. 5, the method includes the following steps S501-S502.

[0166] S501, the terminal device displays N interactive elements in the naked-eye 3D theme on a first interface, where N is an integer greater than or equal to 1.

[0167] Exemplarily, the first interface is the desktop, the lock screen interface or the screen-off interface of the terminal device, and the type of the first interface is not limited in the embodiment.

[0168] S502, the terminal device displays a naked-eye 3D dynamic effect corresponding to at least one interactive element on the first interface in response to a control operation, where the naked-eye 3D dynamic effect is generated according to the control operation.

[0169] Exemplarily, the control operation can be that the user shakes the terminal device, the user controls the terminal device through a gesture, the user controls the terminal device through a line of sight, the user controls the terminal device through a touch operation, or the user unfolds the terminal device in a folded state, and the type of the control operation is not limited in the embodiment.

[0170] Through the method provided in the embodiment of the application, the terminal device can not only display each element in the theme with a more stereoscopic and vivid 3D visual effect, but also control the elements to interact with the user in response to the user behavior, thereby providing a better user experience.

[0171] According to different naked-eye 3D themes, the interaction modes of the interaction elements in the themes with the user are usually different. According to different interaction modes, the interaction scenes include various types, for example, (1) a shaking scene, (2) a gesture control scene, (3) a sight line tracking scene, (4) a touch operation scene, (5) a folding device unfolding scene, and the like. The method provided by the embodiments of the present application is specifically described below in combination with different interaction scenes.

[0172] (1) Shaking scene

[0173] In the shaking scene, the terminal device can respond to the operation of the user shaking the terminal device to control the interaction elements therein to display corresponding interaction effects. For ease of understanding, the embodiments of the present application take a naked-eye 3D theme with circular interaction elements, for example, an Emoji theme, a sphere theme, a star theme, a coin theme, and the like, as an example to introduce the shaking scene. Of course, the shaking scene can also be applicable to any interaction element, taking the circular interaction element as an example.

[0174] In this embodiment, the shaking scene includes tilting the terminal device and shaking the terminal device. Details are shown as follows.

[0175] (1) Tilting the terminal device

[0176] In the process of the user tilting the terminal device, the terminal device can simulate the force-receiving scene of each element, move in the direction in which the terminal device is tilted, display a collision interesting visual effect, and enhance the use and experience of the user. Illustratively, the forces received by the interaction elements include gravity and support forces, and the like. For example, in the process of simulating the rolling of a sphere on a desktop, the forces received by the sphere include gravity and the support force of the desktop.

[0177] In some embodiments, in response to the terminal device being tilted to a first direction, the terminal device controls the interaction element in the display interface to move to the first direction. The interaction element can be a circular or other shaped interaction element. The display interface can be a desktop, a lock screen interface, or an off-screen interface. The embodiments are not limited in this regard.

[0178] Example 1-1:

[0179] The Emoji theme of the naked-eye 3D visual effect currently used by the terminal device, and the interaction elements in the Emoji theme are naked-eye 3D Emoji expressions. When the user tilts the terminal device, the dynamic effects of each Emoji expression are as shown below.

[0180] Referring to (a)-(b) in FIG. 6, the lock screen interface of the terminal device displays a containing space (e.g., a box) of naked-eye 3D visual effects, and the bottom of the containing space displays a plurality of randomly arranged naked-eye 3D visual effects of Emoji expressions. In response to the user controlling the terminal device to tilt to the right, each Emoji expression in the containing space simulates a state of bearing gravity and / or supporting force and rolls to the right in the containing space. If the terminal device remains in the tilted right position, the Emoji expressions will gather on the right side of the containing space (i.e., near the right side of the terminal device).

[0181] In addition, although not shown in the figure, when the user controls the terminal device to tilt to the left, each Emoji expression in the containing space simulates a state of bearing gravity and supporting force and rolls to the left in the containing space. If the terminal device remains in the tilted left position, the Emoji expressions will gather on the left side of the containing space (i.e., near the right side of the terminal device).

[0182] Alternatively, when the user controls the top edge of the terminal device to tilt downward, each Emoji expression in the containing space simulates a state of bearing gravity and supporting force and rolls to the top of the containing space. If the terminal device remains in the tilted top-down position, the Emoji expressions will gather on the top of the containing space (i.e., near the top edge of the terminal device).

[0183] Example 1-2:

[0184] The naked-eye 3D theme currently used by the terminal device is a billiards theme, and the display interface (such as the screen-off interface or the lock screen interface) of the terminal device includes a billiard table surface of naked-eye 3D visual effects of billiards and billiard pockets at the four corners and the middle positions of each side of the billiard table surface. In response to the user tilting the terminal device, each billiard ball rolls in the tilt direction on the billiard table surface, and if its rolling path passes through a billiard pocket, it visually falls into the pocket. For example, when the user tilts the terminal device to the left, a billiard ball rolls to the left, and its rolling path passes through the pocket at the middle position of the left side, so the billiard ball will roll into the pocket.

[0185] (2) Shaking the terminal device

[0186] Compared with tilting the terminal device, the frequency and amplitude of the user's actions when shaking the terminal device are usually larger. In other words, shaking the terminal device can be interpreted as tilting the terminal device more quickly and violently.

[0187] During the process of the user shaking the terminal device, the terminal device can not only simulate each interactive element bearing gravity and supporting force moving in the direction in which the terminal device is tilted, but also determine to display the motion effect of each interactive element according to the force of the user shaking the terminal device. There is a preset mapping relationship between the shaking force and the motion effect of the interactive element.

[0188] Taking the Emoji theme as an example, the mapping relationship between the shaking strength and the dynamic effect of the Emoji expression is shown in FIG. 7. As the user shakes the terminal device with increasing strength, the emotion of the Emoji expression gradually becomes angry. Exemplarily, the shaking strength can be divided into multiple interval ranges, and different interval ranges correspond to Emoji expressions with different emotions. For example, the shaking strength is divided into range 1-range 6 according to the order from small to large, and range 1-range 6 correspond to the Emoji expressions of “naughty”, “question”, “unhappy”, “dead tired”, “dizzy” and “angry” in turn.

[0189] Based on this, referring to (a)-(c) in FIG. 8, the terminal device displays multiple randomly arranged Emoji expressions at the bottom of the lock screen interface. In response to the user shaking the terminal device with a strength within range 5, and range 5 corresponding to the expression of “dizzy”, the terminal device displays each Emoji expression on the lock screen interface as the Emoji expression of “dizzy”.

[0190] Optionally, the theme can also be pre-installed with an Easter egg special effect (such as an Easter egg Emoji expression), and the Easter egg special effect is not selectable and editable on the custom editing page, i.e., the user cannot perceive the Easter egg special effect on the custom editing page. The Easter egg special effect enables the terminal device theme to have a lightweight hidden mechanism for the user to explore, which can bring greater surprises to the user. Exemplarily, when the terminal device detects that the shaking strength is within a specific interval, the terminal device displays the Easter egg special effect. In the process of displaying the Easter egg special effect, the terminal device can also simultaneously accompany with a ritual sense scene such as vibration, sound effect or firework celebration, to enhance the playability of the theme.

[0191] After introducing the user operation and interface display process in the terminal device shaking scenario, the control method of the terminal device for implementing the above process is described below.

[0192] FIG. 9 is a flowchart of a display method of a naked-eye 3D theme according to an embodiment of the present application. The method is applied to a terminal device provided with a naked-eye 3D theme, and specifically includes the following steps S901-S904.

[0193] S901, the terminal device displays at least one interactive element of a naked-eye 3D visual effect on a display interface.

[0194] Exemplarily, the interactive element can be a circular or other shaped interactive element. The display interface can be a desktop, a lock screen interface or an off-screen interface. The present embodiment does not limit this.

[0195] S902, the terminal device detects the posture information and acceleration information of itself.

[0196] In some embodiments, the terminal device detects its own attitude information using the gyroscope sensor 280B and detects acceleration information of the terminal device using the acceleration sensor 280E.

[0197] The sensor of the terminal device measures the corresponding parameters based on a sensor coordinate system. The sensor coordinate system is also referred to as a device coordinate system or a screen coordinate system. Referring to FIG. 10, for most terminal devices, the origin of the sensor coordinate system is the center of the screen of the terminal device, the X-axis points vertically from the left side to the right side of the terminal device, the Y-axis points vertically from the bottom to the top of the terminal device, and the Z-axis points upward perpendicular to the screen of the terminal device. It should be noted that the sensor coordinate system is a three-axis coordinate system used by the sensor framework of the terminal device to display sensor data. The directions of the three axes objectively exist and are set based on the natural direction of the terminal device and do not change with changes in the position or angle of the device.

[0198] Based on this, the attitude information of the terminal device can be represented by the direction angle a, the pitch angle b, and the roll angle g, i.e., (a, b, g). The direction angle a, the pitch angle b, and the roll angle g represent the angles of rotation of the terminal device around the Z-axis, the X-axis, and the Y-axis of the sensor coordinate system, respectively. The acceleration information of the terminal device detected by the acceleration sensor can be represented by (x, y, z), where x, y, and z are the three-axis accelerations along the X-axis, the Y-axis, and the Z-axis of the sensor coordinate system, respectively.

[0199] In S903, the terminal device simulates the simulated acceleration of the interactive element after being subjected to an external force according to its own attitude information and acceleration information.

[0200] In this embodiment, the terminal device obtains the attitude information from the gyroscope sensor 280B and the acceleration information from the acceleration sensor 280E through the parameter management module 334 of the theme engine 330. Subsequently, the dynamic effect and interaction control unit in the DSL parsing and execution module 331 simulates the simulated acceleration of the interactive element after being subjected to an external force according to the attitude information and the acceleration information.

[0201] In the shaking scenario, the external force simulated by the terminal device for the interactive element includes gravity, support force, and force applied to the interactive element when the user shakes the device. According to Newton's second law F = ma, when the interactive element is subjected to unbalanced external forces, the interactive element will move according to the simulated acceleration a = F / m, where F is the resultant force acting on the interactive element and m is the simulated mass of the interactive element. Therefore, the parameter that affects the motion of the interactive element is actually the simulated acceleration.

[0202] In the embodiments of the present application, the terminal device can simulate the simulated acceleration of the interactive element after the external force according to the device posture and the acceleration information detected by the acceleration sensor. For example, the acceleration is superimposed with the component of the gravitational acceleration along the direction of the terminal device screen tilt to obtain the simulated acceleration of the interactive element.

[0203] In S904, the terminal device displays the naked-eye 3D dynamic effect of the interactive element according to the simulated acceleration.

[0204] In the embodiments, the DSL parsing and execution module 331 of the terminal device generates the dynamic effect of the interactive element according to the simulated acceleration. The unified desktop module 320 displays the corresponding dynamic effect.

[0205] For example, if the acceleration of the interactive element is along the positive direction of the X axis, the terminal device moves the interactive element in the display interface along the positive direction of the X axis, that is, towards the right side of the terminal device, as shown in (b) of FIG. 6. If the acceleration of the interactive element is along the negative direction of the Y axis, the terminal device moves the interactive element in the display interface along the negative direction of the Y axis, that is, towards the bottom of the terminal device. The terminal device can display the rolling and collision scenes of the interactive element according to the simulated acceleration by using the theme engine shown in the foregoing, and display the scenes in the interface.

[0206] Optionally, the terminal device can control the motion direction of the interactive element according to the size of the simulated acceleration, and determine the shaking strength of the terminal device by the change frequency of the acceleration direction, wherein the higher the change frequency of the acceleration direction, the higher the shaking frequency and the greater the shaking strength of the terminal device. For example, the terminal device can display different emotional expressions according to the shaking strength of the user.

[0207] It should be noted that the terminal device will cyclically execute the above steps S902-S904 until the terminal device exits the current interface. The cyclic processing process can control the interactive element to respond to the operation of the user and improve the user experience.

[0208] (II) Gesture control scene

[0209] In the gesture control scene, the terminal device can respond to the gesture of the user in the air to control the interactive element in the display interface to display the corresponding interactive dynamic effect. For example, the target interactive element is selected according to the gesture in the air, or the target interactive element is controlled according to the gesture in the air.

[0210] In some embodiments, in response to the first air gesture of the user at the first position, the terminal device selects the first interactive element from the plurality of interactive elements of the current display interface and displays the first naked-eye 3D dynamic effect of the first interactive element; and in response to the first air gesture of the user at the second position, the terminal device deselects the first interactive element, selects the second interactive element from the plurality of interactive elements of the current display interface, and displays the naked-eye 3D dynamic effect of the second interactive element. The first position is different from the second position, and the first interactive element is different from the second interactive element.

[0211] In other words, when the user inputs the first air gesture, the terminal device triggers the element selection scenario. When the user moves the first air gesture against the screen, the terminal device switches the selected interactive element according to the position of the gesture movement.

[0212] In some other embodiments, in response to the first air gesture of the user, the terminal device randomly selects the first interactive element from the current display interface and displays the first naked-eye 3D dynamic effect of the first interactive element; and in response to the second air gesture of the user, the terminal device displays the second naked-eye 3D dynamic effect of the first interactive element.

[0213] In yet some embodiments, during the movement of the first interactive element, if there is an interaction between the first interactive element and a third interactive element, the third naked-eye 3D dynamic effect of the third interactive element is displayed during the movement of the first interactive element.

[0214] In some other embodiments, after selecting the first interactive element, the terminal device can also display the fourth naked-eye 3D dynamic effect of the interactive element other than the first interactive element.

[0215] In yet some embodiments, the terminal device can display the second naked-eye 3D dynamic effect of the first interactive element according to the gesture strength of the second air gesture. There is a preset mapping relationship between the gesture strength and the second naked-eye 3D dynamic effect.

[0216] Example 2-1:

[0217] Taking the current naked-eye 3D theme used by the terminal device as an Emoji theme for example, the terminal device displays a plurality of Emoji expressions arranged in an array in the screen-off interface. The styles of the Emoji expressions are different, but the sizes and the visual directions are the same. The terminal device can select and switch the target Emoji expression and interact with the target Emoji expression through the following process.

[0218] Referring to (a) of FIG. 11, in response to the user making a palm opening gesture (i.e., a first midair gesture) towards the lower half of the screen, the terminal device selects the emoji in the second column of the sixth row according to the position corresponding to the user's palm. After the terminal device selects the emoji, the terminal device displays the emoji in a floating and enlarged manner, and controls the eyes of other emojis to look at the selected emoji.

[0219] Referring to (b) of FIG. 11, in response to the user making a palm opening gesture (i.e., a first midair gesture) towards the middle of the screen, the terminal device selects the emoji in the second column of the fourth row according to the position corresponding to the user's palm. After the terminal device selects the emoji, the terminal device displays the emoji in a floating and enlarged manner, and controls the eyes of other emojis to look at the selected emoji.

[0220] It can be seen that as the user's first midair gesture moves upward, the emoji selected by the terminal device also switches upward. Of course, although not shown in the figure, as the user's first midair gesture moves downward / leftward / rightward, the emoji selected by the terminal device also switches downward / leftward / rightward.

[0221] Referring to (c) of FIG. 11, in response to the user making a palm downward patting gesture (i.e., a second midair gesture) based on the user having selected the emoji in the second column of the fourth row, the terminal device switches the emoji to another preset emoji, for example, an angry emoji. Alternatively, the terminal device can control the emoji to make a rotating, jumping or playing dynamic expression action.

[0222] Optionally, for the emoji theme, the terminal device can detect the force of the second midair gesture, and switch the selected emoji to an emoji corresponding to the force. For example, the terminal device can divide the control force of the second midair gesture into three levels of low, medium and high, and each level of force corresponds to two emojis, for example, the low level corresponds to "naughty" and "question" emojis, the medium level corresponds to "unhappy" and "dead tired" emojis, and the high level corresponds to "dizzy" to "angry" emojis. For the two emojis of each level, the terminal device can randomly select one to display.

[0223] Example 2-2:

[0224] Taking a basketball as an example, the naked-eye 3D theme currently used by the terminal device is basketball, as shown in (a) of FIG. 12, a plurality of randomly distributed basketballs are displayed at the bottom of the terminal device screen, and no basket is displayed. As shown in (b) of FIG. 12, in response to the user picking up the terminal device, or in response to the user tapping the terminal device screen, or in response to the terminal device detecting user facial information or gesture information, etc., the terminal device displays a full-screen screen-off interface, which includes a plurality of basketballs and a basket. In the screen-off interface, the terminal device can control the basketballs to interact with the user through the following process.

[0225] As shown in (a) of FIG. 13, in response to the user's gesture of opening the palm and facing the terminal device (i.e., the first air gesture), the terminal device randomly selects one of the basketballs displayed at the bottom and controls it to float in the air.

[0226] As shown in (b) of FIG. 13, in response to the user's gesture of slapping the palm downward (i.e., the second air gesture), the terminal device controls the selected basketball to shoot.

[0227] Optionally, during the movement of the first interactive element, if there is an interaction between the first interactive element and the third interactive element, a third naked-eye 3D dynamic effect of the third interactive element is displayed during the movement of the first interactive element. For example, during the process of the basketball entering the basket and falling, the terminal device can control the basketball net to perform a corresponding feedback dynamic effect. For example, the terminal device controls the basketball net to sway after the basketball falls; or the terminal device controls the basketball net to be deformed after being stretched by the falling basketball and then bounce back; or as shown in (a) and (b) of FIG. 14, the terminal device controls the basketball net to be stretched by the falling basketball and then bounce back, so that the dynamic effect of the shooting scene is more vivid and lifelike.

[0228] Optionally, the terminal device can also determine the distance of the basketball shot according to the strength of the user's palm swing downward (i.e., the strength of the second air gesture). For example, when the strength is large or small, the user cannot shoot the basketball into the basket; and when the strength of the shot is within a predetermined range, the user can shoot the basketball into the basket.

[0229] Optionally, the terminal device can also simulate gravity for the moving element, so that the movement process is closer to the real scene. For example, the terminal device can detect the gravity direction through the acceleration sensor 280E, and control the basketball to fall along the gravity direction after shooting. Through the basketball shooting scene, the gravity simulation can be presented in a more interesting way, which has a good user experience.

[0230] Optionally, the terminal device can also set a shooting Easter egg for the shooting scene, for example, after the user successfully shoots each time, or after successfully shooting multiple times in a row, the terminal device vibrates and / or displays corresponding cheers to improve the user experience.

[0231] Through the above control process, the throwing process of the ball can be smoother, and the satisfaction of the user during use can be improved.

[0232] It should be noted that the above only takes the shooting scene as an example to describe the process of controlling the display of the dynamic effect by the user through the midair gesture. Of course, in addition to the shooting scene, the above method is also applicable to other game interaction scenes such as playing badminton, playing tennis, playing volleyball, etc.

[0233] Example 2-3:

[0234] In some embodiments, in response to a midair gesture moving towards a first direction, the terminal device controls the interactive element in the display interface to turn towards the first direction; in response to a midair gesture moving towards a second direction, the terminal device controls the interactive element in the display interface to turn towards the second direction; wherein the first direction and the second direction are different.

[0235] For example, for a naked-eye 3D theme including an eye element, in response to a midair gesture moving towards a first direction by the user, the eyes of a plurality of naked-eye 3D elements in the naked-eye 3D theme look towards the first direction. In other words, the user can control the line-of-sight direction of the naked-eye 3D element through the gesture direction, and the control process is more interesting.

[0236] The terminal device displays an array of Emoji expressions in the screen-off interface, each Emoji expression has a different style, but the size and line-of-sight direction are the same. Referring to (a)-(b) in FIG. 15, when the user's palm is towards the screen, the Emoji expressions do not turn. In response to a gesture of the user's palm swinging to the left, each Emoji expression in the screen-off interface looks to the left of the illustrated direction. In addition, although not shown in the figure, in response to a gesture of the user's palm swinging to the right, each Emoji expression in the screen-off interface looks to the right of the illustrated direction. In response to a gesture of the user's palm swinging upwards, each Emoji expression in the screen-off interface looks upwards. In response to a gesture of the user's palm swinging downwards, each Emoji expression in the screen-off interface looks downwards.

[0237] Example 2-4:

[0238] In the process of the user selecting the first interactive element and controlling the first interactive element to display the second naked-eye 3D dynamic effect through the second midair gesture, the terminal device can also control the physical size (such as length, height, width, radius, etc.) or rotation speed of the first interactive element according to the distance of the second midair gesture. The second midair gesture is the same as or different from the first midair gesture. The second midair gesture includes a gesture changing the distance from the first interactive element, and the second naked-eye 3D dynamic effect includes a dynamic effect related to the physical size of the first interactive element or a dynamic effect related to the rotation speed of the first interactive element.

[0239] In some embodiments, the physical size of the first interactive element gradually increases as the second air-gesture gradually approaches the first interactive element, and gradually decreases as the second air-gesture gradually moves away from the first interactive element. For example, after the user selects an emoji by the first air-gesture, the radius of the emoji gradually increases as the user's palm gradually approaches the emoji (i.e., gradually approaches the screen), and gradually decreases as the user's palm gradually moves away from the emoji.

[0240] In some other embodiments, the rotation speed of the first interactive element gradually increases as the second air-gesture gradually approaches the first interactive element, and gradually decreases as the second air-gesture gradually moves away from the first interactive element.

[0241] In some embodiments, the terminal device receives user data collected by other electronic devices (e.g., a second electronic device) such as a smart bracelet or a heart rate detector, and displays a corresponding naked-eye 3D dynamic effect of the interactive element according to the user data. For example, when the interactive element in the naked-eye 3D theme is an expression element, the terminal device can detect the heart rate data of the user by the smart bracelet, and display a corresponding naked-eye 3D dynamic effect of the expression element according to the heart rate data of the user, wherein the heart rate data is related to the emotion of the expression element. For example, the faster the heart rate, the more excited the user's emotion, and the emotion of the expression element can be happy; the slower the heart rate, the more stable the user's emotion, and the emotion of the expression element can be expressionless.

[0242] By the method provided in the embodiments of the present application, the terminal device can collect more abundant user data by means of other electronic devices, so as to display expression dynamic effects that are more in line with the user's mood.

[0243] After introducing the user operation and interface display process in the gesture control scenario of the terminal device, the control method of the terminal device for implementing the above process is described in detail.

[0244] FIG. 16 is a flowchart of a display method of a shaking scenario of a naked-eye 3D theme according to an embodiment of the present application. The method is applied to a terminal device provided with a naked-eye 3D theme, and specifically includes the following steps S1601-S1604.

[0245] S1601, the terminal device displays at least one interactive element of a naked-eye 3D visual effect on a display interface.

[0246] For example, the interactive element can be a circular or other shaped interactive element. The display interface can be a desktop, a lock screen interface, or an off-screen interface. The present embodiment does not limit this.

[0247] S1602, the terminal device collects image information through the camera.

[0248] S1603, the terminal device identifies the collected image information to determine the related information of the mid-air gesture.

[0249] In this embodiment, the terminal device acquires image information from the camera module through the parameter management module 334 of the theme engine 330. Then, the operation and execution module 335 determines the related information of the mid-air gesture, such as the position, type and strength of the mid-air gesture, according to the image information.

[0250] In this embodiment, the terminal device can identify the collected image information by using the Swing algorithm to determine the related information of the mid-air gesture. For example, the related information of the mid-air gesture includes at least one of the type, strength and position of the mid-air gesture. According to the theme applied by the terminal device, the terminal device needs to determine different related information. For example, in the Emoji expression scene provided in the above example 2-1, the terminal device needs to determine the type, strength and position of the mid-air gesture. In the shooting scene shown in the above example 2-2, the terminal device only needs to determine the type and strength of the mid-air gesture, without paying attention to the position of the gesture. In the Emoji expression scene shown in the above example 2-3, the terminal device only needs to determine the type of the mid-air gesture, without paying attention to the position and strength of the gesture.

[0251] Regarding the determination of the type of the mid-air gesture:

[0252] In this embodiment, the terminal device uses an artificial intelligence (AI) model to identify the type of the user's mid-air gesture from the image information collected by the terminal device. The AI model is trained according to different sizes, user ages, shapes and palm images of the gesture, which helps to improve the accuracy of gesture recognition of the AI model.

[0253] Regarding the determination of the strength of the mid-air gesture:

[0254] During the movement of the gesture, the background of the palm can be considered as unchanged, and this part does not need special attention. Therefore, in this embodiment, the terminal device takes the palm as the analysis subject to identify the position of the gesture.

[0255] In some embodiments, the terminal device detects the image information collected using the background difference method, removes the background in the image that does not change, and leaves the changing objects (i.e., the palm) in the scene. The terminal device extracts the target object (i.e., the palm) in the first frame of image through the algorithm element, and then frames the region of the target object and records the position. The above process is repeated in the next frame of image, and finally the positions of different regions of the same object in the two frames of image are compared. The movement distance in pixels and frames can be converted into the corresponding acceleration and speed of the object. Different acceleration and speed correspond to different gesture strength, which can produce different interactive effects on the interactive element.

[0256] Regarding the determination of the position of the midair gesture:

[0257] In this embodiment, the positions of the user's palm that can be recognized by the camera are taken as planes, and 9 points where the user's palm is located directly above the screen of the device are collected, including the four vertices of the screen, the midpoints of the edges of the screen, and the center point of the screen. According to these points, a rectangular plane corresponding to the interactive element (such as an array of Emoji expressions) on the screen can be constructed, and the mapping relationship between the position of the gesture and the interactive element in the interface can be obtained.

[0258] S1604, the terminal device detects the gravity direction.

[0259] In this embodiment, the terminal device obtains the acceleration information from the acceleration sensor 280E through the parameter management module 334 of the theme engine 330, and determines the gravity direction according to the acceleration information.

[0260] It should be noted that S1604 is an optional step, i.e., the terminal device only needs to detect the gravity direction in a specific scenario. For example, the terminal device needs to detect the gravity direction in the shooting scenario.

[0261] S1605, the terminal device displays the naked-eye 3D dynamic effect of the interactive element according to the related information of the midair gesture and the gravity direction.

[0262] In this embodiment, the dynamic effect and interaction control unit in the DSL parsing and execution module 331 of the terminal device generates the dynamic effect of the interactive element according to the related information of the midair gesture and the gravity direction. The one-tablet desktop module 320 displays the corresponding dynamic effect.

[0263] For example, the terminal device first determines the dynamic effect of selecting and throwing the sphere according to the type, strength and gravity direction of the midair gesture. Then, the terminal device uses the theme engine to render and display the dynamic effect of selecting and throwing the sphere in the throwing stage. Finally, the terminal device uses the theme engine to render and display the dynamic effect of the falling sphere in the falling stage.

[0264] It should be noted that if the terminal device does not perform S1604, in S1605, the terminal device only needs to display the naked-eye 3D dynamic effect of the interactive element according to the related information of the mid-air gesture.

[0265] (III) Gaze tracking scenario

[0266] In the embodiments of the present application, for interactive elements with eyes, such as Emoji, cartoon characters, pets, and characters, etc., the terminal device can detect the angle at which the user gazes at the screen (i.e., the viewing angle) when displaying these interactive elements, and control the eyes of the interactive elements to rotate with the viewing angle of the user, i.e., the line of sight of the eyes of the interactive elements is always directed at the eyes of the user, or in other words, the eyes of the interactive elements are always looking at the user, thereby interacting with the user and increasing the entertainment and fun of using the terminal device for the user.

[0267] In some embodiments, in response to the user gazing at the display screen of the terminal device at a first viewing angle, the terminal device controls the interactive element in the first interface to rotate according to the first viewing angle; and in response to the user gazing at the display screen of the terminal device at a second viewing angle, the terminal device controls the interactive element in the first interface to rotate according to the second viewing angle; wherein the first viewing angle is different from the second viewing angle.

[0268] Taking the case that the naked-eye 3D theme applied by the terminal device is an Emoji theme and the display interface is a lock screen interface, when the user is within the camera shooting range of the terminal device, the terminal device can display the naked-eye 3D dynamic effect through the following process.

[0269] First, referring to FIG. 17, the terminal device displays a plurality of arrayed Emoji according to the user operation on the lock screen interface.

[0270] In some embodiments, when the terminal device is in an off-screen state or an off-screen state, in response to the user pressing the power button, the terminal device first displays an off-screen interface, and displays the naked-eye 3D element on the off-screen interface, for example, a part of the area of the display interface displays an arrayed Emoji. Subsequently, in response to the user clicking the screen, the terminal device displays the lock screen interface, and displays the corresponding naked-eye 3D element on the lock screen interface, for example, an arrayed Emoji in the entire area of the lock screen interface.

[0271] In some other embodiments, when the terminal device is in an off-screen state or an off-screen state, in response to the user standing up the terminal device from a horizontal state, or in response to the user gazing at the screen of the terminal device, or in response to the user pressing the power button of the terminal device, the terminal device enters the lock screen interface, and displays the corresponding naked-eye 3D element on the lock screen interface, for example, an arrayed Emoji in the entire area of the lock screen interface.

[0272] Subsequently, the user approaches the terminal device screen so that the face is within a preset range from the terminal device. Exemplarily, the preset range can be within 50 cm, within 30 cm, etc.

[0273] Then, the terminal device displays the view angle of the Emoji according to the view angle of the user. Exemplarily, referring to (a) in FIG. 18, when the user gazes at the screen from the bottom of the terminal device, the view of the Emoji turns to the bottom of the terminal device and looks at the user. Referring to (b) in FIG. 18, when the user gazes at the screen from the top of the terminal device, the view of the Emoji turns to the top of the terminal device and looks at the user. Referring to (c) in FIG. 18, when the user gazes at the screen from the left side of the terminal device, the view of the Emoji turns to the left side of the terminal device and looks at the user. Referring to (d) in FIG. 18, when the user gazes at the screen from the right side of the terminal device, the view of the Emoji turns to the right side of the terminal device and looks at the user.

[0274] In the above process, the user can keep the pose of the terminal device unchanged and adjust the view angle of gazing at the screen by changing the pose of the user; or keep the position of the user unchanged and adjust the view angle of gazing at the screen by changing the pose of the terminal device; or change the pose of the user and the terminal device simultaneously to adjust the view angle of gazing at the screen.

[0275] In summary, when the user is within the shooting range of the camera of the terminal device, the terminal device can detect the view angle of the user and control the view angle of the Emoji to gradually move with the view angle of the user, so that the Emoji always looks at the eyes of the user, that is, always looks at the user, which has strong interactivity and interestingness.

[0276] In addition, when the user exits the shooting range of the camera of the terminal device, the terminal device cannot interact with the user according to the view angle of the user because the view angle of the user cannot be detected. In this case, the terminal device displays the naked-eye 3D dynamic effect in the preset view angle in the lock screen interface. For example, the terminal device displays the naked-eye 3D dynamic effect in the view effect of the vertical screen.

[0277] After introducing the user operation and interface display process in the shaking scenario of the terminal device, the control method of the terminal device for implementing the above process is described below.

[0278] FIG. 19 is a schematic flowchart of displaying a naked-eye 3D theme in a view tracking scenario according to an embodiment of the present application. The method includes the following steps S1901-S1905.

[0279] S1901, the terminal device displays at least one interactive element of a naked-eye 3D view effect in a display interface.

[0280] Exemplarily, the interactive element can be a circular or other shaped interactive element. The display interface can be a desktop, a lock screen interface, or an off screen interface. The embodiments are not limited thereto.

[0281] S1902, the terminal device collects image information through a camera.

[0282] S1903, the terminal device determines a visual angle of the user gazing at the screen of the terminal device according to the collected image information.

[0283] In the embodiment, the terminal device acquires the image information from the camera module through the parameter management module 334 of the theme engine 330. Then, the visual angle of the user gazing at the screen of the terminal device is determined according to the image information by the operation and execution module 33.

[0284] In the embodiment, the terminal device detecting the visual angle of the user includes two parts: (1) eye region detection, and (2) determining the visual angle of the user based on the eye region. Details are shown as follows.

[0285] (1) Eye region detection:

[0286] Firstly, the terminal device performs histogram transformation on the collected user image to obtain a processed gray image. Exemplarily, the terminal device can determine a neighborhood (a rectangular region) in the user image, and then performs histogram transformation on the user image gradually by moving the center of the neighborhood. For example, the center of the neighborhood is moved one pixel at a time along the image from left to right and from top to bottom, and at each pixel position, the histogram transformation is performed on the neighborhood of the pixel point. Finally, the center point of the neighborhood is processed in gray, and the processed gray image is obtained. This method can weaken the influence of different light environments on image recognition and restore the dark details in the original image that cannot be seen due to light factors.

[0287] Then, the terminal device extracts facial features on the processed gray image by using active shape models (ASM) local sensitive histogram processing to determine the region where the human eye is located. It should be noted that after the terminal device recognizes the human eye, the terminal device marks the eye feature in the image and puts the marked image as a new shape into the training set for training.

[0288] Through the above process, even in the case of uncontrolled light and free movement of the head, the terminal device can accurately recognize the eye region.

[0289] (2) Determining the visual angle of the user based on the eye region

[0290] In this embodiment, the terminal device estimates the gaze direction by the iris center and the inner corner of the eye in the detected eye region. Because the inner corner of the eye is not sensitive to facial expression and eye state changes, the inner corner of the eye is usually regarded as the reference point for gaze estimation, and the gaze direction determined based on the inner corner of the eye is also more accurate.

[0291] In some embodiments, the terminal device can detect the eye region for iris detection. For example, the terminal device first estimates the iris radius, and then determines the position of the iris center in combination with the picture energy intensity and edge information.

[0292] In some embodiments, the terminal device identifies the inner corner of the eye through a trained neural network model.

[0293] The terminal device determines the vector from the iris center to the inner corner of the eye, and maps the vector and the distance from the face to the terminal device in the sensor coordinate system, so that the eye screen viewing angle conversion can be realized, and the viewing angle of the human eye gazing at the screen of the terminal device can be determined.

[0294] S1904, the terminal device displays the naked-eye 3D dynamic effect of the interactive element according to the viewing angle of the user gazing at the screen of the terminal device.

[0295] For example, the terminal device renders and displays each interactive element in the lock screen interface according to the viewing angle of the user gazing at the screen of the terminal device through the theme engine.

[0296] In summary, the display method of the naked-eye 3D theme provided in this embodiment can detect the viewing angle of the user gazing at the screen, and control the eyes of the interactive elements in the theme to always look at the eyes of the user, that is, always make eye contact with the user, which has strong interactivity and interestingness.

[0297] (Four) Touch control scene

[0298] In some embodiments, in response to the touch operation of the user on the target interactive element, the terminal device displays a first naked-eye 3D dynamic effect of the interactive element, and controls other elements except the target element to display a second naked-eye 3D dynamic effect. Wherein, the first naked-eye 3D dynamic effect is different from the second naked-eye 3D dynamic effect.

[0299] Taking the current naked-eye 3D theme used by the terminal device as an example, the Emoji theme, the terminal device can control the Emoji expression to interact with the user through the following process.

[0300] Referring to (a) shown in FIG. 20, the lock screen interface of the terminal device displays an array of Emoji expressions, each Emoji expression has a different style, but the size and the gaze direction are the same.

[0301] Referring to (b) of FIG. 20, in response to the user clicking the emoji in the second row and the second column, the terminal device switches the emoji to a preset emoji (e.g., an angry emoji). Meanwhile, the styles of other emojis remain unchanged, but the eyes of all the emojis look at the emoji.

[0302] Referring to (c) of FIG. 20, in response to the user clicking the emoji in the third row and the first column, the terminal device switches the emoji to a preset emoji (e.g., an angry emoji). Meanwhile, the styles of other emojis remain unchanged, but the eyes of all the emojis look at the emoji.

[0303] Referring to (d) of FIG. 20, in response to the user clicking the emoji in the fourth row and the third column, the terminal device switches the emoji to a preset emoji (e.g., an angry emoji). Meanwhile, the styles of other emojis remain unchanged, but the eyes of all the emojis look at the emoji.

[0304] By the method provided in the embodiments of the present application, the user can have interesting interaction with each element in the naked-eye 3D theme through a touch operation, and has a better user experience.

[0305] In some embodiments, the terminal device can also display and transform the interactive element according to the user operation in the screen-off interface. For example, as shown in FIG. 21, the screen-off interface of the terminal device is partially lighted and displays a plurality of emojis. In response to the user touching the screen-off interface, or in response to the user picking up the terminal device, the screen-off interface of the terminal device is fully lighted and displays the emojis. Alternatively, as shown in FIG. 22, the screen-off interface of the terminal device is fully lighted and displays a plurality of basketballs on the left side and the right side. When the user does not operate the terminal device for a long time, or when the user presses the power key, the upper half of the screen-off interface is gradually extinguished, and the basketballs on the left side and the right side gradually fall to the bottom of the screen-off interface.

[0306] (Five) Folding device unfolding scene

[0307] In some embodiments, when the display method of the naked-eye 3D theme provided in the embodiments is applied to a folding device, when the folding device is in a folded state, the first screen of the folding device is used to display a user interface. In this case, in response to an operation of unfolding the folding device, the terminal device switches from the first screen to the second screen to display the naked-eye 3D dynamic effect of the interactive element. The first screen is a display screen used by the terminal device in the folded state, and the second screen is a display screen used by the terminal device in the unfolded state.

[0308] The folding device can display corresponding naked-eye 3D dynamic effects in the unfolded screen according to its unfolding type.

[0309] For example, referring to (a)-(e) in FIG. 23, for a folding device unfolded upward, interactive elements (such as Emoji, basketball, etc.) gradually fall from the top of the screen to the folding position during the folding device is folded to half unfolded. During the folding device is unfolded from half unfolded to full unfolded, the interactive elements gradually fall from the folding position to the bottom of the screen.

[0310] For example, referring to (a)-(c) in FIG. 24, for a folding device unfolded leftward, interactive elements (such as Emoji, basketball, etc.) gradually collide from the upper regions of the two sides of the screen and fall to the bottom of the screen.

[0311] For example, for a folding device unfolded leftward, interactive elements (such as Emoji, basketball, etc.) gradually collide from the upper regions of the two sides of the screen and fall to the bottom of the screen.

[0312] The method provided in the embodiment can control the interaction between the interactive elements and the user during the unfolding of the folding device, so that the unfolding process of the folding device is more interesting.

[0313] Based on the same concept, as an implementation of the above method, the embodiment of the present application provides a display device. The device embodiment corresponds to the foregoing method embodiment, and for the sake of reading, the details of the foregoing method embodiment will not be described one by one. However, it should be clear that the device in the embodiment can correspondingly implement all the contents of the foregoing method embodiment.

[0314] FIG. 25 is a schematic diagram of an information sharing device provided in an embodiment of the present application. The display device 2500 is applied to a terminal device, and the terminal device applies a naked-eye 3D theme. The display device 2500 includes a first display module 2501 and a second display module 2502. The functions of the modules are as follows.

[0315] The first display module 2501 is configured to display N interactive elements in the naked-eye 3D theme on a first interface, where N is an integer greater than or equal to 1.

[0316] The second display module 2502 is configured to display a naked-eye 3D dynamic effect corresponding to at least one interactive element on the first interface in response to a control operation, where the naked-eye 3D dynamic effect is generated according to the control operation. The process of displaying the naked-eye 3D dynamic effect corresponding to at least one interactive element on the first interface in response to the control operation is described above and will not be described here.

[0317] It should be understood that the size of the serial number of each step in the above embodiments does not mean the order of execution, and the execution order of each process should be determined according to its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0318] The embodiments of the present application further provide an electronic device, which comprises a display configured to execute the display method of the naked-eye 3D theme shown in each of the above embodiments.

[0319] The embodiments of the present application further provide a chip, which comprises a processor and a memory, and the memory stores a computer program, and the computer program is executed by the processor to realize the display method of the naked-eye 3D theme in each of the above embodiments.

[0320] The embodiments of the present application further provide a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to realize the display method of the naked-eye 3D theme provided in each of the above embodiments.

[0321] The embodiments of the present application further provide a computer program product, which comprises a computer program, and when the computer program is run by an electronic device, the electronic device realizes the display method of the naked-eye 3D theme provided in each of the above embodiments.

[0322] It should be understood that the processor mentioned in the embodiments of the present application can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0323] It should also be understood that the memory mentioned in the embodiments of the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically EPROM (EEPROM) or a flash memory. The volatile memory can be a random access memory (RAM) used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchlink DRAM (SLDRAM) and direct rambus RAM (DR RAM).

[0324] In the embodiments provided by the present application, the division of each framework or module is only a logical function division, and another division mode can be used in actual implementation, for example, a plurality of frameworks or modules can be combined or integrated into another system, or some features can be ignored or not executed. In addition, each function module in each embodiment of the present application can be integrated in one processing module, or each module can exist physically, or two or more modules can be integrated in one module. The integrated module can be realized in the form of hardware or in the form of a software function module.

[0325] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the system, device and unit described above can refer to the corresponding process in the foregoing method embodiments, which will not be described here.

[0326] Reference throughout this specification to "one embodiment" or "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the application. Thus, the appearances of the phrases "in one embodiment" or "in an embodiment" or "in a various embodiment" or "in some embodiments" in various places throughout this specification are not necessarily referring to the same embodiment, but can, but not necessarily, refer to different embodiments. The terms "including," "comprising," "having" and variations thereof are meant to encompass the items listed thereafter, but do not exclude other items from also being present. The term "consisting of" is meant to exclude any item not specified, but "consisting essentially of" does not exclude other items that do not "materially" affect the essential characteristics of the application.

[0327] The above-described embodiments are merely intended to illustrate the technical solutions of the present application, but not to limit the same; even though the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: they can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacements to some technical features therein; and these modifications or replacements do not make the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A display method of a naked-eye 3D theme, characterized by, The method is applied to a terminal device, and comprises the following steps: displaying N interactive elements in a naked-eye 3D theme in a first interface, N≥1 and being an integer, the first interface comprising a desktop, a lock screen interface or an off-screen interface of the terminal device; in response to a control operation, displaying a naked-eye 3D dynamic effect corresponding to at least one of the interactive elements in the first interface; wherein the naked-eye 3D dynamic effect is generated according to the control operation.

2. The method of claim 1, wherein, The step of displaying the naked-eye 3D dynamic effect corresponding to at least one of the interactive elements in the first interface in response to the control operation comprises the following steps: in response to a first hovering gesture at a first position, selecting a first interactive element from the N interactive elements and displaying a first naked-eye 3D dynamic effect of the first interactive element; in response to the first hovering gesture at a second position, deselecting the first interactive element from the N interactive elements, selecting a second interactive element, and displaying a naked-eye 3D dynamic effect of the second interactive element; wherein the first position is different from the second position, and the first interactive element is different from the second interactive element.

3. The method of claim 1, wherein, The step of displaying the naked-eye 3D dynamic effect corresponding to at least one of the interactive elements in the first interface in response to the control operation comprises the following steps: in response to a first hovering gesture, randomly selecting a first interactive element from the N interactive elements and displaying a first naked-eye 3D dynamic effect of the first interactive element.

4. The method according to claim 2 or 3, characterized in that, In the case that the first interactive element is selected, the method further comprises the following steps: in response to a second hovering gesture, displaying a second naked-eye 3D dynamic effect of the first interactive element in the first interface.

5. The method of claim 4, wherein, The second hovering gesture comprises a gesture with a distance change from the first interactive element, and the second naked-eye 3D dynamic effect comprises a dynamic effect related to a physical size of the first interactive element or a dynamic effect related to a rotation speed of the first interactive element.

6. The method of claim 5, wherein, The step of displaying the second naked-eye 3D dynamic effect of the first interactive element in the first interface in response to the second hovering gesture comprises the following steps: determining a gesture strength of the second hovering gesture; displaying the second naked-eye 3D dynamic effect of the first interactive element in the first interface according to the gesture strength.

7. The method according to any one of claims 4 to 6, characterized in that, The first hovering gesture is the same as the second hovering gesture.

8. The method according to any one of claims 4 to 7, characterized in that, The method further comprises the following steps: in the second naked-eye 3D dynamic effect, if there is an interaction between the first interactive element and a third interactive element, a third naked-eye 3D dynamic effect corresponding to the third interactive element is displayed, and the first naked-eye 3D dynamic effect is different from the third naked-eye 3D dynamic effect.

9. The method according to any one of claims 2 to 8, characterized in that, The method further comprises the following steps: in response to the first hovering gesture, displaying a fourth naked-eye 3D dynamic effect of other interactive elements except the first interactive element in the first interface, and the first naked-eye 3D dynamic effect is different from the fourth naked-eye 3D dynamic effect; wherein in the other interactive elements, naked-eye 3D dynamic effects of at least two interactive elements are the same.

10. The method of claim 1, wherein, The step of displaying the naked-eye 3D dynamic effect corresponding to at least one of the interactive elements in the first interface in response to the control operation comprises the following steps: in response to a hovering gesture moving towards a first direction, controlling the interactive element to rotate towards the first direction; in response to a hovering gesture moving towards a second direction, controlling the interactive element to rotate towards the second direction; The first direction and the second direction are different.

11. The method of claim 1, wherein, The response to the control operation includes: The interactive element rotates according to the first visual angle in response to the user gazing at the display screen of the terminal device at a first visual angle. The interactive element rotates according to the second visual angle in response to the user gazing at the display screen of the terminal device at a second visual angle.

12. The method of claim 1, wherein, The response to the control operation includes: In response to the terminal device tilting to a first direction under the control operation, the interactive element moves to the first direction on the first interface. The posture of the terminal device tilting to the first direction is determined by data collected by a gyroscope sensor.

13. The method of claim 1, wherein, The response to the control operation includes: According to the shaking strength of the terminal device, the naked-eye 3D dynamic effect corresponding to the interactive element is displayed. The shaking strength is determined according to acceleration data collected by an acceleration sensor, and there is a mapping relationship between the shaking strength and the naked-eye 3D dynamic effect.

14. The method of claim 13, wherein, The interactive element in the naked-eye 3D theme is an expression element, and the display of the corresponding naked-eye 3D dynamic effect of the interactive element according to the shaking strength of the terminal device includes: According to the shaking strength, the naked-eye 3D dynamic effect corresponding to the emotion of the expression element is displayed.

15. The method of claim 1, wherein, The response to the control operation includes: User data collected by a second electronic device is received. According to the user data, the corresponding naked-eye 3D dynamic effect of the interactive element is displayed.

16. The method of claim 15, wherein, The interactive element in the naked-eye 3D theme is an expression element, and the user data includes heart rate data of the user. According to the heart rate data, the naked-eye 3D dynamic effect corresponding to the expression element is displayed.

17. The method of claim 1, wherein, The response to the control operation includes: In response to a touch operation on a target interactive element, the target element displays a first naked-eye 3D dynamic effect, and other interactive elements except the target element display a second naked-eye 3D dynamic effect, the first naked-eye 3D dynamic effect being different from the second naked-eye 3D dynamic effect.

18. The method of claim 1, wherein, The terminal device is a folding device, and the method further includes: In response to an operation of unfolding the folding device, the naked-eye 3D dynamic effect of the interactive element is displayed by switching from a first screen to a second screen.

19. An electronic device, comprising: The computer program stored in the memory and executable on the processor includes the method of any one of claims 1-18. The computer program stored in the memory and executable on the processor includes the method of any one of claims 1-18.

20. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the method in any one of claims 1-18.

21. A chip, characterized by The chip comprises a processor and a memory, and the memory stores a computer program, and the computer program is executed by the processor to implement the method in any one of claims 1-18.

Images