Light and shadow effect display method and electronic equipment

Abstract

The invention provides a light and shadow effect display method and electronic equipment, and relates to the technical field of terminals. According to the method and the device, the light and shadow effect display can be adaptively realized based on the preset illumination information in combination with the current display information, the interface light and shadow effect is enriched, and the use experience of a user is improved. The method comprises the steps that first operation of a user is detected, and the electronic equipment determines to refresh a first interface which is being displayed; the electronic device obtains illumination information and current display information, wherein the illumination information is a pre-configured global illumination definition. And the electronic equipment performs illumination rendering according to the illumination information and the current display information. Afterwards, the electronic device displays a second interface, and the first component in the second interface has a light and shadow effect formed after illumination rendering.

Description

[0001] This application is a divisional application. The application number of the original application is 202310963290.9, and the original application date is July 31, 2023. The entire contents of the original application are incorporated into this application by reference. Technical Field

[0002] The embodiments of the present application relate to the field of terminal technology, and in particular to a light and shadow effect display method and an electronic device. Background Art

[0003] With the development of terminal technology, virtual light sources are pre-set in electronic devices and some presentation methods in the real physical world are introduced to achieve the light and shadow display effect of the graphical user interface (GUI) of electronic devices. For example, based on the preset virtual light source, the control shadow is displayed according to the Z-axis altitude of the control, thereby enriching the display effect and improving the user experience.

[0004] However, the current light and shadow display effects are limited to some controls with simple shapes. For example, shadow display only supports circular controls and rounded rectangular controls, and cannot achieve better GUI light and shadow effects. Summary of the invention

[0005] In order to solve the above technical problems, the present application provides a method and electronic device for displaying light and shadow effects. The technical solution provided by the present application realizes the display of light and shadow effects adaptively based on preset lighting information and current display information, enriches GUI light and shadow effects, and improves the user experience.

[0006] In order to achieve the above technical objectives, this application provides the following technical solutions:

[0007] In a first aspect, a method for displaying light and shadow effects is provided, which is applied to an electronic device. The method includes: detecting a first operation of a user, and determining to refresh a first interface being displayed. Obtaining illumination information and current display information, where the illumination information is a preconfigured global illumination definition. Performing illumination rendering according to the illumination information and the current display information. Displaying a second interface, where a first component in the second interface has a light and shadow effect formed after illumination rendering.

[0008] In this way, the electronic device combines the lighting information and the current display information to provide the user with an interface lighting effect that better meets the user's needs, thereby improving the user's experience.

[0009] According to the first aspect, the global illumination definition is used to indicate one or more of the physical illumination principles, illumination types, illumination properties, scene illumination effects, illumination models, illumination materials, and shadow properties in the real world.

[0010] According to the first aspect, or any implementation method of the first aspect above, the current display information includes one or more of the electronic device's current time information, location information, motion status information, interface virtual light source information, component height information, component material information, and component transparency information.

[0011] According to the first aspect, or any implementation method of the first aspect above, the rendering process of the first component includes border rendering, lighting layer rendering, user interface canvas rendering, surface layer rendering, and shadow bearing layer rendering; wherein, the border rendering is used to draw at least one of the self-luminous edge effect, highlight edge effect, and contour line effect of the first component; the lighting layer is used to receive lighting and complete the lighting drawing corresponding to the first component; the user interface canvas is used to complete the drawing of the style content of the first component; the surface layer is used to draw the display container of the first component; the shadow bearing layer is used to carry the layer shadow and the overflow lighting drawing.

[0012] According to the first aspect, or any implementation method of the first aspect above, lighting rendering is performed according to the lighting information and the current display information, including: outputting visual features according to the light source information, rendering equations and basic physical properties indicated by the lighting information and the current display information, the visual features including one or more of color information, projection information and physical material information.

[0013] According to the first aspect, or any implementation of the first aspect above, the light source information includes the lighting type and lighting properties; the basic physical properties include one or more of the base color, metallicity, roughness, reflectivity, self-luminescence, normal information, refractive index, transparency, absorptivity, and transmittance.

[0014] According to the first aspect, or any implementation of the first aspect above, the light and shadow effects include a shadow rendering effect, and the shadow rendering effect corresponds to one or more of the height of the first component, the position of the light source, the shape of the first component, and the material of the first component, and the shape of the first component is a regular shape or an irregular shape.

[0015] For example, the shadow rendering effect is related to the height of the first component. The shadow of the first component includes the umbra and penumbra. The higher the height of the first component and the closer it is to the light source, the larger its penumbra is, and the more obvious the shadow display effect is.

[0016] For another example, the shadow rendering effect is related to the position of the light source. For example, if the light source is located at the upper right of the first component, the shadow is located at the lower left of the first component. For another example, if the light source is located directly above the first component, the shadow is located directly below the first component. If the light source is located at the upper left of the first component, the shadow is located at the lower right of the first component.

[0017] For another example, the shadow rendering effect is related to the shape of the first component. The electronic device can output shadow display effects of different shapes according to the different shapes of the first component. In addition to supporting shadow rendering of circles and rounded rectangles, the electronic device also supports rendering of various special-shaped shadows.

[0018] For another example, the shadow rendering effect is related to the material of the first component. For example, if the translucent material supports color projection, then the shadow color is similar to the color of the first component of the translucent material. For example, the color of the first component to be rendered by the electronic device is blue, and the material of the first component is a translucent material. Then, the electronic device can draw a light blue shadow based on the color and material of the first component, so that the shadow display effect is more consistent with the shadow effect in the real physical world, so as to enhance the user experience.

[0019] In this way, electronic devices can bring different shadow rendering effects according to different light sources and physical properties, enrich the GUI interface display, and enhance the user experience.

[0020] In addition, when the shape of the first component is relatively complex, the electronic device may also combine the lighting information and the current display information to generate the light and shadow effects of the first component of the complex shape to enrich the display of the interface.

[0021] According to the first aspect, or any implementation of the first aspect above, performing illumination rendering according to illumination information and current display information includes: determining a shadow offset and direction according to the light source coordinates indicated by the illumination information and the current display information and the height of the first component, and obtaining a shadow rendering effect. Alternatively, copying the first component and creating a shadow map. Obtaining a shadow rendering effect by blurring the shadow map. Alternatively, obtaining a shadow rendering effect by a shadow rendering algorithm according to the illumination information and the current display information.

[0022] According to the first aspect, or any implementation of the first aspect above, when the material of the first component is a translucent material, the light and shadow effects include one or more of frosted glass effect, rainbow glass effect, and acrylic glass effect.

[0023] In this way, when the first component is made of a translucent material, the electronic device can also combine the lighting information and the current display information to generate the light and shadow effects of the first component of the translucent material to enrich the display of the interface.

[0024] In a second aspect, an electronic device is provided. The electronic device includes: a processor, a memory and a display screen, the memory and the display screen are coupled to the processor, the memory is used to store computer program code, the computer program code includes computer instructions, when the processor reads the computer instructions from the memory, the electronic device executes: detecting a first operation of the user, determining to refresh the first interface being displayed. Obtaining lighting information and current display information, the lighting information is a preconfigured global lighting definition. Performing lighting rendering according to the lighting information and the current display information. Displaying a second interface, the first component in the second interface has a light and shadow effect formed after lighting rendering.

[0025] According to the second aspect, the global illumination definition is used to indicate one or more of the physical illumination principles, illumination types, illumination properties, scene illumination effects, illumination models, illumination materials, and shadow properties in the real world.

[0026] According to the second aspect, or any implementation method of the second aspect above, the current display information includes one or more of the electronic device's current time information, location information, motion status information, interface virtual light source information, component height information, component material information, and component transparency information.

[0027] According to the second aspect, or any implementation method of the second aspect above, the rendering process of the first component includes border rendering, lighting layer rendering, user interface canvas rendering, surface layer rendering, and shadow bearing layer rendering; wherein, the border rendering is used to draw at least one of the self-luminous edge effect, highlight edge effect, and contour line effect of the first component; the lighting layer is used to receive lighting and complete the lighting drawing corresponding to the first component; the user interface canvas is used to complete the drawing of the style content of the first component; the surface layer is used to draw the display container of the first component; the shadow bearing layer is used to carry the layer shadow and the overflow lighting drawing.

[0028] According to the second aspect, or any implementation method of the second aspect above, lighting rendering is performed according to the lighting information and the current display information, including: outputting visual features according to the light source information, rendering equations and basic physical properties indicated by the lighting information and the current display information, the visual features including one or more of color information, projection information and physical material information.

[0029] According to the second aspect, or any implementation of the second aspect above, the light source information includes the lighting type and lighting properties; the basic physical properties include one or more of the base color, metallicity, roughness, reflectivity, self-luminescence, normal information, refractive index, transparency, absorptivity, and transmittance.

[0030] According to the second aspect, or any implementation of the second aspect above, the light and shadow effects include a shadow rendering effect, and the shadow rendering effect corresponds to one or more of the height of the first component, the position of the light source, the shape of the first component, and the material of the first component, and the shape of the first component is a regular shape or an irregular shape.

[0031] According to the second aspect, or any implementation of the second aspect above, according to the illumination information and the current display information, illumination rendering is performed, including: determining the shadow offset and direction according to the light source coordinates indicated by the illumination information and the current display information and the height of the first component, and obtaining the shadow rendering effect. Alternatively, the first component is copied to create a shadow map. The shadow map is blurred to obtain the shadow rendering effect. Alternatively, according to the illumination information and the current display information, the shadow rendering effect is obtained through a shadow rendering algorithm.

[0032] According to the second aspect, or any implementation of the second aspect above, when the material of the first component is a translucent material, the light and shadow effects include one or more of frosted glass effect, rainbow glass effect, and acrylic glass effect.

[0033] The technical effects corresponding to the second aspect and any implementation method of the second aspect can be found in the technical effects corresponding to the above-mentioned first aspect and any implementation method of the first aspect, and will not be repeated here.

[0034] In a third aspect, an electronic device is provided, which has the function of implementing the light and shadow effect display method described in the first aspect and any possible implementation thereof. The function can be implemented by hardware, or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions.

[0035] In a fourth aspect, a computer-readable storage medium is provided. The computer-readable storage medium stores a computer program (also referred to as an instruction or code), and when the computer program is executed by an electronic device, the electronic device executes the method of the first aspect or any one of the implementations of the first aspect.

[0036] According to a fifth aspect, a computer program product is provided. When the computer program product is run on an electronic device, the electronic device executes the method according to the first aspect or any one of the implementations of the first aspect.

[0037] According to a sixth aspect, a circuit system is provided, the circuit system comprising a processing circuit, wherein the processing circuit is configured to execute the method according to the first aspect or any one of the embodiments of the first aspect.

[0038] In the seventh aspect, a chip system is provided, comprising at least one processor and at least one interface circuit, wherein the at least one interface circuit is used to perform transceiver functions and send instructions to the at least one processor, and when the at least one processor executes the instructions, the at least one processor executes the method of the first aspect or any one of the embodiments of the first aspect.

[0039] The technical effects of the aforementioned aspects can be referenced to each other and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 Schematic diagram of the light and shadow display scene provided in the embodiment of the present application Figure 1 ;

[0041] Figure 2 Schematic diagram of the light and shadow display scene provided in the embodiment of the present application Figure 2 ;

[0042] Figure 3 Schematic diagram of the light and shadow display scene provided in the embodiment of the present application Figure 3 ;

[0043] Figure 4 A schematic diagram of the electronic device provided in the embodiment of the present application;

[0044] Figure 5 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application;

[0045] Figure 6 The software structure diagram provided in this application embodiment is as follows: Figure 1 ;

[0046] Figure 7 The software structure diagram provided in this application embodiment is as follows: Figure 2 ;

[0047] Figure 8 Schematic diagram of the rendering process provided in the embodiment of the present application Figure 1 ;

[0048] Fig. 9 Schematic diagram of the rendering process provided in the embodiment of the present application Figure 2 ;

[0049] Fig.10 A schematic diagram of a light and shadow effect display processing flow provided in an embodiment of the present application;

[0050] Fig.11 Schematic diagram of the light and shadow effect display provided in the embodiment of the present application Figure 1 ;

[0051] Fig.12 Schematic diagram of the light and shadow effect display provided in the embodiment of the present application Figure 2 ;

[0052] Fig.13 Schematic diagram of the light and shadow effect display provided in the embodiment of the present application Figure 3 ;

[0053] Fig.14 A schematic diagram of the basic physical principle of light and shadow effects provided in the embodiments of the present application;

[0054] Fig.15 A schematic diagram of light and shadow effects of different types of light sources provided in an embodiment of the present application;

[0055] Fig.16 A schematic diagram of the UI framework structure provided in an embodiment of the present application;

[0056] Fig.17 A schematic diagram showing the metalness, normal, and roughness provided in an embodiment of the present application;

[0057] Fig.18 A schematic diagram of a scene automatically generated by normal information provided in an embodiment of the present application;

[0058] Fig.19 A schematic diagram of a refractive index change scenario provided in an embodiment of the present application;

[0059] Fig. 20 A schematic diagram of a change in absorption rate provided in an embodiment of the present application;

[0060] Fig.21 A schematic diagram of a light transmittance change scenario provided in an embodiment of the present application;

[0061] Fig. 22 A schematic diagram of a thickness change scenario provided in an embodiment of the present application;

[0062] Fig.23 A schematic diagram of the PBR lighting model principle provided in the embodiment of the present application;

[0063] Fig.24 A schematic diagram of the principle of a ray tracing rendering model provided in an embodiment of the present application;

[0064] Fig.25 A schematic diagram of the principle of the semi-transparent material rendering equation provided in the embodiment of the present application;

[0065] Fig.26 Schematic diagram of the shadow effect provided in the embodiment of the present application Figure 1 ;

[0066] Fig. 27 Schematic diagram of the shadow effect provided in the embodiment of the present application Figure 2 ;

[0067] Fig.28Schematic diagram of the shadow effect provided in the embodiment of the present application Figure 3 ;

[0068] Fig.29 Schematic diagram of the shadow effect provided in the embodiment of the present application Figure 4 ;

[0069] Fig.30 Schematic diagram of the shadow effect provided in the embodiment of the present application Figure 5 ;

[0070] Fig.31 Schematic diagram of the shadow simulation principle provided in the embodiment of the present application Figure 1 ;

[0071] Fig.32 Schematic diagram of the shadow simulation principle provided in the embodiment of the present application Figure 2 ;

[0072] Fig.33 Schematic diagram of the shadow effect generation principle provided in the embodiment of the present application Figure 1 ;

[0073] Fig.34 Schematic diagram of the shadow effect generation principle provided in the embodiment of the present application Figure 2 ;

[0074] Fig.35 Schematic diagram of the shadow effect generation principle provided in the embodiment of the present application Figure 3 ;

[0075] Fig.36 Schematic diagram of the Changhong glass principle provided in the embodiment of this application Figure 1 ;

[0076] Fig.37 Schematic diagram of the Changhong glass principle provided in the embodiment of this application Figure 2 ;

[0077] Fig.38 Schematic diagram of frosted glass effect, rainbow glass effect, and lattice glass effect provided in the embodiments of the present application;

[0078] Fig.39 A schematic diagram of window display effects of different translucent materials provided in an embodiment of the present application;

[0079] Fig.40 Schematic diagram of the main interface light and shadow display effect provided in the embodiment of the present application Figure 1 ;

[0080] Fig.41 Schematic diagram of the main interface light and shadow display effect provided in the embodiment of the present application Figure 2 ;

[0081] Fig.42A schematic diagram of light and shadow display effects during email refresh provided in an embodiment of the present application;

[0082] Fig.43 A schematic diagram of the light and shadow display effect during the pull-up menu startup process provided in an embodiment of the present application;

[0083] Fig.44 A schematic diagram of a light and shadow effect display method provided in an embodiment of the present application;

[0084] Fig.45 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0085] The technical solutions in the embodiments of the present application are described below in conjunction with the accompanying drawings in the embodiments of the present application. Wherein, in the description of the embodiments of the present application, the terms used in the following embodiments are only for the purpose of describing specific embodiments, and are not intended to be used as limitations on the present application. As used in the specification and the appended claims of the present application, the singular expressions "one", "a kind of", "said", "above", "the" and "this" are intended to include expressions such as "one or more", unless there is a clear indication to the contrary in the context. It should also be understood that in the following embodiments of the present application, "at least one", "one or more" refer to one or more (including two).

[0086] References to "one embodiment" or "some embodiments" etc. described in this specification mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Thus, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways. The term "connection" includes direct connection and indirect connection, unless otherwise specified. "First" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated.

[0087] In the embodiments of the present application, the words "exemplarily" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "exemplarily" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplarily" or "for example" is intended to present related concepts in a specific way.

[0088] In some embodiments, some presentation methods in the real physical world are introduced into the electronic device to realize the light and shadow effects of the graphical user interface (GUI) of the electronic device. For example, based on the preset virtual light source, the control shadow is displayed according to the Z-axis altitude of the control, thereby enriching the display effect and improving the user experience.

[0089] For example, Figure 1 As shown in (a), In the example, two virtual light sources are defined, such as virtual light source 11 and virtual light source 12. Figure 1 As shown in (b), the interface 101 displays a search bar 14, a background 15, and an add control 16. Figure 1 As shown in (c), the Z-axis altitudes of different controls displayed on the same interface may be different. Then, in order to achieve the light and shadow display effect of interface 101, the electronic device can draw shadows in combination with the lighting directions of virtual light sources 11 and 12, as well as the Z-axis altitudes of different controls. For example, the shadow of control 16 is added to interface 101, thereby improving the light and shadow display effect of interface 101.

[0090] However, the current light and shadow display effects are limited to some simple-shaped controls, and shadow display only supports circular controls and rounded rectangular controls, and cannot achieve better GUI light and shadow effects.

[0091] In other embodiments, The Universal Windows Platform (UWP) platform widely adopts real physical lighting definitions to enable the controls displayed on the GUI interface to produce lighting effects. The lighting definitions include the lighting type definitions shown in Table 1, the lighting attribute definitions shown in Table 2, the scene lighting effect definitions shown in Table 3, and the lighting model (rendering equation) definitions shown in Table 4. Figure 2 As shown, in the process of interface display, the electronic device combines the lighting definition and performs lighting rendering through the material shader. Among them, the lighting rendering process includes texture sampling, Gaussian blur, texture color mixing calculation and lighting calculation, so as to realize the output of texture. In the lighting rendering process, the material shader combines the lighting definition and uses parameters such as background texture (texture), Gaussian blur (Blur) parameter definition, exclusion mixing (texture / color), coverage color, noise function, etc. to realize lighting rendering.

[0092] Table 1

[0093]

[0094]

[0095] Table 2

[0096] property describe Color color Direction direction CoordinateSpace Coordinate System (which View) Cones Vertebrae (spot) Offset Offset Intensity strength Attenuation Attenuation coefficient

[0097] Table 3

[0098] property describe Normal Map Normal Map Ambient Ambient Light Specular Highlight Diffuse diffusion Reflectance Model Reflection Model

[0099] Table 4

[0100] property describe Blinn Phong Feng Model Physically Based Blinn Phong PBR

[0101] It can be seen that although UWP realizes the real lighting effect of the GUI interface by combining lighting definitions, it uses layer mixing in the lighting rendering process, combining the processing parameters of different layers (such as background texture, Gaussian blur parameter definition, etc.) to complete the lighting rendering. In UWP, acrylic materials are mainly used to express windows and various translucent effects, but the layer blending method results in poor results in achieving acrylic material effects.

[0102] In some 3D scenes, electronic devices can also render objects through 3D real-time rendering. The 3D real-time rendering process includes, for example, analyzing 3D scenes and rendering object data, 3D content modeling, making UV maps, baking maps for 3D models, making materials, real-time rendering and animation processing.

[0103] Although 3D real-time rendering is highly realistic and easy for users to understand, it can be widely used in games and other fields. However, due to the complex process of 3D real-time rendering, it requires high computing power of electronic devices and generates high power consumption, so it cannot be widely used in a variety of electronic devices. In addition, the process of 3D real-time rendering is different from the development process of traditional 2D human-machine interfaces and cannot be directly connected.

[0104] In some other embodiments, some GUI systems also implement some simulated lighting effects by overlaying light maps or using 2D rendering algorithms for simulation. Figure 3 The control glow effect shown. Another example is the text flash effect.

[0105] However, in the current method, different lighting effects need to be developed separately, which has a high development cost. In addition, the lighting effect is not realistic and cannot reflect the actual lighting effect.

[0106] Based on this, an embodiment of the present application provides a method for displaying light and shadow effects. The electronic device adaptively realizes the display of light and shadow effects based on preset lighting information and current display information, enriches the GUI light and shadow effects, and improves the user experience.

[0107] Optionally, the light and shadow effect display method provided in the embodiment of the present application can be applied to the electronic device 100. Figure 4 As shown, the electronic device 100 can specifically be a mobile phone 41, a tablet computer 42, a laptop computer 43, a smart screen 44, a car terminal 45, a wearable device (such as a smart watch, a smart bracelet, etc.) 46, a folding screen mobile phone, a computer, a personal computer (personal computer, PC), an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook, a personal digital assistant (personal digital assistant, PDA), a virtual reality (virtual reality, VR) device / augmented reality (augmented reality, AR) device, an artificial intelligence (artificial intelligence, AI) device and other terminal devices with display functions. The operating system installed in the above electronic device 100 includes but is not limited to Or other operating systems. This application does not limit the specific type of the electronic device 100 and the installed operating system.

[0108] Figure 5 A schematic structural diagram of an electronic device 100 is shown.

[0109] The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc.

[0110] It is to be understood that the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0111] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0112] The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of instruction fetching and execution.

[0113] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0114] In some embodiments, the processor 110 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0115] The MIPI interface can be used to connect the processor 110 with peripheral devices such as the display screen 194 and the camera 193. The MIPI interface includes a camera serial interface (CSI), a display serial interface (DSI), etc. In some embodiments, the processor 110 and the camera 193 communicate via the CSI interface to implement the shooting function of the electronic device 100. The processor 110 and the display screen 194 communicate via the DSI interface to implement the display function of the electronic device 100.

[0116] The USB interface 130 is an interface that complies with the USB standard specification, and specifically can be a Mini USB interface, a Micro USB interface, a USB Type C interface, etc. The USB interface 130 can be used to connect a charger to charge the electronic device 100, and can also be used to transfer data between the electronic device 100 and a peripheral device. It can also be used to connect headphones to play audio through the headphones. The interface can also be used to connect other electronic devices, such as AR devices, etc.

[0117] It is understandable that the interface connection relationship between the modules illustrated in the embodiment of the present application is only a schematic illustration and does not constitute a structural limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0118] The charging management module 140 is used to receive charging input from a charger. The charger may be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 may receive charging input from a wired charger through the USB interface 130. In some wireless charging embodiments, the charging management module 140 may receive wireless charging input through a wireless charging coil of the electronic device 100. While the charging management module 140 is charging the battery 142, it may also power the electronic device through the power management module 141.

[0119] The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, and supplies power to the processor 110, the internal memory 121, the display screen 194, the camera 193, and the wireless communication module 160. The power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle number, battery health status (leakage, impedance), etc. In some other embodiments, the power management module 141 can also be set in the processor 110. In other embodiments, the power management module 141 and the charging management module 140 can also be set in the same device.

[0120] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0121] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0122] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G, etc., applied to the electronic device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the same device as at least some of the modules of the processor 110.

[0123] The modem processor may include a modulator and a demodulator. The modulator is used to modulate the low-frequency baseband signal to be sent into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After the low-frequency baseband signal is processed by the baseband processor, it is passed to the application processor. The application processor outputs a sound signal through an audio device, or displays an image or video through a display screen 194. In some embodiments, the modem processor may be an independent device. In other embodiments, the modem processor may be independent of the processor 110 and be set in the same device as the mobile communication module 150 or other functional modules.

[0124] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the electronic device 100. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, modulates the frequency of the electromagnetic wave signal and performs filtering, and sends the processed signal to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, modulate the frequency of it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0125] In some embodiments, the antenna 1 of the electronic device 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the electronic device 100 can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology. The GNSS may include a global positioning system (GPS), a global navigation satellite system (GLONASS), a Beidou navigation satellite system (BDS), a quasi-zenith satellite system (QZSS) and / or a satellite based augmentation system (SBAS).

[0126] The electronic device 100 implements the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

[0127] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can be manufactured using a liquid crystal display (LCD), such as an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Mini-led, Micro-led, Micro-oled, quantum dot light emitting diodes (QLED), etc. In some embodiments, the electronic device 100 may include 1 or N display screens 194, where N is a positive integer greater than 1.

[0128] The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then passes the electrical signal to the ISP to be converted into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the electronic device 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.

[0129] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function, such as storing music, video and other files in the external memory card.

[0130] The internal memory 121 can be used to store computer executable program codes, which include instructions. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the electronic device 100 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc. The processor 110 executes various functional applications and data processing of the electronic device 100 by running instructions stored in the internal memory 121, and / or instructions stored in a memory provided in the processor.

[0131] The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signal. The audio module 170 can also be used to encode and decode audio signals. In some embodiments, the audio module 170 can be arranged in the processor 110, or some functional modules of the audio module 170 can be arranged in the processor 110. The electronic device 100 can use the audio module 170, such as music playing, recording, etc. The audio module 170 can include a speaker, a receiver, a microphone, a headphone interface, and an application processor to implement audio functions.

[0132] The sensor module 180 may include a pressure sensor, a gyro sensor, an air pressure sensor, a magnetic sensor, an acceleration sensor, a distance sensor, a proximity light sensor, a fingerprint sensor, a temperature sensor, a touch sensor, an ambient light sensor, a bone conduction sensor, and the like.

[0133] The key 190 includes a power key, a volume key, etc. The key 190 may be a mechanical key or a touch key. The electronic device 100 may receive key input and generate key signal input related to user settings and function control of the electronic device 100.

[0134] Motor 191 can generate vibration prompts. Motor 191 can be used for incoming call vibration prompts, and can also be used for touch vibration feedback. Motor 191 can also correspond to different vibration feedback effects for touch operations on different areas of display screen 194. Different application scenarios (for example: time reminder, receiving information, alarm clock, game, etc.) can also correspond to different vibration feedback effects.

[0135] Indicator 192 may be an indicator light, which may be used to indicate charging status, power changes, messages, missed calls, notifications, etc.

[0136] The SIM card interface 195 is used to connect a SIM card. The SIM card can be connected to or disconnected from the electronic device 100 by inserting or removing the SIM card interface 195. The electronic device 100 can support 1 or N SIM card interfaces, where N is a positive integer greater than 1.

[0137] The software system of the electronic device 100 may adopt a layered architecture, an event-driven architecture, a micro-core architecture, a micro-service architecture, or a cloud architecture. The embodiment of the present application takes the Android system of the layered architecture as an example to exemplify the software structure of the electronic device 100.

[0138] Figure 6 It is a software structure block diagram of the electronic device 100 according to an embodiment of the present application.

[0139] The layered architecture divides the software into several layers, each with clear roles and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom, namely, the application layer, the application framework layer, the system layer, and the kernel layer.

[0140] The application layer can include a series of application packages.

[0141] like Figure 6 As shown, the application package may include applications such as calendar, contacts, memo, camera, music, gallery, map, call, video, etc.

[0142] The application framework layer provides an application programming interface (API) and a programming framework for the applications in the application layer. The application framework layer includes some predefined functions.

[0143] like Figure 6 As shown, the application framework layer may include a user interface (UI) framework, a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager, and the like.

[0144] For example, Figure 7 As shown, the UI framework is used to save global illumination definitions, build a hierarchical architecture of UI components, and create rendering tasks.

[0145] Optionally, the UI framework includes a global illumination definition, which is used to indicate various physical illumination principles, illumination types, illumination properties, scene illumination effects, illumination models, illumination materials, shadow properties, etc. in the real world. Based on the global illumination information defined by the UI framework, the shadow effect realization in the subsequent rendering process is enriched.

[0146] Optionally, the UI framework reconstructs the current basic UI components, adds relevant information such as material, lighting, environment, border properties, shadow properties, etc., so as to complete lighting material rendering in a single-layer canvas.

[0147] For example, Figure 8 As shown, the UI components are reconstructed in a single-layer canvas, and the UI framework divides the rendering process of the UI components into 5 layers, including border drawing, lighting layer, UI canvas, surface layer (surface), and shadow bearing layer. Among them, during the rendering process, the lighting definition can be used to indicate information such as the lighting type, lighting spatial position, lighting attenuation coefficient, and lighting color. Border drawing is used to draw the self-luminous edge effects, highlight edge effects, outline and other effects of UI components. The lighting layer is used to receive lighting and complete lighting drawing. The UI canvas is used to complete the drawing of the UI component style content. The surface is used to draw the display container of the system layer. The shadow bearing layer is used to carry the layer shadows and overflow lighting drawing to achieve system-level non-placeholder shadows and lighting.

[0148] Optionally, the UI framework is also used to create rendering tasks, including lighting rendering tasks. In addition, the UI framework sends the created lighting tasks to the rendering service of the system layer to trigger rendering.

[0149] The window manager is used to manage window programs. The window manager can obtain the display screen size, determine whether there is a status bar, lock the screen, capture the screen, etc.

[0150] Content providers are used to store and retrieve data and make it accessible to applications. The data may include videos, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.

[0151] The view system includes visual controls, such as controls for displaying text, controls for displaying images, etc. The view system can be used to build applications. A display interface can be composed of one or more views. For example, a display interface including a text notification icon can include a view for displaying text and a view for displaying images.

[0152] The phone manager is used to provide communication functions of the electronic device 100, such as management of call status (including connecting, hanging up, etc.).

[0153] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0154] The notification manager enables applications to display notification information in the status bar. It can be used to convey notification-type messages and can disappear automatically after a short stay without user interaction. For example, the notification manager is used to notify download completion, message reminders, etc. The notification manager can also be a notification that appears in the system top status bar in the form of a chart or scroll bar text, such as notifications of applications running in the background, or a notification that appears on the screen in the form of a dialog window. For example, a text message is displayed in the status bar, a prompt sound is emitted, an electronic device vibrates, an indicator light flashes, etc.

[0155] The system layer may include multiple functional modules, such as surface manager, graphics processing library (such as OpenGL ES), rendering service, graphics engine (such as SGL), media library, etc.

[0156] The surface manager is used to manage the display subsystem and provide the fusion of 2D and 3D layers for multiple applications.

[0157] The media library supports playback and recording of a variety of commonly used audio and video formats, as well as static image files, etc. The media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

[0158] The graphics processing library is used to implement graphics drawing, image rendering, compositing, and layer processing. In some examples, the graphics processing library includes a rendering service.

[0159] The graphics engine is a drawing engine that draws pictures.

[0160] The kernel layer is the layer between hardware and software. The kernel layer contains at least display driver, camera driver, audio driver, and sensor driver.

[0161] For example, Figure 7 As shown, the rendering service is used to perform unified rendering process, cross-window global visual effects, animation UI thread separation, and element sharing between processes. Among them, unified rendering includes rendering tree, surface management, shadow bearing node management, control tree, lighting material rendering, and shadow rendering.

[0162] Graphics engine, used to support rendering services to achieve lighting rendering capabilities, shadow drawing capabilities, and physical material rendering capabilities. The graphics engine includes, for example, 2D rendering algorithms, shader runtime (ShaderRuntime), and 2D drawing engines. Optionally, shadow drawing includes full shadow (or umbra) drawing and penumbra drawing. Physical materials include general materials and translucent materials. General materials are, for example, physically based rendering (PBR) materials.

[0163] In some embodiments, when the electronic device determines that the display of the current interface needs to be refreshed (such as when a touch event is detected, etc.), one or more layers are drawn through the UI thread (such as Do Frame) in the UI framework; the rendering (Render) thread (such as Draw Frame) performs layer rendering on one or more layers. Afterwards, the synthesis thread (SurfaceFlinger) in the display framework performs layer synthesis on the drawn one or more layers (i.e., one or more layers after rendering) to obtain an image frame. Afterwards, the LCD driver of the hardware display module can receive the synthesized image frame, and the synthesized image frame is displayed by the LCD. After the LCD displays the image frame, the image displayed by the LCD can be perceived by the human eye.

[0164] Optionally, the event that triggers rendering is, for example, a touch operation of a user on a display screen detected by the electronic device, such as clicking an interactive control, or a sliding operation of a user on the display screen (such as sliding up and down, sliding left and right, etc.); or it may be a voice input of a user detected by the electronic device; or it may be an event of an electronic device's foreground application automatically switching screens, etc., which is not limited here. The foreground application is the application corresponding to the interface currently displayed on the display screen of the electronic device.

[0165] It should be noted that the screen content of the foreground application can include not only the screen visible to the user, but also content without a user interface, content of a transparent layer, or content that is blocked by other application interfaces and is invisible to the user. The electronic device needs to obtain all the control data and layout data to be drawn corresponding to the screen content.

[0166] Optionally, the rendering process includes control rendering (including layout rendering) and lighting rendering. The lighting rendering process is introduced in the embodiment of the present application, and the control rendering process can refer to the prior art, which will not be described in detail in the embodiment of the present application.

[0167] In some embodiments, during the UI drawing process, the UI thread can draw a view tree, which is used to represent controls and layouts organized in a tree structure. Based on the tree structure of the view tree, different rendering nodes can be determined during the rendering process.

[0168] For example, Fig. 9 As shown, combined with the control tree structure and Figure 8 The layered rendering structure shown in the figure can reconstruct the rendering process and complete the drawing of UI components and lighting borders in a unified canvas. The rendering service builds a shadow receiving layer based on the Z-axis height, shadow properties and other information of the UI component to achieve non-placed shadows and overflow lighting rendering. For example, Fig. 9 As shown, the rendering service completes the drawing of UI content, lighting drawing and border drawing on the UI canvas. In addition, the overflow shadow corresponding to the UI component can be displayed in the shadow receiving layer to achieve non-placeable shadow and overflow lighting rendering.

[0169] For example, Fig.10 The lighting rendering system shown introduces the overall lighting rendering process in combination with the interaction process of different modules. When a user performs input operations on an electronic device (such as triggering an electronic device to start an application), the kernel layer can generate corresponding input events (such as application startup events) based on the input operations, and report the event to the application framework layer. The UI framework of the application framework layer determines the control data and layout data corresponding to the application, as well as the process logic of the rendering service rendering (such as creating a rendering task), and sends it to the rendering service of the system layer. The rendering service can process the acquired information through the graphics engine, and send the processed rendering results to the display driver of the kernel layer, and the display driver displays the corresponding application interface on the display screen.

[0170] Among them, lighting information is preset in the UI framework, and the lighting information is a global lighting definition. As mentioned above, the lighting information includes light source information (such as light source type, light source properties, etc.), physical properties, image light and shadow, etc. After determining the process logic of the rendering service, the UI framework sends the light source information and rendering task to the rendering service. In addition, the rendering service can also obtain the display information required in the current rendering process, such as time information, location information, motion status information (such as the hand gesture information of the electronic device), interface virtual light source information, control height information, control transparency and other information. Afterwards, the rendering service can combine the lighting information and display information, and perform the rendering task through the graphics engine to output the lighting rendering result.

[0171] Optionally, the graphics engine includes a 2D rendering pipeline, and the rendering service completes the lighting rendering through the 2D rendering pipeline. In some examples, the rendering service can also complete the lighting rendering through the GPU rendering pipeline, thereby obtaining a better rendering effect.

[0172] In this way, by adding a lighting rendering system to the existing 2D GUI interface and by defining and simulating real physical lighting and lighting systems, the interaction and reality enhancement of the existing GUI interface can be achieved.

[0173] In addition, humans are subconsciously familiar with the lighting system, which can avoid increasing learning costs and has high user acceptance. Moreover, the effect of the lighting system is consistent with human cognition, which improves the sophistication and aesthetics of the GUI interface and is easy to gain user recognition.

[0174] For example, based on Fig.10 With the lighting rendering system shown, electronic devices can obtain more realistic lighting effects on the GUI interface. Fig.11 As shown, when the electronic device displays the desktop, the shadow effects on the icons 111 and 112 displayed on the desktop can rotate counterclockwise as time changes. Fig.12 As shown, when the electronic device displays the desktop, the light and shadow effects on the icons displayed on the desktop can change as the posture of the handheld electronic device changes. For example, the size of the light and shadow area on icon 121 changes as the direction and angle of the user shaking the electronic device. For example, Fig.13 As shown, the electronic device receives a pop-up notification, and as the posture of the handheld electronic device changes, the position of the light and shadow on the pop-up notification 131 changes.

[0175] Optionally, the electronic device can confirm the posture of the electronic device through a sensor installed on the electronic device. For example, the electronic device is equipped with a gyroscope sensor, which is used to determine the motion posture of the electronic device. In some embodiments, the angular velocity of the electronic device around three axes (i.e., x, y, and z axes) can be determined by a gyroscope sensor. For another example, the electronic device is equipped with an acceleration sensor, which can detect the magnitude of the acceleration of the electronic device in various directions (generally three axes). When the electronic device is stationary, the magnitude and direction of gravity can be detected. Alternatively, it can also be used to identify the posture of the electronic device and applied to horizontal and vertical screen switching.

[0176] In this way, the electronic device combines the illumination information and the display information to provide the user with a GUI lighting effect that better meets the user's needs, thereby improving the user's experience.

[0177] The above briefly introduces the overall solution of the light and shadow effect display method provided in the embodiment of the present application. The following is a detailed introduction to the specific implementation process of the solution.

[0178] In some embodiments, the main visual features of the physical world are determined by lighting, basic physical properties and lighting rendering equations in the computer vision representation system, wherein the main visual features include color information, projection information and physical material information.

[0179] Among them, in the long-term subconsciousness of human beings, the influence of light on physical visual effects has long been deeply rooted. Therefore, by introducing a lighting rendering system into the GUI system, the lighting rendering system can bring color, brightness, shadow, material and other features to the GUI, which can significantly improve the user experience of the GUI.

[0180] For example, Fig.14 As shown, based on lighting, rendering equations (PBR) and basic physical properties, the electronic device can output visual features (such as color information), projection information, and physical material information on the GUI interface.

[0181] For example, electronic devices build a lighting rendering system that can be used in 2D GUI systems based on the principle of light and shadow material formation in the physical world and the characteristics of 2D GUI systems. By giving graphic controls, icons, cards, pictures, etc. in 2D GUI lighting visual effects, real physical shadow effects, and material effects, users can be more accepting and understanding of the human-computer interaction interface, and ultimately improve the user experience.

[0182] In some embodiments, the system-level lighting capability of the electronic device is implemented by physically real lighting (such as corresponding to lighting information), UI system physical properties, and image lighting. Fig.10 As shown, the lighting information includes light source information, physical properties, and image light and shadow.

[0183] The light source information includes the light source type and light source attributes. As shown in Table 5, the light source type includes diffuse reflection light source, parallel light (Sun) light source, point light source, spotlight light source, natural environment light source, sky light source, etc. For example, Fig.15 The display effects of point light source, spotlight light source and parallel light source are shown.

[0184] As shown in Table 6, the light source attributes include the color, direction, coordinate system (which View) position, cone (spot), offset, intensity, attenuation coefficient, etc. of the light source. For example, the coordinate system position of the light source changes with time (such as the sun rises in the east and sets in the west). Optionally, the scene of the light source attribute change includes, for example, a scene that changes with time, a scene that changes with weather (such as rain, thunder, fog, etc.), a scene that changes with the system body, a scene that changes with the product form, etc.

[0185] Based on different light source types and light source properties, different lighting effects can be produced on UI components, and thus based on the lighting effects, the electronic device can achieve corresponding light and shadow display effects.

[0186] Table 5

[0187] light source describe AmbientLight Diffuse DistantLight Parallel light(Sun) PointLight Point Light SpotLight spotlight HDR Ambient Light Natural ambient light, sky light, etc.

[0188] Table 6

[0189] property describe Color color Direction direction Coordinate Space Coordinate system (which View) Cones Vertebrae (spot) Offset Offset Intensity strength Attenuation Attenuation coefficient

[0190] In some embodiments, as described above, the electronic device adds global illumination definition (i.e., light source information) in the UI framework. Fig.16 As shown, the global illumination positioning position in the UI framework includes graphic controls, layout system, window management framework, and light sources. The defined light source content includes the light source type and light source properties as described above. The defined graphic control content includes control type, graphic style, and physical properties.

[0191] Optionally, the system-level lighting and shadow capabilities of the electronic device are implemented through physical real lighting (such as corresponding to lighting information), UI system physical properties, and image lighting and shadow. Among them, the UI system physical properties include UI element properties, window properties, UI normal properties, and physical properties (such as metalness, roughness, reflectivity, transparency, base color, etc.).

[0192] For example, the basic physical property definitions are preconfigured in the UI framework as shown in Table 7. Then, in the subsequent rendering process, the rendering system can better complete the lighting rendering based on these basic physical property definitions.

[0193] It should be understood that the names and descriptions of the basic physical properties may be different in different operating systems, but the physical essence of the content expressed by different descriptions is the same. For example, the "specular reflection\glossiness" process that may be used in some systems and the "metallicity / roughness" process provided in the embodiments of this application are physically the same, but they are described in different ways.

[0194] Table 7

[0195]

[0196] In some examples, different graphic controls may have different physical properties based on different application scenarios and style appearances. As shown in Table 7 above, default values ​​corresponding to different physical properties can be set in the system.

[0197] Optionally, for some specific application scenarios, or to achieve the developer's design goals, the graphic control needs to write different physical property values. Therefore, the electronic device needs to support developers or users to customize physical properties. For example, the electronic device provides an API interface for setting different physical properties of the graphic control. Alternatively, the electronic device sets themes corresponding to different physical properties and updates the physical properties of the graphic control for different themes. Alternatively, the electronic device sets the physical properties of the graphic control in the control declaration or layout description.

[0198] For example, the following takes setting different physical properties of a graphic control through an API interface as an example. Through the API interface, the metalness (Matellic) of the graphic control can be set to the default value, the roughness (Rouguness) value to 0.2, and the reflectance (Reflectance) value to 0.1.

[0199]

[0200] In some embodiments, the specific physical properties of each pixel of some graphic controls may be different, so the electronic device can set the physical properties by means of mapping.

[0201] For example, in the above text, the physical properties directly set by the electronic device in various ways are the physical properties of each pixel of the graphic control. However, in different scenes, the physical properties of the pixels of some graphic controls are different. For example, in a scene where there is a pattern on the background of the graphic control, the metalness, normal, and roughness of different pixels may be different.

[0202] For example, Fig.17 The schematic diagram of the display of metalness, normal, and roughness is shown in FIG. For example, the electronic device determines that the roughness of each pixel of the metalness of the current graphic control to be rendered is 0.2, and the physical properties of the graphic control can be set by setting the above API interface. For another example, Fig.17 As shown, the electronic device determines that the metalness requirements of different pixels of the current graphic control to be rendered are different. In this case, a texture can be generated first, and then the texture is covered on the surface of the graphic control to change the roughness of the display of the graphic control, that is, the physical properties of the graphic control meet the requirements through the texture.

[0203] In other embodiments, some physical properties of the graphic control may also be automatically generated. For example, some physical properties of the graphic control may be missing due to the difficulty in setting them or the developer's low willingness to set them. Therefore, the electronic device may preset an image conversion algorithm to automatically generate some or all physical properties of the graphic control.

[0204] For example, normal information is an important part of the physical properties of a graphic control, which can significantly improve the lighting effect and the micro-stereoscopic effect produced by the lighting. Therefore, the embodiment of the present application takes the automatic generation process of normal information as an example to introduce the automatic generation process of some physical properties of a graphic control. Fig.18As shown, based on the original data of the graphic control, such as layout information, Z-axis height, etc., the electronic device generates a normal map or a height map through an image conversion algorithm, thereby automatically outputting the discovery information of the graphic control. Optionally, the implementation process of the image conversion algorithm includes, for example, inputting a grayscale image and performing partial derivative calculation based on the grayscale image to output a corresponding display effect.

[0205] Optionally, the system-level lighting capability of the electronic device is implemented through physical real lighting (such as corresponding to lighting information), UI system physical properties, and image lighting, wherein the image lighting includes providing real color projection capability for semi-transparent UI and generating normals for images.

[0206] The following is an introduction to semi-transparent material rendering.

[0207] In some embodiments, the electronic device can display a variety of display interfaces, including some translucent interfaces. For these translucent interfaces, the electronic device can refer to the translucent materials in the real world for rendering. However, for the translucent materials, they have more physical properties, so it is necessary to expand the physical properties supported by the electronic device in the lighting rendering process for the translucent materials.

[0208] For example, the following table 8 illustrates some extended contents of the physical properties of semi-transparent materials, including, for example, the refractive index, transparency, absorptivity, transmittance, etc. of the semi-transparent material.

[0209] Table 8

[0210]

[0211] For example, Fig.19 As shown, materials with different refractive indices (IOR) can produce different refractive effects. For example, when the refractive index is 1.0, observing the object 192 through the translucent material 191 will not produce a refractive effect. When the refractive index is 1.33, observing the object 192 through the translucent material 191 will produce a refractive effect, such as the displayed position of the object 192 in the translucent material 191 is offset from the actual position of the object 192.

[0212] For example, Fig. 20 As shown, in the process where the absorbance changes from (0.0, 0.02, 0.14) to (0.0, 0.36, 2.3), the translucent material produces different light absorption effects.

[0213] For example, Fig.21 As shown, when the transmittance changes from 0.0 to 1.0, the translucent material produces different light transmission effects.

[0214] For example, Fig. 22 As shown, the thickness of the solid transmissive material changes from 0.0 to 3.0, and the translucent material produces different light and shadow effects.

[0215] The above article introduces the relevant contents of illumination, basic physical properties, and extended physical properties. Based on illumination, rendering equations, and basic physical properties, electronic devices can output visual features. Then, the following article introduces the rendering equations.

[0216] In some embodiments, in computer graphics, the rendering equation is an integral equation. It is the theoretical basis of all global illumination methods (such as ray tracing, path tracing, radiosity, etc.). Among them, rendering is to calculate the radiation of light, which is the basic characteristic of the distribution of light in the environment.

[0217] In some embodiments, different rendering equations (which may also be described as lighting calculation models) are preset in the electronic device, and a lighting rendering result is output during the lighting rendering process through any of the different rendering equations.

[0218] Optionally, according to the hardware capabilities of the electronic device, application scenarios, etc., the developer or the system may select a rendering equation to be configured in the electronic device.

[0219] Optionally, the rendering equation includes, for example, a Blinn-Phone lighting model, a PBR lighting model, and a ray-tracing rendering model.

[0220] In some examples, the Blinn-Phone lighting model determines the final lighting effect based on the lighting of diffuse properties and the lighting of specular properties. For example, lighting = diffuse lighting + specular lighting. Among them, diffuse lighting can be determined based on light color, diffuse amount, normal, and light direction. Specular lighting can be determined based on light color, specular amount, normal, and shininess. For example, diffuse lighting = light color*diffuse amount*dot(normal,light direction); specular lighting = light color*specular amount*dot(normal,half way vector)^shininess.

[0221] Among them, the Blinn-Phone illumination model has better calculation speed and output illumination rendering effect, and is widely used in the illumination rendering process. However, the Blinn-Phone illumination model is an empirical model and cannot fully conform to the illumination phenomenon in the real world.

[0222] Some examples include Fig.23 As shown in the figure, the PBR lighting model performs lighting rendering based on the physical energy conservation principle that the incident light is equal to the sum of the specular reflected light and the diffuse reflected light. Among them, the PBR lighting model is the lighting model closest to the real world. However, the PBR lighting model is relatively complex, with more branches and more calculation items.

[0223] Some examples include Fig.24 As shown in the figure, the basic principle of the ray-tracing rendering model is that there is a beam of light, assuming that it emits a photon, and then the photon will fly in a straight line until the photon hits the surface of an object, at which time reflection and refraction will occur and a series of energies will be subtracted, and finally the photon will reach the camera. The ray-tracing rendering model is to reverse everything and use the camera to emit rays, doing the reverse process of the above process. That is: the light enters the scene from the camera, reflects or refracts, and finally hits the light source.

[0224] Among them, the ray tracing rendering model is close to the real physical world, but its calculation is complex and consumes a lot of computing power. It is generally only used in scenes with extremely high requirements for effects or strong computing power of the equipment. In other scenes, the ray tracing rendering model will be used in combination with other rendering models.

[0225] In some embodiments, as described above, there are extended contents of the physical properties of the semi-transparent material. Then, based on the extended physical properties of the semi-transparent material, the rendering equation can also be extended.

[0226] For example, Fig.25 As shown in the figure, when light is incident on a translucent material, the translucent material will generate transmitted light and reflected light. Based on the translucent material, the transmitted light will generate a transmitted light roughness value, and the reflected light will generate a reflected light roughness value. In addition, when light is incident on a translucent material, it will produce attenuation, thickness attribute influence, color and distance influence on absorption rate, etc. Then, the rendering service is based on Fig.25 The extended rendering equation principle shown in the figure is used for lighting rendering.

[0227] Optionally, the developer can select the rendering equation configured in the electronic device based on factors such as the performance of the electronic device. Alternatively, a variety of rendering equations are configured in the electronic device. When lighting rendering is required, the electronic device selects a suitable rendering equation for lighting rendering based on its current state. For example, when lighting rendering is required, the electronic device has sufficient power. Then, the electronic device can select a rendering equation that has a better rendering effect but requires higher computing power.

[0228] The above introduces the rendering equation and the extended rendering equation. The following introduces the specific implementation process of lighting rendering in detail.

[0229] In some embodiments, based on real physical lighting, the generation of shadows is related to basic requirements such as physical properties of UI components (such as height), light source position, UI component shape, and UI component material. The electronic device performs shadow rendering based on the above basic requirements for shadow generation. The lighting rendering process includes shadow rendering.

[0230] For example, the generation of shadows is highly related to the UI components. Fig.26 As shown in (a) and (b), the shadow of a UI component includes an umbra and a penumbra. For the same light source (same position, height, etc.), UI components at different heights produce different shadow effects. For example, the umbra of a UI component appears more solid, while the penumbra appears more virtual. The higher the height of the UI component and the closer it is to the light source, the larger its penumbra will be, and the more obvious the shadow display effect will be. Fig.26 As shown in (c), during the UI component rendering process, the rendering service determines the corresponding shadow display effect according to the height of the UI component.

[0231] For example, the generation of shadows is related to the position of the light source. Fig. 27 As shown in FIG. 1 , based on the direction angle shown in the xy coordinate system, the positional relationship between the light source and the UI component is indicated. Fig. 27 As shown in (a), the light source 271 is located at a 45-degree direction from the UI component 272 (i.e., the light source is located at the upper right of the UI component), and the rendering system outputs the shadow of the UI component 272 as a 45-degree left shadow (i.e., the shadow is located at the lower left of the UI component). Fig. 27 As shown in (b), the light source 271 is located at a 90-degree direction to the UI component 272 (i.e., the light source is located directly above the UI component), and the shadow output by the rendering system for the UI component 272 is a 90-degree bottom shadow (i.e., the shadow is located directly below the UI component). Fig. 27 As shown in (c), the light source 271 is located at 135 degrees to the UI component 272 (i.e., the light source is located at the upper left of the UI component), and the rendering system outputs the shadow of the UI component 272 as a 135-degree right shadow (i.e., the shadow is located at the lower right of the UI component).

[0232] For another example, the generation of shadows is related to the shape of UI components. The rendering system can output shadow display effects of different shapes according to the different shapes of UI components. In addition to supporting the rendering of circular and rounded rectangular shadows, the rendering system also supports the rendering of various special-shaped shadows. For example, Fig.28 As shown in (a), the rendering system completes the shadow rendering of the circular UI component. Fig.28 As shown in (b), the rendering system completes the shadow rendering of the square UI component. Fig.28 As shown in (c), the rendering system completes the shadow rendering of the bird-shaped (alien) UI component.

[0233] For another example, the generation of shadows is related to the material of UI components. For example, if the translucent material supports color projection, then the shadow color is similar to the color of the UI component of the translucent material. Fig.29 As shown, the color of the card to be rendered by the electronic device is blue, and the material of the card is translucent. Then, the rendering service can draw a light blue shadow according to the color and material of the UI component (i.e., the card), so that the shadow display effect is more consistent with the shadow effect in the real physical world, so as to enhance the user experience.

[0234] In some embodiments, different light sources and physical properties can bring about different shadow effects, and the rendering service determines the shadow rendering effect according to the light source and the physical properties.

[0235] In some examples, the same light source and UI components, but different UI component positions and shapes, can output different shadow rendering effects.

[0236] For example, Fig.30 As shown in the figure, the rendering service outputs different shadow rendering effects based on the same light source (such as the same light source type and light emission direction) when the element is lying flat and when the element is standing upright. Fig.30 As shown in (a), element 301 is the product display window to be shadow rendered. Based on the flat state of element 301, the rendering service outputs an elliptical shadow rendering effect as shown by reference numeral 302. Fig.30 As shown in (b), as the element 301 changes from a flat state to an upright state, as shown by reference numeral 303, the shadow of the element 301 changes from an ellipse to an extended rectangle.

[0237] In some examples, different light source types may output different shadow rendering effects for the same UI component. Fig.30 As shown in (a), the rendering service outputs an elliptical shadow rendering effect as shown by reference numeral 302 based on the flat state of element 301 and the natural ambient light source. Fig.30 As shown in (c) , the rendering service outputs a curved rectangular shadow rendering effect as shown by reference numeral 304 based on the flat state of the element 301 and the spotlight light source.

[0238] In this way, the rendering service brings different shadow effects according to different light sources and physical properties, enriches the GUI interface display, and improves the user experience.

[0239] In some embodiments, the electronic device may also preset a shadow rendering algorithm, and implement the simulation of the physical real shadow of the UI component based on the shadow rendering algorithm. Optionally, based on the shadow simulation, the 2D GUI interface may obtain a 3D shadow display effect.

[0240] Optionally, the shadow rendering algorithm includes, for example, a shadow map (ShadowMap) algorithm, a percentage closer filtering (PCF) algorithm, a percentage closer soft shadows (PCSS) algorithm, etc.

[0241] For example, the rendering service uses a shadow mapping algorithm to determine the reachable position of light from the light source based on the depth map of the UI component, and outputs the corresponding shadow rendering effect.

[0242] For example, Fig.31 As shown in (a), the UI component to be shadow rendered is composed of different objects, so the electronic device can obtain the depth map of the UI component. Based on the depth map, the electronic device can use the shadow mapping algorithm to determine whether the light source at different positions on the UI component is reachable, so as to output the corresponding shadow effect. Fig.31 As shown in (b), based on the aforementioned principle, the electronic device can output the shadow rendering effect of the current UI component.

[0243] For example, the rendering service uses the PCF algorithm to sample pixels around the shadow and perform mean filtering based on the percentage. As a result, the edge of the shadow processed by PCF will be blurred without severe jagged edges, thus achieving a better shadow display effect.

[0244] For example, Fig.32 As shown in the figure, the origin of the coordinate system is the center of the shadow, and the circles of different layers are used to represent the distance from the center of the shadow. Then, during the shadow rendering process, the rendering service samples the pixels around the shadow according to the distance of different shadow positions from the center of the shadow, and performs mean filtering according to the proportion to output the shadow rendering effect.

[0245] In some examples, the shadow rendering algorithms in the above examples can output the shadow rendering effects required by users. However, these shadow rendering algorithms have problems such as high computing power, poor shadow rendering effects, or no support for color shadows. Based on this, the following introduces three shadow rendering methods to achieve real shadow display effects in 2D GUI scenes.

[0246] For example, the rendering service can determine the shadow offset and direction based on the light source coordinates and the height of the UI component to output the shadow rendering effect.

[0247] For example, Fig.33 In the coordinate system diagram shown in (a), the rendering service can determine the coordinates of the light source and the coordinates of object 1 (i.e., UI component), and can obtain the height of object 1. Then, the rendering system can render the shadow. For example, Fig.33 As shown in (b), the rendering service is based on Fig.33 According to the principle shown in (a), shadow rendering is performed on object 2, and the shadow rendering effect can reflect the shadow effect in the real physical world.

[0248] For example, the rendering service can copy the original content and build a shadow map. Then, it can achieve the PCF soft shadow effect by blurring.

[0249] For example, Fig.34 As shown in (a), the rendering service copies the UI component, constructs a shadow map 341, and then obtains a shadow rendering effect as shown by reference numeral 342 through blurring.

[0250] As another example, Fig.34 As shown in (b), the rendering service copies the UI component, constructs a shadow map 343, processes the shadow map 343 in a blurring manner, reduces the blur layer, and obtains a shadow rendering effect as shown by reference numeral 344.

[0251] For example, the rendering service uses the Progressive Blur algorithm to achieve soft shadow rendering effects.

[0252] For example, Fig.35 As shown, during the shadow rendering process, the shadow of the UI component is blurred by the ProgressiveBlur algorithm. The farther away from the UI component, the higher the degree of blurring, thereby achieving a soft shadow rendering effect.

[0253] In some embodiments, during the lighting rendering process, for translucent materials, the rendering service can combine the material properties of the translucent material and the real physical lighting to construct the effects of translucent materials such as frosted glass, Changhong glass, and acrylic glass. This makes the display effect of the translucent material UI component in the GUI interface of the electronic device more in line with the user's expectations. For example, the picture to be displayed on the interface is a glass partition door. The electronic device combines the material properties of the translucent material and the real physical lighting to render the current picture so that the display effect of Changhong glass is finally achieved.

[0254] For example, Fig.36 The diagram is a schematic diagram of the Changhong glass principle. The electronic device combines at least one of the roughness, transmittance and thickness of the translucent material with the refractive index map of the translucent material to output various glass material effects. Fig.37 A schematic diagram showing different roughness, different transmittance, and different thickness of a translucent material. Fig.36 The principle shown and Fig.37 The different physical properties shown in the figure can realize the following UI components displayed by the electronic device: Fig.38 The frosted glass effect, rainbow glass effect, and lattice glass effect are shown.

[0255] In some embodiments, during the lighting rendering process, the electronic device can also construct various translucent material applications or application window display effects according to the lighting and UI component materials. For example, the electronic device can construct window display effects such as paper, metal, leather, and colored glass. Fig.39 As shown in (a), the electronic device constructs various translucent materials according to the lighting and UI component materials, thereby achieving Fig.39 Window display effects of different translucent materials shown in (b).

[0256] In this way, by constructing a semi-transparent material display effect, the GUI interface display is enriched to meet the user's usage needs.

[0257] The above introduces the specific implementation principle and process of the light and shadow effect display method provided in the embodiment of the present application. The following is an exemplary description of the display effect of the GUI interface of the electronic device after lighting rendering based on the light and shadow effect display method.

[0258] For example, Fig.40As shown, the electronic device displays a main interface, and the main interface displays contents such as a calendar card, a weather card, and an application icon. Among them, as time changes, it can be seen that there are obvious changes in the lighting effects on the calendar card, the weather card, and the application icon. For example, the position of the lighting point on the calendar card 401 changes with time. Alternatively, the electronic device can adjust the shadow position and size of the card according to the lighting position. Alternatively, the electronic device combines the card color and outputs a shadow of a similar color.

[0259] As another example, Fig.41 As shown, the electronic device displays a lock screen interface, and the lock screen interface displays information such as time, temperature, date, and recommended content. Among them, as shown in the figure mark 411, the electronic device displays the obvious light and shadow effect of the time component based on the Z-axis height of the time component and the position where the light of the light source can reach. In addition, the electronic device can change the light and shadow display effect of the interface as the posture of the electronic device changes according to the posture detection of the sensor. Alternatively, the electronic device uses the normal information of the text component, as shown in the figure mark 412, so that the text component produces a micro-stereoscopic effect.

[0260] As another example, Fig.42 The light and shadow changes of the GUI interface in the email refresh scene shown in the figure. Fig.42 In the interface 4201 shown in (a), the electronic device detects a user sliding down the interface 4201 in the direction indicated by arrow 421 while displaying the email application, triggering a refresh of the currently displayed email list. Fig.42 In the interface 4202 shown in (b), the electronic device determines to perform lighting rendering on the search bar 422 according to the lighting information and the current display information. As shown in the interface 4202, the electronic device displays a shadow 423 above the search bar 422. Fig.42 The interface 4203 shown in (c) and Fig.42 In the interface 4204 shown in (d), as the user slides down, the length of the shadow 423 displayed above the search bar 422 also increases. Thus, when the user refreshes the email list, the shadow changes, which deepens the user's understanding of the details and provides a better human-computer interaction experience for the user.

[0261] Optionally, when the electronic device detects that the user has finished the sliding operation, the display of the mail list interface can be restored in the order of interface 4204-interface 4203-interface 4202-interface 4201. In this process, the electronic device can gradually reduce the length of the shadow 423 displayed above the search bar 422. Therefore, during the process of refreshing and restoring the mail list, the shadow changes can also provide a better human-computer interaction experience for the user.

[0262] For example, Fig.43The pull-up menu shown in the figure starts the GUI interface light and shadow changes in the scene. Fig.43 In the interface 4301 shown in (a), the electronic device detects that the user swipes up along the bottom of the display screen and displays a pull-up menu. During the rendering process of the pull-up menu, the electronic device determines to perform lighting rendering on the pull-up menu 431 based on the lighting information and the current display information. As shown in the interface 4301, the electronic device displays a shadow 432 above the pull-up menu 431. In addition, as shown in the interface 4301 and as shown in Fig.43 In the interface 4302 shown in (b), as the distance of the user's upward swipe increases, the length of the shadow 432 displayed above the pull-up menu 431 also increases. Thus, when the user swipes up the menu bar, the shadow changes, providing a better human-computer interaction experience for the user.

[0263] Fig.44 The flowchart of a display processing method provided by the embodiment of the present application is shown in FIG. Fig.44 The specific order described below is a limitation, and it should be understood that in other embodiments, the order of some steps in the method can be interchanged according to actual needs, or some steps can be omitted or deleted. The method includes the following steps:

[0264] S4401: The electronic device detects a first operation of the user and determines to refresh a first interface being displayed.

[0265] The first operation is, for example, an operation instructing the user to display the desktop, an operation to refresh the current interface (such as a sliding operation on the interface), an operation instructing to jump to other application interfaces, etc. Alternatively, the first operation may also be an unlocking operation, in which case the first interface is a lock screen operation, and the refreshed second interface is an unlocked interface.

[0266] S4402. The electronic device obtains lighting information and current display information.

[0267] The illumination information is a preconfigured global illumination definition, which is used to indicate one or more of the physical illumination principle, illumination type, illumination property, scene illumination effect, illumination model, illumination material, and shadow property in the real world.

[0268] Among them, the current display information includes one or more of the electronic device's current time information, location information, motion status information, interface virtual light source information, component height information, component material information, and component transparency information.

[0269] In some embodiments, the developer may preconfigure the illumination information in the electronic device. When the electronic device determines that the interface display needs to be refreshed, it may obtain the locally stored illumination information and the current display information.

[0270] S4403: The electronic device performs lighting rendering according to the lighting information and current display information.

[0271] In some embodiments, after acquiring the lighting information and the current display information, the electronic device performs lighting rendering on the first component in the next frame of the image to be displayed.

[0272] Some examples include Figure 8 As shown, the rendering process of the first component includes border rendering, lighting layer rendering, user interface canvas rendering, surface layer rendering, and shadow bearing layer rendering; wherein, the border rendering is used to draw at least one of the self-luminous edge effect, highlight edge effect, and contour line effect of the first component; the lighting layer is used to receive lighting and complete the lighting drawing corresponding to the first component; the user interface canvas is used to complete the drawing of the style content of the first component; the surface layer is used to draw the display container of the first component; the shadow bearing layer is used to carry the layer shadow and the overflow lighting drawing.

[0273] In some embodiments, the electronic device outputs visual features based on the light source information, rendering equations, and basic physical properties indicated by the lighting information and current display information, and the visual features include one or more of color information, projection information, and physical material information.

[0274] Optionally, the light source information includes the lighting type and lighting properties. The basic physical properties include one or more of base color, metalness, roughness, reflectivity, self-illumination, normal information, refractive index, transparency, absorptivity, and transmittance.

[0275] For example, for some specific application scenarios, or to achieve the developer's design goals, components need to write different physical property values. Therefore, electronic devices need to support developers or users to customize physical properties. For example, electronic devices provide API interfaces for setting different physical properties of components. Alternatively, electronic devices set themes corresponding to different physical properties, and update the physical properties of components for different themes. Alternatively, the electronic device sets the physical properties of the component in the control declaration or layout description. In some embodiments, the light and shadow effects include shadow rendering effects, and the shadow rendering effects correspond to one or more of the height of the first component, the position of the light source, the shape of the first component, and the material of the first component. Among them, the shape of the first component is a regular shape or a special shape.

[0276] For example, the shadow rendering effect is related to the height of the first component. Fig.26As shown in (a) and (b), the shadow of the first component includes the umbra and penumbra. For the same light source (same position, height, etc.), the shadow effects produced by the first components at different heights are different. For example, the umbra of the first component appears to be more solid, while the penumbra appears to be more virtual. The higher the height of the first component and the closer it is to the light source, the larger its penumbra is and the more obvious the shadow display effect is. Fig.26 As shown in (c), during the rendering process of the first component, the electronic device determines the corresponding shadow display effect according to the height of the first component.

[0277] For example, the shadow rendering effect is related to the position of the light source. Fig. 27 As shown, based on the direction angle shown in the xy coordinate system, the positional relationship between the light source and the first component (such as the UI component) is indicated. Fig. 27 As shown in (a), the light source 271 is located at a 45-degree direction from the UI component 272 (i.e., the light source is located at the upper right of the UI component), and the rendering system outputs the shadow of the UI component 272 as a 45-degree left shadow (i.e., the shadow is located at the lower left of the UI component). Fig. 27 As shown in (b), the light source 271 is located at a 90-degree direction to the UI component 272 (i.e., the light source is located directly above the UI component), and the shadow output by the rendering system for the UI component 272 is a 90-degree bottom shadow (i.e., the shadow is located directly below the UI component). Fig. 27 As shown in (c), the light source 271 is located at 135 degrees to the UI component 272 (i.e., the light source is located at the upper left of the UI component), and the rendering system outputs the shadow of the UI component 272 as a 135-degree right shadow (i.e., the shadow is located at the lower right of the UI component).

[0278] For another example, the shadow rendering effect is related to the shape of the first component. The rendering system can output shadow display effects of different shapes according to the different shapes of the first component. In addition to supporting the rendering of circular and rounded rectangular shadows, the rendering system also supports the rendering of various special-shaped shadows. For example, Fig.28 As shown in (a), the rendering system completes the shadow rendering of the circular UI component. Fig.28 As shown in (b), the rendering system completes the shadow rendering of the square UI component. Fig.28 As shown in (c), the rendering system completes the shadow rendering of the bird-shaped (alien) UI component.

[0279] For another example, the shadow rendering effect is related to the material of the first component. For example, if the translucent material supports color projection, then the shadow color is similar to the color of the first component of the translucent material. Fig.29As shown, the color of the first component to be rendered by the electronic device is blue, and the material of the first component is a translucent material. Then, the electronic device can draw a light blue shadow according to the color and material of the first component, so that the shadow display effect is more consistent with the shadow effect in the real physical world, so as to enhance the user experience.

[0280] In some examples, the electronic device determines the shadow offset and direction to obtain the shadow rendering effect based on the light source coordinates and the height of the first component indicated by the lighting information and the current display information.

[0281] In some other examples, the electronic device copies the first component to create a shadow map, and blurs the shadow map to obtain a shadow rendering effect.

[0282] In some other examples, a shadow rendering effect is obtained through a shadow rendering algorithm according to the lighting information and the current display information.

[0283] In this way, electronic devices can bring different shadow rendering effects according to different light sources and physical properties, enrich the GUI interface display, and enhance the user experience.

[0284] In addition, when the shape of the first component is relatively complex, the electronic device may also combine the lighting information and the current display information to generate the light and shadow effects of the first component of the complex shape to enrich the display of the interface.

[0285] S4404. The electronic device displays a second interface, and the first component in the second interface has a first light and shadow effect formed after light rendering.

[0286] In some embodiments, after performing lighting rendering on the first component, the second electronic device can make the first component displayed in the refreshed interface have the rendering effect formed after the lighting rendering.

[0287] In some examples, the lighting effects include shadow rendering effects.

[0288] In other examples, when the material of the first component is a translucent material, the light and shadow effects include one or more of frosted glass effect, rainbow glass effect, and acrylic glass effect. In this way, when the first component is a translucent material, the electronic device can also combine the illumination information and the current display information to generate the light and shadow effects of the first component of the translucent material to enrich the display of the interface.

[0289] In this way, the electronic device combines the lighting information and the current display information to provide the user with a GUI lighting effect that better meets the user's needs, thereby improving the user's experience.

[0290] In addition, humans are subconsciously familiar with the lighting system, which can avoid increasing learning costs and has high user acceptance. Moreover, the effect of the lighting system is consistent with human cognition, which improves the sophistication and aesthetics of the GUI interface and is easy to gain user recognition.

[0291] Combination of the above Figure 6-Figure 44 The light and shadow effect display method provided by the embodiment of the present application is described in detail. Fig.45 The electronic device provided by the embodiments of the present application is described in detail.

[0292] In one possible design, Fig.45 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Fig.45 As shown, the electronic device 4500 may include: a transceiver unit 4501, a processing unit 4502, and a display unit 4503. The electronic device 4500 may be used to implement the functions of the electronic device involved in the above method embodiment.

[0293] Optionally, the transceiver unit 4501 is used to support the electronic device 4500 to execute Fig.44 S4401 in.

[0294] Optionally, the processing unit 4502 is used to support the electronic device 4500 to execute Fig.44 S4401, S4402 and S4403 in it.

[0295] Optionally, the display unit 4503 is used to support the electronic device 4500 to execute Fig.44 S4404 in.

[0296] Among them, the transceiver unit may include a receiving unit and a sending unit, and may be implemented by a transceiver or a transceiver-related circuit component, and may be a transceiver or a transceiver module. The operations and / or functions of each unit in the electronic device 4500 are respectively to implement the corresponding process of the light and shadow effect display method described in the above method embodiment. All relevant contents of each step involved in the above method embodiment can be referred to the functional description of the corresponding functional unit, and for the sake of brevity, they will not be repeated here.

[0297] Optionally, Fig.45 The electronic device 4500 shown may also include a storage unit ( Fig.45 (not shown), the storage unit stores a program or instruction. When the transceiver unit 4501, the processing unit 4502 and the display unit 4503 execute the program or instruction, Fig.45 The electronic device 4500 shown can execute the light and shadow effect display method described in the above method embodiment.

[0298] Fig.45The technical effects of the electronic device 4500 shown can refer to the technical effects of the light and shadow effect display method described in the above method embodiment, and will not be repeated here.

[0299] In addition to being in the form of electronic device 4500, the technical solution provided in the present application may also be a functional unit or chip in the electronic device, or a device used in conjunction with the electronic device.

[0300] An embodiment of the present application also provides a chip system, including: a processor, the processor is coupled to a memory, the memory is used to store programs or instructions, when the program or instructions are executed by the processor, the chip system implements the method in any of the above method embodiments.

[0301] Optionally, the processor in the chip system may be one or more. The processor may be implemented by hardware or by software. When implemented by hardware, the processor may be a logic circuit, an integrated circuit, etc. When implemented by software, the processor may be a general-purpose processor implemented by reading software code stored in a memory.

[0302] Optionally, the memory in the chip system may also be one or more. The memory may be integrated with the processor or may be separately arranged with the processor, which is not limited in the embodiments of the present application. Exemplarily, the memory may be a non-transient processor, such as a read-only memory ROM, which may be integrated with the processor on the same chip or may be arranged on different chips respectively. The embodiments of the present application do not specifically limit the type of memory and the arrangement of the memory and the processor.

[0303] Exemplarily, the chip system can be a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on chip (SoC), a central processor unit (CPU), a network processor (NP), a digital signal processor (DSP), a microcontroller unit (MCU), a programmable logic device (PLD) or other integrated chips.

[0304] It should be understood that each step in the above method embodiment can be completed by an integrated logic circuit of hardware in a processor or by instructions in the form of software. The method steps disclosed in the embodiments of the present application can be directly embodied as being executed by a hardware processor, or by a combination of hardware and software modules in a processor.

[0305] An embodiment of the present application also provides a computer-readable storage medium, in which a computer program is stored. When the computer program is run on a computer, the computer executes the above-mentioned related steps to implement the light and shadow effect display method in the above-mentioned embodiment.

[0306] The embodiment of the present application also provides a computer program product. When the computer program product is run on a computer, the computer is caused to execute the above-mentioned related steps to implement the light and shadow effect display method in the above-mentioned embodiment.

[0307] In addition, an embodiment of the present application further provides a device. The device may be a component or a module, and the device may include one or more processors and a memory connected to each other. The memory is used to store a computer program. When the computer program is executed by one or more processors, the device performs the light and shadow effect display method in the above-mentioned method embodiments.

[0308] Among them, the device, computer-readable storage medium, computer program product or chip provided in the embodiments of the present application are used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding methods provided above, and will not be repeated here.

[0309] The steps of the method or algorithm described in conjunction with the disclosure of the embodiments of the present application can be implemented in hardware or by executing software instructions by a processor. The software instructions can be composed of corresponding software modules, and the software modules can be stored in random access memory (RAM), flash memory, read only memory (ROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, mobile hard disks, CD-ROMs or any other form of storage medium known in the art. An exemplary storage medium is coupled to a processor so that the processor can read information from the storage medium and can write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an application specific integrated circuit (ASIC).

[0310] Through the description of the above implementation methods, those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above functional modules is used as an example. In practical applications, the above functions can be assigned to different functional modules as needed; that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. The specific working process of the system, device and unit described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0311] In the several embodiments provided in the present application, it should be understood that the disclosed method can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the modules or units is only a logical function division, and there may be other division methods in actual implementation; for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of modules or units, which can be electrical, mechanical or other forms.

[0312] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0313] Computer-readable storage media include, but are not limited to, any of the following: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, and other media that can store program codes.

[0314] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for displaying light and shadow effects, It is characterized in that Applied to electronic equipment, the method comprises: Detecting a first operation of the user, and determining to refresh the first interface being displayed; Get lighting information and current display information; Performing lighting rendering according to the lighting information and the current display information; A second interface is displayed, wherein the first component in the second interface has the light and shadow effect formed after the light rendering.

2. The method according to claim 1, It is characterized in that The lighting information is a preconfigured global lighting definition, which is used to indicate one or more of the physical lighting principles, lighting types, lighting properties, scene lighting effects, lighting models, lighting materials, and shadow properties in the real world.

3. The method according to claim 1 or 2, It is characterized in that The current display information includes one or more of the current time information, location information, motion state information, interface virtual light source information, component height information, component material information, and component transparency information of the electronic device.

4. The method according to any one of claims 1 to 3, It is characterized in that The rendering process of the first component includes border rendering, lighting layer rendering, user interface canvas rendering, surface layer rendering, and shadow bearing layer rendering; wherein the border rendering is used to draw at least one of the self-luminous edge effect, highlight edge effect, and contour line effect of the first component; the lighting layer is used to receive lighting and complete the lighting drawing corresponding to the first component; the user interface canvas is used to complete the drawing of the style content of the first component; the surface layer is used to draw the display container of the first component; the shadow bearing layer is used to carry layer shadows and overflow lighting drawing.

5. The method according to any one of claims 1 to 4, It is characterized in that The performing illumination rendering according to the illumination information and the current display information includes: According to the light source information, rendering equations and basic physical properties indicated by the illumination information and the current display information, visual features are output, where the visual features include one or more of color information, projection information and physical material information.

6. The method according to claim 5, It is characterized in that The light source information includes the lighting type and lighting properties; the basic physical properties include one or more of base color, metallicity, roughness, reflectivity, self-luminescence, normal information, refractive index, transparency, absorptivity, and transmittance.

7. The method according to any one of claims 1 to 6, It is characterized in that The light and shadow effects include shadow rendering effects, which correspond to one or more of the height of the first component, the position of the light source, the shape of the first component, and the material of the first component. The shape of the first component is a regular shape or an irregular shape.

8. The method according to claim 7, It is characterized in that The performing illumination rendering according to the illumination information and the current display information includes: Determine a shadow offset and direction according to the illumination information, the light source coordinates indicated by the current display information, and the height of the first component, and obtain the shadow rendering effect; or, Duplicate the first component to create a shadow map; Obtain the shadow rendering effect by blurring the shadow map; or, The shadow rendering effect is obtained through a shadow rendering algorithm according to the illumination information and the current display information.

9. The method according to any one of claims 1 to 8, It is characterized in that In the case where the material of the first component is a translucent material, the light and shadow effects include one or more of frosted glass effect, rainbow glass effect, and acrylic glass effect.

10. An electronic device, It is characterized in that include: A processor, a memory and a display screen, wherein the memory and the display screen are coupled to the processor, the memory is used to store computer program code, and the computer program code includes computer instructions. When the processor reads the computer instructions from the memory, the electronic device executes the method as described in any one of claims 1 to 9.

11. A computer-readable storage medium, It is characterized in that The computer-readable storage medium comprises a computer program, and when the computer program is executed on an electronic device, the electronic device is enabled to execute the method according to any one of claims 1 to 9.

12. A computer program product, It is characterized in that When the computer program product is executed on a computer, the computer is caused to execute the method according to any one of claims 1 to 9.

Images