Display processing method and electronic device

Abstract

The present application provides a display processing method and an electronic device. By pre-configuring a mapping relationship between simulation grids and rendering grids, a corresponding rendering grid can be directly acquired on the basis of the mapping relationship after a simulation grid is determined. Thus, a physical computation load is reduced to the greatest extent and an exquisite image quality effect is ensured, and an exquisite animation effect experience is realized. The method comprises: in response to a first operation of a user, an electronic device determining a theme used by same, wherein the theme comprises an elastomer displayed on a screen-off interface; then, the electronic device acquiring a simulation grid corresponding to the elastomer; mapping the simulation grid into a rendering grid on the basis of a mapping relationship; and on the basis of the rendering grid, the electronic device displaying the screen-off interface comprising the elastomer.

Description

Display processing method and electronic equipment

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office on August 2, 2023, with application number 202310974211.4 and application name “Display Processing Method and Electronic Device”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The embodiments of the present application relate to the field of terminal technology, and in particular to a display processing method and an electronic device. Background Art

[0003] With the advancement of terminal technology, electronic devices are using animation effects that combine real-world physical motion to increase user interest. For example, dynamic wallpapers displayed on electronic devices can include objects that move in real-world motion. The key to the performance of physical motion effects lies in real-time simulation.

[0004] Currently, real-time simulation on mobile devices (such as mobile phones and tablets) mainly focuses on rigid bodies and small-scale scenes. However, for elastic bodies and large-scale scenes, it is difficult to ensure the display effect of physical motion effects due to issues such as computational complexity.

[0005] Summary of the Invention

[0006] To address the aforementioned technical issues, this application provides a display processing method and electronic device. The technical solution provided by this application pre-configures a mapping relationship between simulation grids and rendering grids. Once the simulation grid is determined, the corresponding rendering grid can be directly retrieved based on the mapping relationship. This ensures a refined image quality while minimizing the physical computation load, resulting in a refined motion experience.

[0007] In order to achieve the above technical objectives, this application provides the following technical solutions:

[0008] In a first aspect, a display processing method is provided for use in an electronic device. The method comprises: in response to a first user operation, determining a theme used by the electronic device, the theme including an elastic body displayed on an off-screen interface; obtaining a simulation mesh corresponding to the elastic body; mapping the simulation mesh to a rendering mesh based on a mapping relationship; and displaying an off-screen interface including the elastic body based on the rendering mesh.

[0009] Thus, compared to existing subdivision shaders that can only achieve subdivision by modifying interpolated vertices so that the generated surface follows the mesh normals, resulting in additional subdivision processing load, the display processing method provided by the embodiment of the present application can directly map the simulation mesh to the rendering mesh in one go based on the offline preset mapping relationship, thereby ensuring the subdivision effect while reducing the subdivision processing load.

[0010] Furthermore, since the mapping between the simulation mesh and the rendering mesh is established offline, it is possible to obtain a more delicate and smooth rendering mesh by modifying the original vertices of the simulation mesh.

[0011] According to the first aspect, obtaining a simulation grid corresponding to an elastomer includes obtaining a first simulation duration for simulating the elastomer by a first platform of an electronic device and a second simulation duration for simulating the elastomer by a second platform of the electronic device. Based on the first simulation duration and the second simulation duration, calling the first platform and / or the second platform to simulate the elastomer.

[0012] According to the first aspect, or any implementation of the first aspect above, based on the first simulation duration and the second simulation duration, calling the first platform and / or the second platform to simulate the elastomer includes: when the first simulation duration is less than the second simulation duration, calling the first platform to simulate the elastomer. Alternatively, when the second simulation duration is less than the first simulation duration, calling the second platform to simulate the elastomer. Alternatively, when the third simulation duration is less than the first simulation duration and the second simulation duration, calling the first platform and the second platform to simulate the elastomer, the third simulation duration being the simulation duration required for calling the first platform and the second platform to perform cooperative simulation of the elastomer, determined based on the first simulation duration and the second simulation duration.

[0013] For example, the first platform is a CPU platform, and the second platform is a GPU platform.

[0014] In this way, compared to the current situation where programmers can only specify whether to compile a program into CPU code or GPU code during the development phase, the display processing method provided by the embodiment of the application can dynamically switch the platform used for calculation during runtime, automatically select the platform with better performance based on different hardware platforms and calculation scales in different business scenarios, and achieve better physical simulation results.

[0015] According to the first aspect, or any implementation of the first aspect above, before obtaining a first simulation duration for simulating the elastomer on the first platform and a second simulation duration for simulating the elastomer on the second platform, the method further includes: obtaining N operators corresponding to the subject, where N is a positive integer. Obtaining a first calculation duration for executing each of the N operators on the first platform, a second calculation duration for executing each of the N operators on the second platform, and a data transmission duration for each of the N operators from the first platform to the second platform.

[0016] For example, the electronic device obtains N operators corresponding to the topic, namely gradient calculation, integral calculation, multiple calculation, and Fourier transform calculation. Then, the electronic device can determine the first calculation duration of the first platform performing the gradient calculation, integral calculation, multiple calculation, and Fourier transform calculation, respectively; the second calculation duration of the second platform performing the gradient calculation, integral calculation, multiple calculation, and Fourier transform calculation, respectively; and determine the data transmission duration from the first platform to the second platform.

[0017] In this way, before selecting a platform, the electronic device first determines the actual calculation time of each operator on different platforms, so that accurate platform scheduling can be achieved based on the actual calculation time, thereby realizing adaptive hardware platforms applied to different devices.

[0018] According to the first aspect, or any implementation of the first aspect above, obtaining the first simulation duration for simulating the elastomer by the first platform and the second simulation duration for simulating the elastomer by the second platform includes: obtaining the number of operators of the N operators corresponding to the elastomer. According to the number of operators and the first calculation duration, obtaining the first duration for the first platform to respectively execute the N operators; and according to the number of operators, the second calculation duration, and the data transmission duration, obtaining the second duration for the second platform to respectively execute the N operators, the second duration including the cumulative transmission duration of each of the N operators from the first platform to the second platform. According to the first duration, determining the first simulation duration. According to the second duration, determining the second simulation duration.

[0019] For example, the first platform performs a gradient calculation for a first calculation duration a, the second platform performs a gradient calculation for a second calculation duration b, the first platform transmits the data for a gradient calculation to the second platform for a duration c, and the elastic body corresponds to d gradient calculations. Therefore, the first platform performs a gradient calculation for a first duration a*d, and the second platform performs a gradient calculation for a second duration (b+c)*d. Here, a, b, and c are positive numbers, and d is a positive integer.

[0020] Exemplarily, the electronic device obtains that the N operators included in the subject are gradient calculation, integral calculation, multiple calculation, and Fourier transform calculation. And the first operator duration and second operator duration for calculating various operators by the first platform and the second platform, as well as the number of various operators, have been determined. Then, for example, the electronic device determines that the first duration for gradient calculation by the first platform is 3 milliseconds, and the second duration for gradient calculation by the second platform is 1 millisecond; the first duration for integral calculation by the first platform is 3.5 milliseconds, and the second duration for integral calculation by the second platform is 1 millisecond; the first duration for multiple calculation by the first platform is 10 milliseconds, and the second duration for integral calculation by the second platform is 1 millisecond. Then, the electronic device can determine that the first simulation duration for calling the first platform to simulate the elastomer is 16.5 milliseconds (3 milliseconds + 3.5 milliseconds + 10 milliseconds), and the first simulation duration for calling the second platform to simulate the elastomer is 3 milliseconds (1 millisecond + 1 millisecond + 1 millisecond). Then, based on the second simulation duration being less than the first simulation duration, the electronic device can determine to call the second platform to simulate the elastomer to obtain a simulation grid.

[0021] According to the first aspect, or any implementation of the first aspect above, the method further includes: determining that a first duration corresponding to a first type of operator among the N types of operators is less than a second duration, and invoking the first platform to execute the first type of operator. Determining that a second duration corresponding to a second type of operator among the N types of operators is less than the first duration, and invoking the second platform to execute the second type of operator.

[0022] For example, the electronic device determines that the first duration for the gradient calculation of the first platform is 3 milliseconds, and the second duration for the gradient calculation of the second platform is 1 millisecond; the first duration for the integral calculation of the first platform is 3.5 milliseconds, and the second duration for the integral calculation of the second platform is 1 millisecond; the first duration for the multiple calculation of the first platform is 1 millisecond, and the second duration for the integral calculation of the second platform is 10 milliseconds. Then, the electronic device can determine that the first type of operator is the multiple calculation, the second type of operator is the gradient calculation and the integral calculation, and the sum of the duration for the first platform to execute the first type of operator and the duration for the second platform to execute the second type of operator is 3 milliseconds (1 millisecond + 1 millisecond + 1 millisecond). Then, the electronic device can determine to call the first platform to perform the multiple calculation, and call the second platform to perform the gradient calculation and the integral calculation. In this way, through the cooperation of the first platform and the second platform, the simulation of the elastomer can be completed in a shorter time and the simulation grid can be output, thereby improving the simulation efficiency.

[0023] In this way, the electronic device pre-evaluates the duration of simulation of the elastomer by the first platform and / or the second platform, and then determines the calling method that can achieve better performance based on the evaluated duration, and calls the corresponding platform to simulate the elastomer, thereby improving the simulation efficiency while ensuring that the simulation grid can be output.

[0024] According to the first aspect, or any implementation of the first aspect above, the method further includes: detecting a second user operation on the screen-off interface, invoking the first platform and / or the second platform to simulate the elastomer; displaying a first animation of the elastomer transitioning from the screen-off interface to the lock screen interface; and displaying the lock screen interface including the elastomer.

[0025] According to the first aspect, or any implementation of the first aspect above, the method further includes: detecting a third user operation on the lock screen, invoking the first platform and / or the second platform to simulate the elastic body; displaying a second animation of the elastic body moving from the lock screen to the desktop; and displaying the desktop.

[0026] According to the first aspect, or any implementation method of the first aspect above, in the process of displaying the first animation, displaying the lock screen interface including the elastomer, and displaying the second animation, the platform called by the electronic device is the same as the platform called by the electronic device in the process of displaying the screen-off interface including the elastomer.

[0027] In this way, after the electronic device assigns a platform for simulating the elastomer according to the elastomer included in the theme, it can continuously use the same platform to simulate the elastomer displayed in multiple interfaces included in the same theme. Based on the consistency of the display effect across the interfaces at different stages of the theme, it can avoid unnecessary repeated platform selection while maintaining display quality and display processing efficiency.

[0028] According to the first aspect, or any implementation of the first aspect above, before determining the theme used by the electronic device in response to the user's first operation, the method also includes: obtaining and saving a mapping relationship, the mapping relationship including a mapping matrix for converting a simulation grid into a rendering grid.

[0029] For example, during the development phase, programmers may create a mapping relationship between the simulation mesh and the rendering mesh offline through an offline topology construction module, and directly preset the mapping relationship into the electronic device.

[0030] In this way, based on the mapping relationship, the electronic device can map the simulation mesh updated by the physical simulation to the high-resolution rendering mesh at one time per frame to reduce the subdivision processing load.

[0031] In a second aspect, an electronic device is provided. The electronic device includes: a processor, a memory, and a display screen, the memory and the display screen are coupled to the processor, the memory is used to store computer program code, the computer program code includes computer instructions, and when the processor reads the computer instructions from the memory, the electronic device executes: in response to a first operation of the user, determining the theme used by the electronic device, the theme including an elastomer displayed on the off-screen interface. Obtain a simulation grid corresponding to the elastomer. According to the mapping relationship, map the simulation grid to a rendering grid. According to the rendering grid, display an off-screen interface including the elastomer.

[0032] According to a second aspect, obtaining a simulation grid corresponding to an elastomer includes obtaining a first simulation duration for simulating the elastomer by a first platform of an electronic device and a second simulation duration for simulating the elastomer by a second platform of the electronic device. Based on the first simulation duration and the second simulation duration, calling the first platform and / or the second platform to simulate the elastomer.

[0033] According to the second aspect, or any implementation of the second aspect above, based on the first simulation duration and the second simulation duration, calling the first platform and / or the second platform to simulate the elastomer includes: when the first simulation duration is less than the second simulation duration, calling the first platform to simulate the elastomer. Alternatively, when the second simulation duration is less than the first simulation duration, calling the second platform to simulate the elastomer. Alternatively, when the third simulation duration is less than the first simulation duration and the second simulation duration, calling the first platform and the second platform to simulate the elastomer, the third simulation duration being the simulation duration required for calling the first platform and the second platform to perform cooperative simulation of the elastomer, determined based on the first simulation duration and the second simulation duration.

[0034] According to the second aspect, or any implementation of the second aspect above, when the processor reads the computer instructions from the memory, it further causes the electronic device to execute: obtaining N operators included in the subject, where N is a positive integer; obtaining a first calculation duration for executing each of the N operators on the first platform, a second calculation duration for executing each of the N operators on the second platform, and a data transmission duration for each of the N operators from the first platform to the second platform.

[0035] According to the second aspect, or any implementation of the second aspect above, obtaining the first simulation duration for simulating the elastomer by the first platform and the second simulation duration for simulating the elastomer by the second platform includes: obtaining the number of operators of the N operators corresponding to the elastomer. According to the number of operators and the first calculation duration, obtaining the first duration for the first platform to respectively execute the N operators; and according to the number of operators, the second calculation duration, and the data transmission duration, obtaining the second duration for the second platform to respectively execute the N operators, the second duration including the cumulative transmission duration of each of the N operators from the first platform to the second platform. According to the first duration, determining the first simulation duration. According to the second duration, determining the second simulation duration.

[0036] According to the second aspect, or any implementation of the second aspect above, when the processor reads the computer instructions from the memory, the electronic device is further caused to execute: determining that a first duration corresponding to a first type of operator among N types of operators is less than a second duration, and invoking the first platform to execute the first type of operator; determining that a second duration corresponding to a second type of operator among the N types of operators is less than the first duration, and invoking the second platform to execute the second type of operator.

[0037] According to the second aspect, or any implementation of the second aspect above, when the processor reads the computer instructions from the memory, the electronic device is further caused to execute the following steps: upon detecting a second user operation on the screen-off interface, invoking the first platform and / or the second platform to simulate the elastomer; displaying a first animation of the elastomer transitioning from the screen-off interface to the lock screen interface; and displaying the lock screen interface including the elastomer.

[0038] According to the second aspect, or any implementation of the second aspect above, when the processor reads computer instructions from the memory, the electronic device is further caused to: upon detecting a third user operation on the lock screen, invoke the first platform and / or the second platform to simulate the elastic body; display a second animation of the elastic body moving from the lock screen to the desktop; and display the desktop.

[0039] According to the second aspect, or any implementation method of the above second aspect, in the process of displaying the first animation, displaying the lock screen interface including the elastomer, and displaying the second animation, the platform called by the electronic device is the same as the platform called by the electronic device in the process of displaying the screen-off interface including the elastomer.

[0040] According to the second aspect, or any implementation of the second aspect above, when the processor reads computer instructions from the memory, it also enables the electronic device to execute: obtaining and saving a mapping relationship, where the mapping relationship includes a mapping matrix for converting a simulation mesh into a rendering mesh.

[0041] In a third aspect, an electronic device is provided, which has the function of implementing the display processing method described in the first aspect and any possible implementation thereof. This function can be implemented by hardware, or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above-mentioned functions.

[0042] In a fourth aspect, a computer-readable storage medium is provided. The computer-readable storage medium stores a computer program (also referred to as instructions or code), which, when executed by an electronic device, causes the electronic device to perform the method of the first aspect or any one of the embodiments of the first aspect.

[0043] In a fifth aspect, a computer program product is provided. When the computer program product is run on an electronic device, the electronic device executes the method of the first aspect or any one of the embodiments of the first aspect.

[0044] The technical effects of the aforementioned aspects can be referenced with each other and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] FIG1 is a schematic diagram of a physics engine based on a Cuda computing platform provided in an embodiment of the present application;

[0046] FIG2 is a schematic diagram of a physics engine flow based on a Cuda computing platform provided in an embodiment of the present application;

[0047] FIG3 is a schematic diagram of a flow chart of a subdivision shader solution provided in an embodiment of the present application;

[0048] FIG4 is a schematic diagram of the subdivision principle provided in an embodiment of the present application;

[0049] FIG5 is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application;

[0050] FIG6 is a schematic diagram of an elastic body scenario provided in an embodiment of the present application;

[0051] FIG7 is a second schematic diagram of an elastic body scenario provided in an embodiment of the present application;

[0052] FIG8 is a third schematic diagram of an elastic body scenario provided in an embodiment of the present application;

[0053] FIG9 is a flow chart of a display processing method according to an embodiment of the present application;

[0054] FIG10 is a functional block diagram of a display processing method provided by an embodiment of the present application;

[0055] FIG11 is a schematic diagram of the interaction between modules in the display processing method provided in an embodiment of the present application;

[0056] FIG12 is a schematic diagram of the load assessment-automatic scheduling principle provided by an embodiment of the present application;

[0057] FIG13 is a schematic diagram of an offline topology acceleration structure provided in an embodiment of the present application;

[0058] FIG14 is a schematic diagram of a simulation grid mapping rendering grid scene provided by an embodiment of the present application;

[0059] FIG15 is a schematic diagram of a rendering grid effect provided by an embodiment of the present application;

[0060] FIG16 is a second flow chart of a display processing method according to an embodiment of the present application;

[0061] FIG17 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0062] The technical solutions in the embodiments of the present application are described below in conjunction with the drawings in the embodiments of the present application. Among them, in the description of the embodiments of the present application, the terms used in the following embodiments are only for the purpose of describing specific embodiments, and are not intended to be used as limitations on the present application. As used in the specification and appended claims of the present application, the singular expressions "one", "a kind of", "said", "above", "the" and "this" are intended to include expressions such as "one or more", unless there is a clear indication to the contrary in the context. It should also be understood that in the following embodiments of the present application, "at least one", "one or more" refer to one or more (including two).

[0063] References to "one embodiment" or "some embodiments" etc. described in this specification mean that the specific features, structures or characteristics described in conjunction with the embodiment are included in one or more embodiments of the present application. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. appearing in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in another way. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in another way. The term "connected" includes direct and indirect connections, unless otherwise stated. "First" and "second" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated.

[0064] In the embodiments of this application, words such as "exemplarily" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "exemplarily" or "for example" in the embodiments of this application should not be interpreted as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplarily" or "for example" is intended to present the relevant concepts in a concrete manner.

[0065] In some embodiments, electronic devices can display animation effects that incorporate real-world physical motion, increasing the fun of the interface display and attracting more user traffic. For example, a dynamic wallpaper displayed by an electronic device may include animations that emulate the movement of objects in real-world physics. The key to the performance of these physics-based animations lies in real-time physics simulation. Physics simulation, also known as simulation, emulation, or physical simulation, involves predicting the behavior or outcome of a real-world or physical system using a computer.

[0066] In some examples, the simulation results are calculated by assigning real physical properties to objects in the computer. The physics engine has the ability to simulate various objects, such as rigid bodies, elastic bodies, fluids, and other simulation results. For example, the physics engine calculates the movement, rotation, and collision response of objects by assigning real physical properties to objects. Among them, a rigid body is a concept in physics, a solid with a finite size that can be displaced or rotated under the action of external forces but will not deform. An elastic body (elastic body / soft body), also known as a soft body or a flexible body, is a concept in physics, a solid with a finite size that can be displaced, rotated, or deformed under the action of external forces and can restore its shape after the external force is removed. Fluid is a concept in physics, a general term for liquids and gases, an object that will continuously deform under any small shear force.

[0067] The performance of physical simulation depends on the development of the physics engine. The mainstream physics engines in the industry have experienced a development from simple rigid body scenes to complex fluid scenes, and from small-scale scenes to large-scale scene support. Table 1 below illustrates several mainstream physics engines in the industry, their corresponding back-end platforms, and the types of devices they are mainly used for. Among them, Havok has a large ecosystem and PhysX has full functions, but these powerful physics engines are mainly used on PCs and consoles. Mobile terminals (such as mobile phones and tablets) have fewer real-time simulation support functions, and currently focus on rigid body or simple cloth simulation scenes. They are almost only based on central processor unit (CPU) calculations, and basically do not support graphics processing unit (GPU) calculations. There is a huge gap in performance optimization for 3D sophisticated physics simulations.

[0068] Table 1

[0069] In some examples, the compute unified device architecture (Cuda) and open computing language (OpenCL) are computing platforms that are currently widely used in the industry. They support the implementation of GPU and CPU versions of various algorithms, enabling GPUs to solve complex computing problems.

[0070] It should be understood that the physical engines and computing platforms illustrated in Table 1 are merely exemplary. Other physical engines and computing platforms may also be applicable to the display processing method provided in the embodiments of the present application, which will not be described in detail below.

[0071] In some embodiments, the computing power comparison between the CPU and GPU of different devices varies greatly. The CPU has strong single-core computing power and fast startup, while the GPU has strong parallel computing power and has additional consumption in the preparation stage of operation startup.

[0072] In some examples, as shown in Table 1 above, current mainstream physics engines support both CPUs and GPUs. Taking the Cuda computing platform as an example, as shown in Figure 1, the device provides both CPU and GPU implementations of various algorithms. Programmers must specify whether their programs should be compiled into CPU or GPU code during the development phase. The upper layer, the physics engine implementation layer, includes the basic components of rigid body physics, fluid physics, and elastic body physics, as well as their corresponding dynamics simulation algorithms. For example, as shown in Figure 2, programmers estimate computational requirements based on the target scenario and computational scale and specify during development whether the code should be compiled for GPU or CPU. The physics engine implementation layer then calls the corresponding underlying computational algorithms to implement the physics simulation functionality. For example, if the programmer selects a GPU backend platform, computations are performed on the GPU during the data transfer and preparation phase; if the programmer selects a CPU backend platform, computations are performed directly on the CPU during the data transfer and preparation phase.

[0073] In some examples, GPU and CPU calculations at different scales produce different performance results. Alternatively, performance can be measured using frame rate. This metric indicates the number of frames displayed in the current animation, measured in frames per second (FPS). A higher frame rate indicates better performance.

[0074] For example, as shown in Table 2 below, when a scene consists of 10*10*10 box-shaped objects falling, the CPU backend platform performs faster. However, when the scale increases to 11*11*11 box-shaped objects, the GPU backend platform performs faster. As the scale of physics calculations increases, the speed gap between the GPU and CPU backend platforms widens.

[0075] However, since the backend platform selection method only supports programmers to specify during the development phase, and programmers in the development phase cannot predict the specific business scenarios during the subsequent user use of the device, the computing backend platform specified by the programmer may not provide the optimal display performance for the device.

[0076] Table 2

[0077] Based on this, the present application provides a display processing method in which the physics engine computing platform can adapt to different computing devices and different business scenarios, adaptively selecting a CPU backend platform or a GPU backend platform. This allows complex physics calculations to be performed in different configurations and scenarios, achieving better computing performance.

[0078] In some examples, the high-quality and refined effects of the display screen require high model resolution. Real-time physical simulation needs to respond to the motion state in each frame of the display screen. The simulation performance is strongly related to the scale of the grid calculation. In addition, while ensuring image quality, the physical calculation load needs to be reduced. The current common solution in the industry is to separate the model grid into a simulation grid and a rendering grid. Among them, in computer simulation calculations, objects represented continuously need to be converted into simulation grids represented in a discrete manner. When the computer presents an object on the screen, the objects represented continuously need to be converted into rendering grids represented in a discrete manner. Among them, the simulation grid has a lower resolution, and the rendering grid has a higher resolution. By converting the simulation grid into a rendering grid, the resolution is improved, and the simulation grid is output first and then converted into the rendering grid to achieve high resolution while reducing the computing load.

[0079] For example, a method for implementing a separate simulated rendering mesh using a tessellation shader is shown in Figure 3. A tessellation shader is a computer program used for special effects production and is also a shader that runs on a GPU. For example, in a business scenario, a tessellation shader divides a simple mesh into finer meshes by calculating a specific function.

[0080] As shown in Figure 3, after the application passes vertex and image data to the vertex buffer in the GPU's video memory, the vertex buffer passes the vertex array to the vertex shader, which performs operations such as rotation, scaling, and coordinate conversion. The subdivision shader then converts the simulated mesh processed by the vertex shader into a rendering mesh and passes the rendering mesh to the geometry shader. The geometry shader assembles primitives based on the connectivity between defined vertices and determines the graphics drawing method. The geometry shader then rasterizes the primitives, iterating over each screen pixel to determine whether it is covered by the primitive and passing it to the fragment shader to shade each pixel. The fragment shader then outputs the results to the frame buffer. Finally, the contents of the frame buffer are displayed on the screen.

[0081] In some examples, the specific working process of the subdivision shader is analyzed using the creation engine tool Unity as an example. Unity's built-in subdivision shader has multiple implementations, but all are limited to triangular meshes. For example, the subdivision shader subdivides a fixed amount of surface, directly adding a fixed amount of surface tessellation to each mesh cell. Distance-based tessellation changes the surface tessellation level based on the distance between the mesh cell and the virtual camera, e.g., the closer the distance, the higher the level of tessellation. For example, the distance is determined based on the depth information of the mesh cell. Edge-length-based tessellation calculates the subdivision level based on the edge length of the screen triangle, with mesh cells with longer edges having higher levels of tessellation. Phong tessellation can modify the position of the generated subdivision surface so that the generated subdivision surface matches the mesh normal. Phong tessellation can smooth the surface mesh. Phong tessellation is a type of subdivision shader whose operation rule is to modify interpolated vertices so that the generated surface follows the mesh normal. For example, as shown in FIG4(a), the subdivision shader determines three vertices and three edges of a triangle mesh, and subdivides the one triangle mesh into four triangle meshes as shown in FIG4(b) according to the edge lengths.

[0082] In the above process, the higher the subdivision level, the better the final display quality. However, such methods often cannot modify the original mesh vertices; for example, the vertices of a triangle mesh cannot be modified. Furthermore, the high subdivision level required results in a large subdivision processing load.

[0083] Furthermore, subdivision shaders run on the GPU backend platform and depend on the compiler type, resulting in different results for different chips. Therefore, a subdivision shader that achieves high-quality display on device 1 may produce poor display quality when run on device 2. This also means that consistency in display quality across different device types cannot be guaranteed.

[0084] Another solution to reduce the physics computation load while maintaining image quality is to achieve fine mesh division directly on the CPU through loop subdivision or mesh regularization calculations. Loop subdivision is an algorithm that subdivides the mesh on the CPU, modifying interpolated vertices and the original mesh vertices to obtain a finer mesh.

[0085] However, although this method can achieve good rendering effects, it has a very high computational cost and is generally performed as model post-processing, making it difficult to use in real-time animation.

[0086] Based on this, the present application provides a display processing method that pre-configures the mapping relationship between the simulation grid and the rendering grid. After the computing platform outputs the simulation grid, the corresponding rendering grid can be directly obtained based on the mapping relationship. This ensures that the image quality is refined while minimizing the physical computing load, thus achieving a refined UI dynamic effect experience.

[0087] For example, the display processing method provided in the embodiment of the present application is applied to an electronic device 100. The electronic device 100 may be, for example, a mobile phone, a tablet computer, a laptop computer, a smart screen, a wearable device, a vehicle-mounted terminal, an augmented reality (AR) / VR, a PS, an Xbox, a PC, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), an artificial intelligence (AI) device, or other terminal devices. The operating system installed on the electronic device 100 includes but is not limited to The embodiments of the present application do not limit the specific type of the electronic device 100 or the installed operating system.

[0088] FIG5 shows a schematic structural diagram of the electronic device 100 .

[0089] The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc.

[0090] It should be understood that the structures illustrated in the embodiments of the present application do not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown, or may combine or separate certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0091] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). The different processing units may be independent devices or integrated into one or more processors.

[0092] The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of instruction fetching and execution.

[0093] Processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in processor 110 is a cache memory. This memory can store instructions or data that have just been used or are being recycled by processor 110. If processor 110 needs to use the same instruction or data again, it can directly access the memory. This avoids duplicate accesses, reduces processor 110 latency, and thus improves system efficiency.

[0094] In some embodiments, the processor 110 may include one or more interfaces. The interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface.

[0095] The I2C interface is a bidirectional synchronous serial bus that includes a serial data line (SDA) and a serial clock line (SCL). In some embodiments, the processor 110 may include multiple I2C bus lines. The processor 110 may be coupled to a touch sensor, a charger, a flash, a camera 193, and the like via different I2C bus interfaces. For example, the processor 110 may be coupled to a touch sensor via an I2C interface, enabling communication between the processor 110 and the touch sensor via the I2C bus interface, thereby enabling touch functionality of the electronic device 100.

[0096] The MIPI interface can be used to connect the processor 110 to peripheral devices such as the display 194 and the camera 193. MIPI interfaces include the camera serial interface (CSI) and the display serial interface (DSI). In some embodiments, the processor 110 and the camera 193 communicate via the CSI interface to implement the camera function of the electronic device 100. The processor 110 and the display 194 communicate via the DSI interface to implement the display function of the electronic device 100.

[0097] The USB interface 130 is an interface that complies with USB standards and may be a Mini USB interface, a Micro USB interface, a USB Type-C interface, or the like. The USB interface 130 can be used to connect a charger to charge the electronic device 100, or to transfer data between the electronic device 100 and peripheral devices. It can also be used to connect headphones to play audio. This interface can also be used to connect other electronic devices, such as augmented reality devices.

[0098] It is understood that the interface connection relationship between the modules illustrated in the embodiments of the present application is merely an illustrative illustration and does not constitute a structural limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection methods from the above embodiments, or a combination of multiple interface connection methods.

[0099] The charging management module 140 is configured to receive charging input from a charger. The charger can be either a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 can receive charging input from the wired charger via the USB interface 130. In some wireless charging embodiments, the charging management module 140 can receive wireless charging input via the wireless charging coil of the electronic device 100. While charging the battery 142, the charging management module 140 can also provide power to the electronic device via the power management module 141.

[0100] The power management module 141 is used to connect the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, and provides power to the processor 110, the internal memory 121, the display 194, the camera 193, and the wireless communication module 160. The power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle count, and battery health status (leakage, impedance). In some other embodiments, the power management module 141 can also be set in the processor 110. In other embodiments, the power management module 141 and the charging management module 140 can also be set in the same device.

[0101] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0102] Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve antenna utilization. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In other embodiments, the antennas can be used in conjunction with a tuning switch.

[0103] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G applied to the electronic device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the same device as at least some of the modules of the processor 110.

[0104] The modem processor may include a modulator and a demodulator. The modulator is used to modulate the low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After being processed by the baseband processor, the low-frequency baseband signal is passed to the application processor. The application processor outputs the sound signal through the audio device or displays an image or video through the display screen 194. In some embodiments, the modem processor may be an independent device. In other embodiments, the modem processor may be independent of the processor 110 and be provided in the same device as the mobile communication module 150 or other functional modules.

[0105] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the electronic device 100. The wireless communication module 160 can be one or more devices that integrate at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, frequency modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, frequency modulate it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0106] In some embodiments, the antenna 1 of the electronic device 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the electronic device 100 can communicate with a network and other devices through wireless communication technologies. The wireless communication technologies may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-CDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology. The GNSS may include a global positioning system (GPS), a global navigation satellite system (GLONASS), a Beidou navigation satellite system (BDS), a quasi-zenith satellite system (QZSS) and / or a satellite based augmentation system (SBAS).

[0107] Electronic device 100 implements display functionality through a GPU, display screen 194, and an application processor. A GPU is a microprocessor for image processing that connects display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs that execute program instructions to generate or modify display information.

[0108] Display screen 194 is used to display images, videos, etc. Display screen 194 includes a display panel. The display panel can be manufactured using a liquid crystal display (LCD), for example, an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a Mini-LED, a Micro-LED, a Micro-OLED, or a quantum dot light-emitting diode (QLED). In some embodiments, electronic device 100 can include one or N display screens 194, where N is a positive integer greater than one.

[0109] The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, and then passes the electrical signal to the ISP for conversion into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the electronic device 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.

[0110] The external memory interface 120 can be used to connect an external memory card, such as a MicroSD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 via the external memory interface 120 to implement data storage functions. For example, files such as music and videos can be stored on the external memory card.

[0111] The internal memory 121 can be used to store computer executable program codes, which include instructions. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the electronic device 100 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc. The processor 110 executes various functional applications and data processing of the electronic device 100 by running instructions stored in the internal memory 121 and / or instructions stored in a memory provided in the processor.

[0112] The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 170 can also be used to encode and decode audio signals. In some embodiments, the audio module 170 can be set in the processor 110, or some functional modules of the audio module 170 can be set in the processor 110. The electronic device 100 can use the audio module 170, such as music playback and recording. The audio module 170 may include a speaker, a receiver, a microphone, a headphone jack, and an application processor to implement audio functions.

[0113] The sensor module 180 may include a pressure sensor, a gyro sensor, an air pressure sensor, a magnetic sensor, an acceleration sensor, a distance sensor, a proximity light sensor, a fingerprint sensor, a temperature sensor, a touch sensor, an ambient light sensor, a bone conduction sensor, and the like.

[0114] The buttons 190 include a power button, a volume button, and the like. The buttons 190 may be mechanical buttons or touch buttons. The electronic device 100 may receive key inputs and generate key signal inputs related to user settings and function control of the electronic device 100.

[0115] Motor 191 can generate vibration alerts. This can be used for incoming call vibration alerts or touch vibration feedback. Depending on the touch operation applied to different areas of the display 194, motor 191 can also generate different vibration feedback effects. Different application scenarios (e.g., time reminders, receiving messages, alarm clocks, gaming, etc.) can also correspond to different vibration feedback effects.

[0116] The indicator 192 may be an indicator light, which may be used to indicate the charging status, power level changes, messages, missed calls, notifications, etc.

[0117] The SIM card interface 195 is used to connect a SIM card. The SIM card can be connected to and disconnected from the electronic device 100 by inserting or removing the SIM card into or from the SIM card interface 195. The electronic device 100 may support one or N SIM card interfaces, where N is a positive integer greater than one.

[0118] The following describes in detail the display processing method provided in the embodiment of the present application, taking Cuda and OpenCL as computing platforms and CPU and GPU as back-end platforms supported by the physics engine as examples.

[0119] For easier understanding, the following examples illustrate several elastic body animation display scenarios.

[0120] In some scenarios, electronic devices are equipped with a theme function. Through this function, users can set at least one of the display stages of the screen off interface, lock screen interface, and desktop display at a time, allowing users to more quickly and conveniently personalize their electronic devices. Depending on the theme, each stage of the theme display may include elastic body animation, such as displaying elastic body animation on the screen off interface, displaying elastic body animation on the lock screen interface, and displaying elastic body animation during the transition from the screen off interface to the lock screen interface.

[0121] For example, in response to user operations, the electronic device sets a sports ball theme and displays an elastomeric motion effect display scene of an elastomeric ball moving on the off-screen interface and the lock screen interface.

[0122] For example, as shown in (a) of FIG6 , when the electronic device detects a user clicking on the display screen or detects a user approaching through a sensor (such as a camera, etc.) during the black screen process, the screen-off interface as shown in (b) of FIG6 may be displayed, and a function window 61 is displayed on the screen-off interface. In the process of displaying the screen-off interface, the elements (such as elastic balls) in the screen-off interface move on the screen-off interface based on the theme of sports balls. For example, as shown in (b) of FIG6 , the electronic device may display element motion effects in the function window 61, such as displaying the movement of multiple elastic balls in the function window. As shown in (b) to (c) of FIG6 , as time changes, the motion states of the multiple elastic balls displayed in the function window change. For example, the size, motion trajectory, shape, etc. will change due to the collision between the elastic balls. In this way, the electronic device displays the scene through the elastic body motion effect, which increases the fun of the screen-off interface display.

[0123] As another example, as shown in (c) in FIG6 , the electronic device may display the lock screen interface after detecting the user's operation on the lock screen button 62 or detecting the user's double-click operation on the display screen during the process of displaying the screen-off interface. In the process of displaying the lock screen interface, the electronic device, based on the theme of the sports ball, moves on the lock screen interface in the elements in the lock screen interface (such as the elastic ball). Optionally, in order to maintain the consistency of the theme, the motion effects of different stages may be displayed continuously. For example, the electronic device may link the elastic body motion effect of the screen-off interface with the elastic body motion effect of the lock screen interface. In the process of switching the screen-off interface to the lock screen interface in response to the user's operation, the electronic device uses the elastic body motion effect displayed on the screen-off interface as the motion starting point of the elastic body motion effect displayed on the lock screen interface.

[0124] For example, when the electronic device is displaying the screen-off interface as shown in (c) in Figure 6, if it detects the user's instruction to switch to the lock screen interface, the electronic device can control the elastic ball displayed in the function window 61 to move toward the entire display screen. For example, in the lock screen interface shown in (a) in Figure 7, the elastic ball has moved from being displayed in the function window to being displayed outside the function window. The movement of the elastic ball continues until the elastic ball moves to the edge of the display screen as shown in (b) in Figure 7, such as when the elastic ball touches the edge of the lock screen interface. In this way, the electronic device displays the scene through the elastic motion effect, which increases the fun of the lock screen interface display.

[0125] As another example, when the electronic device is displaying the dynamic effect of the elastic ball moving from the function window 61 to the entire display screen, as shown in (a) in FIG8 , the electronic device may stop displaying the current dynamic effect after detecting the user's operation on the lock screen button 62. For example, the electronic device reversely controls the elastic ball to move into the function window 61. When the electronic device switches from the lock screen interface to the off-screen interface in response to the user's operation, as shown in (b) in FIG8 , the elastic ball moving back to the function window 61 is displayed. In this way, the electronic device displays the scene through the elastic body dynamic effect, increases the correlation between the dynamic effects of the interface switching between the off-screen interface and the lock screen interface, and enhances the user experience.

[0126] In other scenarios, electronic devices can also respond to user operations and set the elastic body animation display scene for each interface separately. For example, if the user selects a dynamic wallpaper for the off-screen interface, the dynamic wallpaper will be displayed with an elastic body animation effect. For another example, if the user selects a dynamic wallpaper for the lock screen interface, the lock screen wallpaper will be displayed with an elastic body animation effect. The elastic body animations between different interfaces can be associated or unassociated.

[0127] In some examples, real-time simulation of elastic bodies involves complex collision detection and collision response, which has high computational requirements.

[0128] It should be understood that the display processing method provided in the embodiment of the present application can be applied not only to the dynamic display scene of elastic bodies, but also to the dynamic display scene of rigid bodies, fluids, etc. The embodiment of the present application only uses the dynamic display scene of elastic bodies with higher computing requirements as an example to illustrate the display processing process.

[0129] In addition, the display processing method provided in the embodiment of the present application can be applied to small-scale dynamic effect scenes or large-scale dynamic effect scenes, which will not be described in detail below. Among them, small-scale dynamic effect scenes and large-scale dynamic effect scenes are only relative descriptions of the scale of dynamic effect. For example, if the number of objects executing the dynamic effect does not exceed the threshold, it can be regarded as a small-scale dynamic effect scene; if the number of objects executing the dynamic effect exceeds the threshold, it can be regarded as a large-scale dynamic effect scene.

[0130] In some embodiments, the application is configured with a physics engine to process the motion reactions of objects during the display of dynamic effects. The application can be a native system application or a third-party application. For example, the application can be a lock screen application, a screen saver application, a system UI, a third-party theme application, a third-party wallpaper application, etc. Alternatively, the application can be another game application.

[0131] In some embodiments, as shown in FIG9 , in response to user operations, the application triggers an elastomer scene (i.e., step 901). For example, in response to the user's operation of the lock screen button, the lock screen application determines that a dynamic effect needs to be displayed, and the dynamic effect to be displayed is an elastomer dynamic effect. Then, the application uses the physics engine to give real physical properties to the elastomers in the interface to be displayed, such as determining the programming code, script, etc. that realizes the movement, rotation, or collision reflection of these elastomers. Afterwards, the application adaptively determines a back-end platform for physical calculation with better performance (i.e., step 902) based on the physical calculation scale in the current business scenario, and performs physical simulation through the determined back-end platform to output a simulation mesh (i.e., step 903). Among them, elastomers are often represented as grids in computers.

[0132] In this way, for different physical computing scales and different computing platforms, automatic scheduling can be achieved to adapt to different computing powers and achieve performance optimization on each platform.

[0133] It should be understood that the lock screen interface is not necessarily the lock screen interface in the locked state, but can also be the lock screen interface in the unlocked state. In some examples, the electronic device can also be unlocked by face recognition. In this scenario, the electronic device can display the unlocked lock screen interface after unlocking. In this regard, the electronic device also responds to the user's operation on the unlocked lock screen interface, such as sliding up or sliding down, and the electronic device can enter the desktop from the unlocked lock screen interface. In this process, the electronic device can display the second animation mentioned in the following embodiments and use the methods provided in each embodiment to process the elastic body of the theme.

[0134] In some cases, a single CPU core offers strong computing power and fast startup. GPUs offer strong parallel computing capabilities, but require additional preparation time for startup. Currently, Cuda and OpenCL are widely used computing platforms in the industry. These platforms flexibly support various GPU and CPU implementations at the compilation level.

[0135] Exemplarily, as shown in FIG10 , the physics engine is configured with a hybrid scheduling computing base of CPU and GPU. Among them, as shown in the figure mark 101, in the device development stage or the application development stage, the programmer abstracts the commonly used physical calculation operators (operators), such as abstractly encapsulating each physical calculation operator separately. And, as shown in the figure mark 102, all operators after abstract encapsulation can be implemented in both CPU and GPU modes. For example, the physical calculation backend includes a CPU backend 102a and a GPU backend 102b, and all operators after abstract encapsulation can be executed by the CPU backend 102a and the GPU backend 102b. Among them, the operator can also be described as an operator. An operator is a mathematical concept that can be used to represent a mapping relationship. An element of a vector space is calculated to obtain another element in the current or another vector space through this mapping.

[0136] For example, physical calculation operators include addition, subtraction, multiplication (dot), sort, cross-product, gradient, reduction, Fourier transform, integration, etc.

[0137] Then, as shown in the figure mark 103, when the application determines to trigger the elastic body scene, the load assessment-automatic scheduling module adaptively schedules the operations used in the physical calculation to the CPU or GPU for execution.

[0138] For example, as shown in FIG11 , the process of the application determining the backend platform for physical computing in step 902 includes a load evaluation phase, an automatic scheduling phase, and a distribution execution phase.

[0139] In some examples, during the load evaluation phase, the load evaluation module analyzes the execution efficiency of different physical simulation scenarios on different devices, and dynamically adjusts whether the computing operations are executed on the GPU or the CPU in the subsequent automatic adjustment phase to obtain better physical simulation performance.

[0140] For example, before performing a specific physical simulation, the load assessment module first calculates the CPU calculation time, GPU calculation time, and CPU-to-GPU data transfer time for each operator required in the current elastic body scene. The mesh code for the image frame acquired by the application is stored in the CPU's memory. Therefore, if it is determined that the GPU needs to perform computational processing, this data needs to be transferred from the CPU to the GPU. In other words, during the load assessment phase, the CPU-to-GPU data transfer time also needs to be considered.

[0141] For example, for operator i, the load evaluation module evaluates the CPU execution time cputime i , GPU execution time gputime i , and the data transfer time from CPU to GPU cpu2gputime i .

[0142] It should be understood that the hardware performance in different electronic devices may be different, so the calculation time of the same operator on different back-end platforms may also be different. Therefore, before selecting the back-end platform, the actual calculation time of each operator on different back-end platforms is first determined by the load evaluation module, so that accurate back-end platform scheduling can be achieved based on the actual calculation time. Therefore, the display method provided in the embodiment of the present application can be adaptively applied to the hardware platforms of different devices.

[0143] In some embodiments, after downloading a theme, an electronic device can obtain the operators included in the elastic body calculation corresponding to the current theme. During the load assessment phase, the load assessment module needs to evaluate the execution time and data transmission time of these operators on different backend platforms. The elastic body calculation corresponding to the theme is used to implement the elastic body animation effect display on one or more interfaces applying the current theme.

[0144] For example, a default theme is pre-installed in the electronic device. Before the electronic device runs the default theme, such as the first time, it can first obtain the operators included in the elastic body calculation corresponding to the default theme to evaluate the execution time and data transmission time of these operators on different back-end platforms. In this way, when the default theme is displayed later, the back-end platform used for calculation can be determined. For another example, after the electronic device downloads a custom theme in response to user operation, it can obtain the operators included in the elastic body calculation corresponding to the custom theme to evaluate the execution time and data transmission time of these operators on different back-end platforms. In this way, when the custom theme is displayed later, the back-end platform used for calculation can be determined.

[0145] For example, as shown in FIG12 , the elastic body calculation includes gradient calculation, integral calculation, multiple calculation, and Fourier transform calculation. In the load evaluation phase, the load evaluation module determines that the CPU performs gradient calculation for 3 milliseconds (ms) and the GPU performs gradient calculation for 1 ms; the CPU performs integral calculation for 3.5 ms and the GPU performs integral calculation for 1 ms; the CPU performs multiple calculation for 10 ms and the GPU performs integral calculation for 1 ms. The data transmission time from the CPU to the GPU is not shown in FIG12 .

[0146] Afterwards, as shown in Figure 11, after the load evaluation phase is completed, the automatic scheduling module can obtain the load evaluation results output by the load evaluation module. Then, the automatic scheduling module can execute the automatic scheduling phase according to the load evaluation results and determine whether to schedule the CPU or GPU. For example, the automatic scheduling module uses the automatic scheduling algorithm to analyze the time consumed by each operator using the GPU or the CPU based on the calculation time of each operator, and determines which back-end platform consumes less time to calculate a certain operator, so as to determine which back-end platform needs to be used to process the operator. This improves the overall computing efficiency and improves the physical performance in elastic scenarios. Among them, the calculation time of the operator includes the CPU calculation time, GPU calculation time, and data transmission time from the CPU to the GPU of the operator previously determined by the load evaluation module.

[0147] Exemplarily, the automatic scheduling module determines the back-end platform for scheduling based on the number of vertices of the elastic body included in the image frame and the duration of each operator to be used. Among them, the elastic body is often represented by a grid in the computer, and the grid has a corresponding number of vertices. As shown in Figure 12, in the current elastic body scene, the elastic body calculations involved include gradient calculation, integral calculation, multiple calculation, and Fourier transform calculation. In the automatic scheduling stage, the automatic scheduling module calculates the calculation time of various operators on the CPU and the calculation time on the GPU respectively based on the number of elastic body vertices to determine the scheduling result. For example, in the current elastic body scene, through the following formula (1), the automatic scheduling module determines that the gradient calculation, integral calculation, multiple calculation, and Fourier transform calculation can be calculated in a shorter time using the GPU. Therefore, the automatic scheduling module can instruct the distribution execution module to call the GPU for calculation.

[0148] in,

[0149] For example, in the above formula (1), during the CPU calculation process, the status of the operator before and after will not change, that is, status i =status i+1 Then, the CPU calculation time of operator i is stepTime i =cuptime+0 1 ×cup2gputime=cuptime. During the GPU calculation process, since the operator needs to be transferred from the CPU to the GPU, the status of the operator will change before and after. i ≠status i+1 Then, the GPU computation time of operator i is stepTime i =gputime+0 0 ×cup2gputime=gputime+cup2gputime.

[0150] After the automatic scheduling phase, as shown in Figure 11, the dispatch execution module receives the scheduling results output by the automatic scheduling module. These results indicate the backend platforms on which each operator should run. For example, as shown in Figure 12, the dispatch execution module instructs each operator to run on the GPU according to the scheduling results, allowing the GPU to perform the computation.

[0151] Thus, compared to the current situation where programmers can only specify whether to compile a program into CPU code or GPU code during the development phase, the display processing method provided by the embodiment of the present application can dynamically switch the back-end platform used for calculation during runtime, automatically select the back-end platform with better performance based on different hardware platforms and calculation scales in different business scenarios, and achieve better physical simulation results.

[0152] For example, based on the above-mentioned method of adaptively scheduling the back-end platform, electronic devices can adaptively schedule to the CPU back-end platform to perform calculations in low-load small-scale scenarios; and schedule to the GPU back-end platform to perform calculations in high-load large-scale scenarios.

[0153] The following example illustrates an experimental scenario to illustrate the effect of the above-mentioned method of adaptive scheduling backend platform.

[0154] For example, based on the elastic body collision physics simulation application, the physical operation operators include integral operators, sampling operators, difference operators, linear equation operators, spatial gradient operators, matrix dot multiplication, and matrix cross multiplication. Through 100 iterations to correspond to the multi-vertex calculation process, the following test verification scenario is designed.

[0155] In some examples, such as test scenario 1 shown in Table 3 below, in a load scenario of the same scale, the tests are run on test machine 1 and test machine 2 respectively to test whether the computing power can be scheduled to run to better performance. For example, as shown in Table 3 below, the CPU performance of test machine 1 and test machine 2 is the same. Test machine 1 determines to schedule the CPU by comparing the CPU performance and GPU performance. For example, the CPU calculation time of test machine 1 is shorter. Test machine 2 determines to schedule the GPU by comparing the CPU performance and GPU performance. For example, the GPU calculation time of test machine 2 is shorter. It can be seen that different test machines can be adaptively scheduled to the back-end platform with better computing power.

[0156] Table 3

[0157] In some examples, such as the test scenario 2 shown in Table 4 below, different load scenarios are run on the test machine 1, such as running a 4x4 low-load elastic body falling collision scenario and a 12x12 high-load elastic body falling collision scenario respectively, to test whether the computing power can be scheduled to run to better performance. For example, as shown in Table 4 below, in the 4x4 low-load elastic body falling collision scenario, the test machine 1 determines to schedule the CPU by comparing the CPU performance and the GPU performance. For example, in the low-load elastic body falling collision scenario, the CPU calculation time is shorter. In the 12x12 high-load elastic body falling collision scenario, the test machine 1 determines to schedule the GPU by comparing the CPU performance and the GPU performance. For example, in the high-load elastic body falling collision scenario, the GPU calculation time is shorter. It can be seen that the same test machine can be adaptively scheduled to the back-end platform with better computing power according to different load scenarios.

[0158] Table 4

[0159] In some embodiments, the application determines the backend platform through the above method and performs calculations through the backend platform. Among them, the elastomer is often represented by a grid in the computer, so in the elastomer scene, the backend platform can output a simulation grid. Among them, the simulation grid is a coarse force grid and cannot obtain a good display effect. At present, users have a great demand for high-quality and refined effects of electronic devices. Therefore, electronic devices need to process the simulation grid to obtain a finer-grained rendering grid that can achieve better display effects.

[0160] In some cases, the simulation mesh can also be described as a physical topology mesh.

[0161] The following describes the process of converting a simulation mesh into a rendering mesh.

[0162] In some embodiments, multiple subdivisions of a coarser-grained simulation mesh can yield a finer-grained rendering mesh. Therefore, during the development phase, programmers can pre-establish a mapping between the simulation mesh and the rendering mesh and directly pre-set this mapping into the electronic device. This way, when mesh subdivision is needed, the electronic device can directly determine the rendering mesh corresponding to the current simulation mesh based on the mapping. This improves resolution and display quality while reducing the subdivision processing load.

[0163] Exemplarily, as shown by reference numeral 104 in FIG. 10 , in the physical engine functional business layer, a mapping matrix between the offline calculated simulation grid and the rendering grid is preset, and the mapping matrix is ​​used to indicate the mapping relationship from the simulation grid to the rendering grid.

[0164] As shown in Figure 11, during the topology construction phase, the offline topology construction module creates a mapping between the simulation mesh and the rendering mesh. For example, the offline topology construction module subdivides the simulation mesh to produce a high-resolution rendering mesh. As shown in Figure 13, after obtaining the simulation mesh, the offline topology construction module performs multiple subdivision cycles to output a rendering mesh that meets the required frame rate.

[0165] In some examples, the offline topology construction module defines x as the original simulation mesh vertex and x' as the mesh vertex after the next subdivision. The mapping relationship between the two is Where x′ j is the jth subdivision mesh vertex, which is related to the n vertices of the original simulation mesh, x i is the i-th related simulation mesh vertex, w i is the corresponding weight coefficient.

[0166] After that, the offline topology construction module sequentially substitutes the coarse mesh vertices of the previous step into the fine mesh vertices of the next step. As shown in Figure 13, the offline topology construction module calculates the original simulation mesh vertex coordinates. Determine the mesh vertex coordinates obtained after the first loop subdivision is This completes the initial update of vertex coordinates. Subsequently, during n subdivision cycles, the offline topology construction module sequentially updates the mesh vertex coordinates. Ultimately, the offline topology construction module establishes the relationship between all original simulation mesh vertices x and all final rendered mesh vertices x', as shown below, thereby obtaining the mapping matrix w.

[0167] In some embodiments, after the electronic device triggers the display of the elastic body scene and outputs the simulated mesh through the back-end platform, as shown in Figure 9, the simulated mesh can be mapped to the rendering mesh through a preset mapping relationship, thereby realizing the dynamic display based on the rendering mesh (i.e., steps 904 and 905). In this way, based on the mapping relationship, the mesh updated by the physical simulation is mapped to the high-resolution rendering mesh once per frame to reduce the subdivision processing load.

[0168] For example, as shown in Figure 11, during the online computation process, the distribution execution module schedules the backend platform to execute the computation and output a physical simulation mesh. The online acceleration module then accesses the mapping relationships pre-set by the offline topology construction module and, based on these relationships, directly maps the physical simulation mesh into a refined rendering mesh. As shown in Figure 14, this simple mapping of the relationships eliminates the need for the application to perform multiple subdivision cycles and still achieves a higher-resolution rendering mesh, resulting in better display quality.

[0169] Thus, compared to existing subdivision shaders that can only achieve subdivision by modifying interpolated vertices so that the generated surface follows the mesh normals, resulting in additional subdivision processing load, the display processing method provided in the embodiments of the present application can directly map the simulation mesh to the rendering mesh in one go based on the offline preset mapping relationship, thereby reducing the subdivision processing load while ensuring the subdivision effect.

[0170] Furthermore, since the mapping between the simulation mesh and the rendering mesh is established offline, it is possible to obtain a more delicate and smooth rendering mesh by modifying the original vertices of the simulation mesh.

[0171] In addition, the display processing method provided in the embodiment of the present application is independent of the GPU hardware type, thus avoiding the possibility that different compiler types may cause different segmentation effects.

[0172] It should be understood that in some scenarios, the programmer pre-specifies the backend platform. Then, after the designated backend platform outputs the simulation mesh, the electronic device can also map the simulation mesh to the rendering mesh based on the mapping relationship. That is, the above-mentioned method of adaptively scheduling the backend platform and the method of obtaining the rendering mesh based on the mapping relationship can be performed separately or in combination, and this embodiment of the application does not limit this.

[0173] The following example illustrates an experimental scenario to illustrate the effect of the above-mentioned method of obtaining a rendering mesh based on a mapping relationship.

[0174] For example, on the same electronic device, in the same elastic body collision scenario, three methods for calculating the rendering mesh were used: a GPU-side subdivision shader, a CPU-side classic loop subdivision method, and a mapping method provided by an embodiment of the present application. The electronic device performance, rendering mesh resolution, and subdivision effect were compared. As shown in Table 5 below, the electronic device performance measurement items include frame rate, power consumption increment, and double data rate (DDR) frequency.

[0175] Table 5

[0176] The experimental results shown in Table 5 above show that the mapping method provided by the present embodiment achieves a 67% power saving and a 473% reduction in DDR frequency compared to using GPU-side subdivision shaders. Compared to using the classic loop subdivision method on the CPU, the frame rate is increased by 46.5 fps and the DDR frequency is reduced by 965.1. Furthermore, the classic loop subdivision method on the CPU achieves a frame rate of only 12 fps, severely dropping frames and making it unsuitable for real-time electronic device scenarios. Therefore, the experimental results above do not compare the incremental power consumption of the mapping method with the subdivision method.

[0177] In addition, Figure 15 (a) shows the display effect after obtaining the rendered mesh using the classic loop subdivision method on the CPU side. Figure 15 (b) shows the display effect after obtaining the rendered mesh using the mapping method provided in the embodiment of the present application. It can be clearly seen that the mapping method provided in the embodiment of the present application can achieve better subdivision effects and thus better display effects.

[0178] FIG16 is a flow chart of a display processing method provided in an embodiment of the present application. It should be noted that the method is not limited to FIG16 and the specific sequence described below. It should be understood that in other embodiments, the order of some steps in the method can be interchanged according to actual needs, or some steps can be omitted or deleted. The method includes the following steps:

[0179] Step 1601: The electronic device determines a theme used by the electronic device in response to a first operation by the user, where the theme includes an elastic body displayed on an off-screen interface.

[0180] In some examples, the first operation is, for example, a user clicking on the display screen of the electronic device when the screen is black; or a user approaching the electronic device. After the user approaches the electronic device, the sensor of the electronic device can detect the user, thereby determining that the user needs to turn on the phone.

[0181] In some embodiments, after detecting a user's first operation, the electronic device determines that a screen-off interface needs to be displayed. In some examples, the electronic device is configured with a theme function, which can be used to set the display of the screen-off interface, lock screen interface, desktop, and other stages. For example, an elastic body is displayed on the screen-off interface. In some examples, different themes may have different display content and display effects. Then, after detecting the user's first operation, the electronic device may determine the elastic body indicated in the theme currently used by the electronic device to be displayed on the screen-off interface.

[0182] Step 1602: The electronic device obtains a simulation mesh corresponding to the elastic body.

[0183] In some embodiments, after acquiring the elastomer, the electronic device may determine to trigger a simulation of the elastomer to output a simulation mesh. In some examples, the electronic device may simulate the elastomer using the first platform and / or the second platform. To improve simulation efficiency and enhance electronic device performance, the electronic device may select the platform that consumes less time to simulate the elastomer.

[0184] In some examples, the following examples take the first platform as a CPU and the second platform as a GPU as an example to provide a detailed description of the platform selection process during the simulation process. It should be understood that in another example, the first platform may also be a GPU and the second platform may also be a CPU.

[0185] In some embodiments, the electronic device obtains a first simulation duration of the elastic body simulated by a first platform of the electronic device and a second simulation duration of the elastic body simulated by a second platform of the electronic device. The electronic device then invokes the first platform and / or the second platform to simulate the elastic body based on the first simulation duration and the second simulation duration.

[0186] For example, if the first simulation duration is shorter than the second simulation duration, the electronic device may use the first platform to simulate the elastic body. That is, the first platform alone takes less time to simulate the elastic body, so the electronic device may choose to use the first platform to simulate the elastic body.

[0187] For another example, if the second simulation duration is shorter than the first simulation duration, the electronic device may use the second platform to simulate the elastic body. That is, if the second platform alone takes less time to simulate the elastic body, the electronic device may choose to use the second platform to simulate the elastic body.

[0188] For another example, if the third simulation duration is less than the first and second simulation durations, the electronic device may call upon the first and second platforms to simulate the elastomer. The third simulation duration is the simulation duration required for the case where the first and second platforms are called upon to collaboratively simulate the elastomer, as determined based on the first and second simulation durations. That is, the case where the first and second platforms collaboratively simulate the elastomer takes less time, and therefore the electronic device may choose to call upon the first and second platforms to simulate the elastomer.

[0189] Exemplarily, as shown in FIG10 , the lower layer of the physics engine includes a hybrid scheduling computing base of the GPU and the CPU, wherein the encapsulated operators can be calculated by both the GPU and the CPU. Then, after the GPU executes the operator operation, the simulation grid can be uploaded to the physics engine business layer located in the upper layer for processing. Alternatively, after the CPU executes the operator operation, the simulation grid can be uploaded to the physics engine business layer located in the upper layer for processing. Alternatively, after the GPU and the CPU execute the operator operation respectively, their respective simulation results can be uploaded to the physics engine business layer located in the upper layer. After the physics engine processes the obtained GPU simulation results and the CPU simulation results, the simulation grid can be obtained.

[0190] In some embodiments, after detecting a user's first operation and determining the theme used by the electronic device, the electronic device can obtain N operators required for the theme, where N is a positive integer. The electronic device can then obtain a first calculation duration for each of the N operators executed by the first platform, a second calculation duration for each of the N operators executed by the second platform, and a data transmission duration from the first platform to the second platform.

[0191] For example, as shown in FIG12 , the electronic device obtains N operators corresponding to the topic, namely gradient calculation, integral calculation, multiple calculation, and Fourier transform calculation. Then, the electronic device can determine the first calculation duration of the first platform performing the gradient calculation, integral calculation, multiple calculation, and Fourier transform calculation, respectively, and the second calculation duration of the second platform performing the gradient calculation, integral calculation, multiple calculation, and Fourier transform calculation, respectively, and determine the data transmission duration of each of the N operators from the first platform to the second platform.

[0192] In this way, before selecting a platform, the electronic device first determines the actual calculation time of each operator on different platforms, so that accurate platform scheduling can be achieved based on the actual calculation time, thereby realizing adaptive hardware platforms applied to different devices.

[0193] In some embodiments, when the electronic device determines that an elastomer needs to be simulated, it can obtain the number of operators of the N operators corresponding to the elastomer. For example, simulating an elastomer requires M gradient calculations, and the number of operators for the gradient calculation is M, where M is a positive integer. Afterwards, the electronic device obtains the first duration for the first platform to execute the N operators respectively based on the number of operators and the first calculation duration; and obtains the second duration for the second platform to execute the N operators respectively based on the number of operators, the second calculation duration, and the data transmission duration, the second duration includes the cumulative transmission duration of each of the N operators transmitted from the first platform to the second platform. Afterwards, the electronic device determines the first simulation duration based on the first duration. Determine the second simulation duration based on the second duration.

[0194] In some examples, the electronic device determines that a first duration corresponding to a first type of operator among N types of operators is less than a second duration, and invokes the first platform to execute the first type of operator. The electronic device determines that a second duration corresponding to a second type of operator among N types of operators is less than the first duration, and invokes the second platform to execute the second type of operator.

[0195] Exemplarily, as shown in FIG12 , the electronic device determines that the first duration for the CPU to perform gradient calculation is 3ms, and the second duration for the GPU to perform gradient calculation is 1ms; the first duration for the CPU to perform integral calculation is 3.5ms, and the second duration for the GPU to perform integral calculation is 1ms; the first duration for the CPU to perform multiplier calculation is 10ms, and the second duration for the GPU to perform integral calculation is 1ms. Then, the electronic device can determine that the first simulation duration for calling the CPU to simulate the elastomer is 16.5ms (3ms+3.5ms+10ms), and the first simulation duration for calling the GPU to simulate the elastomer is 3ms (1ms+1ms+1ms). Then, based on the fact that the second simulation duration is less than the first simulation duration, the electronic device can determine to call the GPU to simulate the elastomer to obtain a simulation grid.

[0196] As another example, the electronic device determines that the first time length for the CPU to perform gradient calculation is 3ms, and the second time length for the GPU to perform gradient calculation is 1ms; the first time length for the CPU to perform integral calculation is 3.5ms, and the second time length for the GPU to perform integral calculation is 1ms; the first time length for the CPU to perform multiple calculation is 1ms, and the second time length for the GPU to perform integral calculation is 10ms. Then, the electronic device can determine that the first type of operator is multiple calculation, the second type of operator is gradient calculation and integral calculation, and the sum of the time length for the first platform to execute the first type of operator and the time length for the second platform to execute the second type of operator is 3ms (1ms+1ms+1ms). Then, the electronic device can determine to call the CPU to perform multiple calculation, and call the GPU to perform gradient calculation and integral calculation. In this way, through the cooperation of the CPU and GPU, the simulation of the elastomer can be completed in a shorter time, and the simulation mesh can be output, thereby improving the simulation efficiency.

[0197] In this way, compared to the current situation where programmers can only specify whether to compile a program into CPU code or GPU code during the development phase, the display processing method provided by the embodiment of the application can dynamically switch the platform used for calculation during runtime, automatically select the platform with better performance based on different hardware platforms and calculation scales in different business scenarios, and achieve better physical simulation results.

[0198] Step 1603: The electronic device maps the simulation grid to a rendering grid according to the mapping relationship.

[0199] In some embodiments, before the electronic device is used for the first time, the electronic device can obtain and save a mapping relationship, where the mapping relationship includes a mapping matrix for converting a simulation mesh into a rendering mesh.

[0200] For example, as shown in FIG11 , during the development phase, programmers can create a mapping relationship between a simulation mesh and a rendering mesh offline through an offline topology construction module, and directly preset the mapping relationship into the electronic device.

[0201] In this way, based on the mapping relationship, the electronic device can map the simulation mesh updated by the physical simulation to the high-resolution rendering mesh at one time per frame to reduce the subdivision processing load.

[0202] Thus, compared to existing subdivision shaders that can only achieve subdivision by modifying interpolated vertices so that the generated surface follows the mesh normals, resulting in additional subdivision processing load, the display processing method provided by the embodiment of the present application can directly map the simulation mesh to the rendering mesh in one go based on the offline preset mapping relationship, thereby ensuring the subdivision effect while reducing the subdivision processing load.

[0203] Furthermore, since the mapping between the simulation mesh and the rendering mesh is established offline, it is possible to obtain a more delicate and smooth rendering mesh by modifying the original vertices of the simulation mesh.

[0204] In addition, the display processing method provided in the embodiment of the present application is independent of the GPU hardware type, thus avoiding the possibility that different compiler types may cause different segmentation effects.

[0205] Step 1604: The electronic device displays a screen-off interface including the elastic body based on the rendered grid.

[0206] In some embodiments, after acquiring the rendering mesh, the electronic device can perform the subsequent drawing and rendering process of the elastic body, and then send the synthesized image frame to the display screen for display through layer synthesis. In this way, the electronic device can display an off-screen interface including the elastic body.

[0207] Exemplarily, as shown in (a) in FIG6 , the electronic device detects the first operation of the user on the display screen during the black screen process, obtains the sports ball theme currently used by the electronic device, obtains the elastomeric balls displayed on the off-screen interface included in the sports ball theme, simulates the elastomeric balls through the adaptive selection platform of the above-mentioned step 1602 to obtain a simulation grid, and directly maps the simulation grid to a rendering grid based on the mapping relationship through step 1603. Then, based on the rendering grid, it can display the off-screen interface as shown in (b) in FIG6 , including the elastomeric balls shown in the function window 61.

[0208] As another example, while displaying the screen-off interface, the electronic device detects a second user operation on the screen-off interface and calls the first platform and / or the second platform to simulate the elastomer. The electronic device displays a first animation of the elastomer from the screen-off interface to the lock screen interface. Thereafter, the electronic device may display a lock screen interface including the elastomer. Thereafter, while displaying the lock screen interface, the electronic device detects a third user operation on the lock screen interface and calls the first platform and / or the second platform to simulate the elastomer. The electronic device displays a second animation of the elastomer from the lock screen interface to the desktop. Thereafter, the electronic device displays the desktop.

[0209] For example, as shown in FIG6(c), when the electronic device detects a second user operation on the display screen while displaying the screen-off interface, the electronic device may invoke the first platform and / or the second platform to simulate the elastic body to display a first animation of the elastic body ball from the screen-off interface to the lock screen interface, as shown in FIG7(a). Subsequently, the electronic device may display the lock screen interface including the elastic body ball, as shown in FIG7(b).

[0210] In some examples, during the process of displaying the first animation, displaying the lock screen interface including the elastomer, and displaying the second animation, the platform called by the electronic device is the same as the platform called by the electronic device during the process of displaying the off-screen interface including the elastomer.

[0211] In this way, after the electronic device assigns a platform for simulating the elastomer according to the elastomer included in the theme, it can continuously use the same platform to simulate the elastomer displayed in multiple interfaces included in the same theme. Based on the consistency of the display effect across the interfaces at different stages of the theme, it can avoid unnecessary repeated platform selection while maintaining display quality and display processing efficiency.

[0212] The display processing method provided by the embodiment of the present application is described in detail above with reference to Figures 6 to 16. The electronic device provided by the embodiment of the present application is described in detail below with reference to Figure 17.

[0213] In one possible design, Figure 17 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. As shown in Figure 17, electronic device 1700 may include: a transceiver unit 1701, a processing unit 1702, and a display unit 1703. Electronic device 1700 may be used to implement the functions of the electronic device involved in the above method embodiments.

[0214] Optionally, the transceiver unit 1701 is used to support the electronic device 1700 to execute step 901 in Figure 9; and / or to support the electronic device 1700 to execute step 1601 in Figure 16.

[0215] Optionally, the processing unit 1702 is used to support the electronic device 1700 to execute step 902, step 903, and step 904 in Figure 9; and / or to support the electronic device 1700 to execute step 1602 and step 1603 in Figure 16.

[0216] Optionally, the display unit 1703 is used to support the electronic device 1700 to execute step 905 in Figure 9; and / or to support the electronic device 1700 to execute step 1604 in Figure 16.

[0217] The transceiver unit may include a receiving unit and a transmitting unit, and may be implemented by a transceiver or transceiver-related circuit components, and may be a transceiver or a transceiver module. The operations and / or functions of each unit in electronic device 1700 are respectively to implement the corresponding processes of the display processing method described in the above method embodiment. All relevant content of each step involved in the above method embodiment can be referenced to the functional description of the corresponding functional unit. For the sake of brevity, it is not repeated here.

[0218] Optionally, the electronic device 1700 shown in FIG17 may further include a storage unit (not shown in FIG17 ) storing a program or instruction. When the transceiver unit 1701, the processing unit 1702, and the display unit 1703 execute the program or instruction, the electronic device 1700 shown in FIG17 may perform the display processing method described in the above method embodiment.

[0219] The technical effects of the electronic device 1700 shown in FIG17 may refer to the technical effects of the display processing method described in the above method embodiment, and will not be repeated here.

[0220] In addition to being in the form of the electronic device 1700, the technical solution provided in this application may also be a functional unit or chip in the electronic device, or a device used in conjunction with the electronic device.

[0221] An embodiment of the present application also provides a chip system, including: a processor, the processor is coupled to a memory, the memory is used to store programs or instructions, when the program or instructions are executed by the processor, the chip system implements the method in any of the above method embodiments.

[0222] Optionally, there may be one or more processors in the chip system. The processor may be implemented in hardware or software. When implemented in hardware, the processor may be a logic circuit, an integrated circuit, etc. When implemented in software, the processor may be a general-purpose processor implemented by reading software code stored in a memory.

[0223] Optionally, the memory in the chip system may be one or more. The memory may be integrated with the processor or may be provided separately from the processor, which is not limited in the embodiments of the present application. For example, the memory may be a non-transient processor, such as a read-only memory (ROM), which may be integrated with the processor on the same chip or provided on different chips. The embodiments of the present application do not specifically limit the type of memory or the configuration of the memory and the processor.

[0224] Exemplarily, the chip system may be a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on chip (SoC), a central processor unit (CPU), a network processor (NP), a digital signal processor (DSP), a microcontroller unit (MCU), a programmable logic device (PLD) or other integrated chips.

[0225] It should be understood that each step in the above method embodiment can be completed by hardware integrated logic circuits in a processor or by software instructions. The method steps disclosed in the embodiments of the present application can be directly embodied as being executed by a hardware processor, or by a combination of hardware and software modules in a processor.

[0226] An embodiment of the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program runs on a computer, the computer executes the above-mentioned related steps to implement the display processing method in the above-mentioned embodiment.

[0227] An embodiment of the present application further provides a computer program product. When the computer program product is run on a computer, the computer is caused to execute the above-mentioned related steps to implement the display processing method in the above-mentioned embodiment.

[0228] In addition, embodiments of the present application further provide a device. Specifically, the device may be a component or module, and may include one or more processors and a memory connected to each other. The memory is used to store a computer program. When the computer program is executed by one or more processors, the device performs the display processing method described in each of the above method embodiments.

[0229] The apparatus, computer-readable storage medium, computer program product, or chip provided in the embodiments of the present application are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects of the corresponding methods provided above, and will not be repeated here.

[0230] The steps of the method or algorithm described in conjunction with the disclosure of the embodiments of the present application can be implemented in a hardware manner or can be implemented by a processor executing software instructions. The software instructions can be composed of corresponding software modules, and the software modules can be stored in a random access memory (RAM), a flash memory, a read-only memory (ROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a register, a hard disk, a mobile hard disk, a read-only compact disc (CD-ROM) or any other form of storage medium known in the art. An exemplary storage medium is coupled to a processor so that the processor can read information from the storage medium and can write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an application specific integrated circuit (ASIC).

[0231] Through the description of the above embodiments, those skilled in the art will clearly understand that for the sake of convenience and brevity, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed; that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. The specific working processes of the above-mentioned systems, devices, and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0232] In the several embodiments provided in this application, it should be understood that the disclosed methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of the modules or units is only a logical function division, and there may be other division methods in actual implementation; for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of modules or units, which can be electrical, mechanical or other forms.

[0233] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0234] Computer-readable storage media include, but are not limited to, any of the following: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, and other media that can store program code.

[0235] The above is only a specific embodiment of the present application, but the scope of protection of this application is not limited to this. Any changes or substitutions within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A display processing method, characterized in that: Applied to electronic equipment, the method comprises: In response to a first operation of a user, determining a theme used by the electronic device, the theme comprising an elastic body displayed on a screen-off interface; Obtaining a simulation grid corresponding to the elastic body; According to the mapping relationship, mapping the simulation grid to a rendering grid; According to the rendering grid, a screen-off interface including the elastic body is displayed.

2. The method according to claim 1, characterized in that: The obtaining of the simulation grid corresponding to the elastic body comprises: Acquire a first simulation duration for simulating the elastomer by a first platform of the electronic device, and a second simulation duration for simulating the elastomer by a second platform of the electronic device; According to the first simulation duration and the second simulation duration, the first platform and / or the second platform are called to simulate the elastomer.

3. The method according to claim 2, characterized in that The calling the first platform and / or the second platform to simulate the elastic body according to the first simulation duration and the second simulation duration includes: When the first simulation duration is less than the second simulation duration, calling the first platform to simulate the elastic body; or, When the second simulation duration is shorter than the first simulation duration, calling the second platform to simulate the elastic body; or, When the third simulation duration is less than the first simulation duration and the second simulation duration, the first platform and the second platform are called to simulate the elastomer, and the third simulation duration is the simulation duration required for calling the first platform and the second platform to perform cooperative simulation of the elastomer determined according to the first simulation duration and the second simulation duration.

4. The method according to claim 2 or 3, characterized in that: Before obtaining a first simulation duration for simulating the elastomer by the first platform and a second simulation duration for simulating the elastomer by the second platform, the method further includes: Obtain N operators corresponding to the topic, where N is a positive integer; Obtain a first calculation duration for the first platform to execute each of the N operators, a second calculation duration for the second platform to execute each of the N operators, and a data transmission duration for each of the N operators from the first platform to the second platform.

5. The method according to claim 4, characterized in that The obtaining of a first simulation duration of the first platform simulating the elastomer and a second simulation duration of the second platform simulating the elastomer comprises: Obtain the number of operators of N operators corresponding to the elastic body; According to the number of operators and the first calculation duration, obtaining first durations for the first platform to respectively execute the N types of operators; and obtaining, according to the number of operators, the second calculation duration, and the data transmission duration, second durations for the second platform to respectively execute the N types of operators, where the second duration includes a cumulative transmission duration for the first platform to transmit each of the N types of operators to the second platform; Determining the first simulation duration according to the first duration; The second simulation duration is determined according to the second duration.

6. The method according to claim 5, characterized in that The method further comprises: Determine that the first duration corresponding to a first type of operator among the N types of operators is less than the second duration, and call the first platform to execute the first type of operator; It is determined that the second duration corresponding to the second type of operators among the N types of operators is less than the first duration, and the second platform is called to execute the second type of operators.

7. The method according to any one of claims 1 to 6, characterized in that: The method further comprises: Detecting a second operation of the user on the screen-off interface, calling the first platform and / or the second platform to simulate the elastic body; Displaying a first animation of the elastic body from the screen-off interface to the screen-lock interface; The lock screen interface including the elastomer is displayed.

8. The method according to claim 7, characterized in that The method further comprises: Detecting a third operation of the user on the lock screen interface, calling the first platform and / or the second platform to simulate the elastic body; Display a second animation of the elastic body from the lock screen interface to the desktop; The desktop is displayed.

9. The method according to claim 7 or 8, characterized in that: During the process of displaying the first animation, displaying the lock screen interface including the elastomer, and displaying the second animation, the platform called by the electronic device is the same as the platform called by the electronic device during the process of displaying the screen-off interface including the elastomer.

10. The method according to any one of claims 1 to 9, characterized in that: Before determining the theme used by the electronic device in response to the first operation of the user, the method further includes: The mapping relationship is acquired and saved, wherein the mapping relationship includes a mapping matrix used to convert the simulation mesh into the rendering mesh.

11. An electronic device, characterized in that: include: A processor, a memory and a display screen, wherein the memory and the display screen are coupled to the processor, the memory is used to store computer program code, and the computer program code includes computer instructions. When the processor reads the computer instructions from the memory, the electronic device executes the method as described in any one of claims 1 to 10.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium comprises a computer program, and when the computer program is executed on an electronic device, the electronic device is enabled to execute the method according to any one of claims 1 to 10.

13. A computer program product, characterized in that When the computer program product is executed on a computer, the computer is caused to execute the method according to any one of claims 1 to 10.

Images