Display processing method and electronic equipment

Abstract

According to the display processing method and the electronic equipment provided by the invention, the mapping relationship between the simulation grid and the rendering grid is pre-configured, so that the corresponding rendering grid can be directly obtained according to the mapping relationship after the simulation grid is determined. Therefore, the picture quality delicate effect is ensured, the physical calculation load is reduced as much as possible, and the delicate dynamic effect experience is realized. The method comprises the steps that the electronic equipment responds to a first operation of a user, a theme used by the electronic equipment is determined, and the theme comprises an elastomer displayed on a screen-off interface; and then the electronic equipment obtains a simulation grid corresponding to the elastomer. And mapping the simulation grid into a rendering grid according to the mapping relation. And according to the rendering grid, the electronic equipment displays a screen-off interface comprising the elastomer.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of terminal technology, and in particular to a display processing method and an electronic device. Background Art

[0002] With the development of terminal technology, electronic devices display animation effects by combining real physical motion to increase the fun of interface display and attract more user traffic. For example, the dynamic wallpaper displayed by electronic devices includes the motion effects of objects that conform to real physical motion. Among them, the key to the performance of physical motion effects lies in real-time simulation.

[0003] Currently, real-time simulation on mobile terminals (such as mobile phones and tablets) mainly focuses on rigid bodies and small-scale scenes. However, for elastic bodies and large-scale scenes, it is difficult to ensure the display effect of physical motion effects due to issues such as the amount of calculation. Summary of the invention

[0004] In order to solve the above technical problems, the present application provides a display processing method and an electronic device. The technical solution provided by the present application pre-configures the mapping relationship between the simulation grid and the rendering grid, so that after determining the simulation grid, the corresponding rendering grid can be directly obtained according to the mapping relationship. This ensures that the image quality is exquisite while minimizing the physical calculation load and achieving an exquisite dynamic effect experience.

[0005] In order to achieve the above technical objectives, this application provides the following technical solutions:

[0006] In a first aspect, a display processing method is provided, which is applied to an electronic device. The method includes: in response to a first operation of a user, determining a theme used by the electronic device, the theme including an elastic body displayed on an off-screen interface. Obtaining a simulation grid corresponding to the elastic body. Mapping the simulation grid to a rendering grid according to a mapping relationship. Displaying an off-screen interface including the elastic body according to the rendering grid.

[0007] Thus, compared with the existing subdivision shader, which can only achieve subdivision by modifying the interpolated vertices so that the generated surface follows the mesh normal, resulting in additional subdivision processing load, the display processing method provided in the embodiment of the present application can directly map the simulation mesh to the rendering mesh at one time according to the offline preset mapping relationship, thereby reducing the subdivision processing load while ensuring the subdivision effect.

[0008] Furthermore, since the mapping relationship between the simulation mesh and the rendering mesh is established in an offline process, a rendering mesh with a more delicate and smooth effect can be obtained by modifying the original vertices of the simulation mesh.

[0009] According to the first aspect, obtaining a simulation grid corresponding to an elastomer includes: obtaining a first simulation duration for simulating the elastomer by a first platform of an electronic device, and a second simulation duration for simulating the elastomer by a second platform of the electronic device. Based on the first simulation duration and the second simulation duration, calling the first platform and / or the second platform to simulate the elastomer.

[0010] According to the first aspect, or any implementation of the first aspect above, the first platform and / or the second platform are called to simulate the elastomer according to the first simulation duration and the second simulation duration, including: when the first simulation duration is less than the second simulation duration, the first platform is called to simulate the elastomer. Alternatively, when the second simulation duration is less than the first simulation duration, the second platform is called to simulate the elastomer. Alternatively, when the third simulation duration is less than the first simulation duration and the second simulation duration, the first platform and the second platform are called to simulate the elastomer, and the third simulation duration is the simulation duration required for calling the first platform and the second platform to perform cooperative simulation of the elastomer determined according to the first simulation duration and the second simulation duration.

[0011] For example, the first platform is a CPU platform, and the second platform is a GPU platform.

[0012] Thus, compared with the current situation that only supports programmers to specify whether to compile a program into CPU code or GPU code during the development phase, the display processing method provided in the embodiment of the present application can dynamically switch the platform used for calculation at runtime, automatically select a platform with better performance according to different hardware platforms and different business scenarios, and achieve better physical simulation.

[0013] According to the first aspect, or any implementation of the first aspect above, before obtaining the first simulation duration of the first platform simulating the elastic body and the second simulation duration of the second platform simulating the elastic body, the method further includes: obtaining N operators corresponding to the subject, where N is a positive integer. Obtaining the first calculation duration of each of the N operators executed by the first platform, the second calculation duration of each of the N operators executed by the second platform, and the data transmission duration of each of the N operators from the first platform to the second platform.

[0014] Exemplarily, the electronic device obtains N operators corresponding to the subject, which are gradient calculation, integral calculation, multiple calculation, and Fourier transform calculation. Then, the electronic device can determine the first calculation duration of the first platform performing the gradient calculation, integral calculation, multiple calculation, and Fourier transform calculation, the second calculation duration of the second platform performing the gradient calculation, integral calculation, multiple calculation, and Fourier transform calculation, and determine the data transmission duration from the first platform to the second platform.

[0015] In this way, before the electronic device selects a platform, it first determines the actual calculation time of different platforms for each operator, so that accurate platform scheduling can be achieved based on the actual calculation time, thereby realizing adaptive hardware platforms applied to different devices.

[0016] According to the first aspect, or any implementation of the first aspect above, obtaining a first simulation duration for simulating an elastomer by a first platform and a second simulation duration for simulating an elastomer by a second platform includes: obtaining the number of operators of N operators corresponding to the elastomer. According to the number of operators and the first calculation duration, obtaining the first duration for the first platform to respectively execute the N operators; and according to the number of operators, the second calculation duration, and the data transmission duration, obtaining the second duration for the second platform to respectively execute the N operators, the second duration including the cumulative transmission duration for each of the N operators transmitted from the first platform to the second platform. According to the first duration, determine the first simulation duration. According to the second duration, determine the second simulation duration.

[0017] For example, the duration of a gradient calculation performed by the first platform is a first calculation duration a, the duration of a gradient calculation performed by the second platform is b, the duration of a gradient calculation data transmitted by the first platform to the second platform is c, and the elastic body corresponds to d gradient calculations. Then, the first duration of the gradient calculation performed by the first platform is a*d, and the second duration of the gradient calculation performed by the second platform is (b+c)*d. Where a, b, c are positive numbers, and d is a positive integer.

[0018] Exemplarily, the electronic device obtains the N operators included in the subject, which are gradient calculation, integral calculation, multiple calculation, and Fourier transform calculation. And the first operator duration and the second operator duration for calculating various operators by the first platform and the second platform, as well as the number of various operators, have been determined. Then, for example, the electronic device determines that the first duration for gradient calculation by the first platform is 3 milliseconds, and the second duration for gradient calculation by the second platform is 1 millisecond; the first duration for integral calculation by the first platform is 3.5 milliseconds, and the second duration for integral calculation by the second platform is 1 millisecond; the first duration for multiple calculation by the first platform is 10 milliseconds, and the second duration for integral calculation by the second platform is 1 millisecond. Then, the electronic device can determine that the first simulation duration for calling the first platform to simulate the elastomer is 16.5 milliseconds (3 milliseconds + 3.5 milliseconds + 10 milliseconds), and the first simulation duration for calling the second platform to simulate the elastomer is 3 milliseconds (1 millisecond + 1 millisecond + 1 millisecond). Then, based on the second simulation duration being less than the first simulation duration, the electronic device can determine to call the second platform to simulate the elastomer to obtain a simulation grid.

[0019] According to the first aspect, or any implementation of the first aspect above, the method further includes: determining that a first duration corresponding to a first type of operator among the N types of operators is less than a second duration, and calling the first platform to execute the first type of operator. Determining that a second duration corresponding to a second type of operator among the N types of operators is less than the first duration, and calling the second platform to execute the second type of operator.

[0020] Exemplarily, the electronic device determines that the first duration of the gradient calculation by the first platform is 3 milliseconds, and the second duration of the gradient calculation by the second platform is 1 millisecond; the first duration of the integral calculation by the first platform is 3.5 milliseconds, and the second duration of the integral calculation by the second platform is 1 millisecond; the first duration of the multiple calculation by the first platform is 1 millisecond, and the second duration of the integral calculation by the second platform is 10 milliseconds. Then, the electronic device can determine that the first type of operator is multiple calculation, the second type of operator is gradient calculation and integral calculation, and the sum of the duration of the first platform executing the first type of operator and the duration of the second platform executing the second type of operator is 3 milliseconds (1 millisecond + 1 millisecond + 1 millisecond). Then, the electronic device can determine to call the first platform to perform multiple calculation, and call the second platform to perform gradient calculation and integral calculation. In this way, through the cooperation of the first platform and the second platform, the simulation of the elastomer can be completed in a shorter time, and the simulation grid can be output, thereby improving the simulation efficiency.

[0021] In this way, the electronic device pre-evaluates the duration of simulation of the elastomer by the first platform and / or the second platform respectively, and then determines the calling method that can achieve better performance based on the evaluated duration, and calls the corresponding platform to simulate the elastomer, thereby improving the simulation efficiency while ensuring that the simulation grid can be output.

[0022] According to the first aspect, or any implementation of the first aspect above, the method further includes: detecting a second operation of the user on the screen-off interface, calling the first platform and / or the second platform to simulate the elastomer. Displaying a first animation of the elastomer from the screen-off interface to the lock screen interface. Displaying a lock screen interface including the elastomer.

[0023] According to the first aspect, or any implementation of the first aspect above, the method further includes: detecting a third operation of the user on the lock screen interface, calling the first platform and / or the second platform to simulate the elastic body. Displaying a second animation of the elastic body from the lock screen interface to the desktop. Displaying the desktop.

[0024] According to the first aspect, or any implementation of the first aspect above, during the process of displaying the first animation, displaying the lock screen interface including the elastomer, and displaying the second animation, the platform called by the electronic device is the same as the platform called by the electronic device during the process of displaying the screen-off interface including the elastomer.

[0025] In this way, after the electronic device allocates a platform for simulating the elastomer according to the elastomer included in the theme, it can continuously call the same platform to simulate the elastomer displayed in multiple interfaces included in the same theme. Based on the consistency of the display effect on the interface at different stages of the theme, it can avoid unnecessary repeated selection of platforms while ensuring the display effect and display processing efficiency.

[0026] According to the first aspect, or any implementation of the first aspect above, before determining the theme used by the electronic device in response to the user's first operation, the method also includes: acquiring and saving a mapping relationship, the mapping relationship including a mapping matrix for converting a simulation grid into a rendering grid.

[0027] For example, during the development phase, programmers may create a mapping relationship between a simulation grid and a rendering grid offline through an offline topology construction module, and directly preset the mapping relationship into the electronic device.

[0028] In this way, based on the mapping relationship, the electronic device can map the simulation mesh updated by the physical simulation to the high-resolution rendering mesh once per frame to reduce the subdivision processing load.

[0029] In a second aspect, an electronic device is provided. The electronic device includes: a processor, a memory, and a display screen, the memory and the display screen are coupled to the processor, the memory is used to store computer program code, the computer program code includes computer instructions, and when the processor reads the computer instructions from the memory, the electronic device executes: in response to a first operation of the user, determining a theme used by the electronic device, the theme including an elastomer displayed on an off-screen interface. Obtain a simulation grid corresponding to the elastomer. According to the mapping relationship, map the simulation grid to a rendering grid. According to the rendering grid, display an off-screen interface including the elastomer.

[0030] According to the second aspect, obtaining a simulation grid corresponding to an elastomer includes: obtaining a first simulation duration for simulating the elastomer by a first platform of an electronic device, and a second simulation duration for simulating the elastomer by a second platform of the electronic device. Based on the first simulation duration and the second simulation duration, calling the first platform and / or the second platform to simulate the elastomer.

[0031] According to the second aspect, or any implementation of the second aspect above, according to the first simulation duration and the second simulation duration, calling the first platform and / or the second platform to simulate the elastomer includes: when the first simulation duration is less than the second simulation duration, calling the first platform to simulate the elastomer. Alternatively, when the second simulation duration is less than the first simulation duration, calling the second platform to simulate the elastomer. Alternatively, when the third simulation duration is less than the first simulation duration and the second simulation duration, calling the first platform and the second platform to simulate the elastomer, the third simulation duration being the simulation duration required for calling the first platform and the second platform to perform cooperative simulation of the elastomer determined according to the first simulation duration and the second simulation duration.

[0032] According to the second aspect, or any implementation of the second aspect above, when the processor reads the computer instruction from the memory, the electronic device is also caused to execute: obtaining N operators included in the subject, where N is a positive integer. Obtaining a first calculation duration for executing each of the N operators on the first platform, a second calculation duration for executing each of the N operators on the second platform, and a data transmission duration for each of the N operators from the first platform to the second platform.

[0033] According to the second aspect, or any implementation of the second aspect above, obtaining the first simulation duration of the first platform simulating the elastomer and the second simulation duration of the second platform simulating the elastomer includes: obtaining the number of operators of the N operators corresponding to the elastomer. According to the number of operators and the first calculation duration, obtaining the first duration of the first platform executing the N operators respectively; and according to the number of operators, the second calculation duration, and the data transmission duration, obtaining the second duration of the second platform executing the N operators respectively, the second duration including the cumulative transmission duration of each of the N operators transmitted from the first platform to the second platform. According to the first duration, determine the first simulation duration. According to the second duration, determine the second simulation duration.

[0034] According to the second aspect, or any implementation of the second aspect above, when the processor reads the computer instruction from the memory, the electronic device is further caused to execute: determining that the first duration corresponding to the first type of operator among the N types of operators is less than the second duration, and calling the first platform to execute the first type of operator. Determining that the second duration corresponding to the second type of operator among the N types of operators is less than the first duration, and calling the second platform to execute the second type of operator.

[0035] According to the second aspect, or any implementation of the second aspect above, when the processor reads the computer instruction from the memory, the electronic device is also caused to execute: upon detecting a second operation of the user on the screen-off interface, calling the first platform and / or the second platform to simulate the elastomer. Displaying a first animation of the elastomer from the screen-off interface to the lock screen interface. Displaying a lock screen interface including the elastomer.

[0036] According to the second aspect, or any implementation of the second aspect above, when the processor reads the computer instruction from the memory, the electronic device is further caused to execute: detecting a third operation of the user on the lock screen interface, calling the first platform and / or the second platform to simulate the elastic body. Displaying a second animation of the elastic body from the lock screen interface to the desktop. Displaying the desktop.

[0037] According to the second aspect, or any implementation of the second aspect above, during the process of displaying the first animation, displaying the lock screen interface including the elastomer, and displaying the second animation, the platform called by the electronic device is the same as the platform called by the electronic device during the process of displaying the screen-off interface including the elastomer.

[0038] According to the second aspect, or any implementation of the second aspect above, when the processor reads the computer instruction from the memory, it also enables the electronic device to execute: obtaining and saving a mapping relationship, where the mapping relationship includes a mapping matrix for converting a simulation mesh into a rendering mesh.

[0039] In a third aspect, an electronic device is provided, which has the function of implementing the display processing method described in the first aspect and any possible implementation thereof. The function can be implemented by hardware, or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions.

[0040] In a fourth aspect, a computer-readable storage medium is provided. The computer-readable storage medium stores a computer program (also referred to as an instruction or code), and when the computer program is executed by an electronic device, the electronic device executes the method of the first aspect or any one of the implementations of the first aspect.

[0041] According to a fifth aspect, a computer program product is provided. When the computer program product is run on an electronic device, the electronic device executes the method according to the first aspect or any one of the implementations of the first aspect.

[0042] The technical effects of the aforementioned aspects can be referenced to each other and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 A schematic diagram of a physical engine based on a Cuda computing platform provided in an embodiment of the present application;

[0044] Figure 2 A schematic diagram of a physical engine flow based on a Cuda computing platform provided in an embodiment of the present application;

[0045] Figure 3 A schematic diagram of the solution flow of the subdivision shader provided in an embodiment of the present application;

[0046] Figure 4 A schematic diagram of the subdivision principle provided for an embodiment of the present application;

[0047] Figure 5 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application;

[0048] Figure 6 Schematic diagram of the elastic body scene provided in the embodiment of the present application Figure 1 ;

[0049] Figure 7 Schematic diagram of the elastic body scene provided in the embodiment of the present application Figure 2 ;

[0050] Figure 8 Schematic diagram of the elastic body scene provided in the embodiment of the present application Figure 3 ;

[0051] Fig. 9 Schematic diagram of the display processing method provided in the embodiment of the present application Figure 1 ;

[0052] Fig.10 A schematic diagram of a functional block diagram of a display processing method provided in an embodiment of the present application;

[0053] Fig.11 A schematic diagram of the interaction between modules in the display processing method provided in an embodiment of the present application;

[0054] Fig.12 A schematic diagram of the load assessment-automatic scheduling principle provided in an embodiment of the present application;

[0055] Fig.13 A schematic diagram of the offline topology acceleration structure provided in an embodiment of the present application;

[0056] Fig.14 A schematic diagram of a simulated grid mapping rendering grid scene provided in an embodiment of the present application;

[0057] Fig.15 A schematic diagram of a rendering grid effect provided in an embodiment of the present application;

[0058] Fig.16 Schematic diagram of the display processing method provided in the embodiment of the present application Figure 2 ;

[0059] Fig.17 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0060] The technical solutions in the embodiments of the present application are described below in conjunction with the accompanying drawings in the embodiments of the present application. Wherein, in the description of the embodiments of the present application, the terms used in the following embodiments are only for the purpose of describing specific embodiments, and are not intended to be used as limitations on the present application. As used in the specification and the appended claims of the present application, the singular expressions "one", "a kind of", "said", "above", "the" and "this" are intended to include expressions such as "one or more", unless there is a clear indication to the contrary in the context. It should also be understood that in the following embodiments of the present application, "at least one", "one or more" refer to one or more (including two).

[0061] References to "one embodiment" or "some embodiments" etc. described in this specification mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Thus, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways. The term "connection" includes direct connection and indirect connection, unless otherwise specified. "First" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated.

[0062] In the embodiments of the present application, the words "exemplarily" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "exemplarily" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplarily" or "for example" is intended to present related concepts in a specific way.

[0063] In some embodiments, electronic devices display animation effects by combining real physical motion to increase the fun of the interface display and attract more user traffic. For example, the dynamic wallpaper displayed by the electronic device includes the motion effects of objects that conform to real physical motion. Among them, the key to the performance of physical motion effects lies in real-time physical simulation. Physical simulation, also known as simulation, simulation, and physical simulation, predicts the behavior or results of a real world or physical system in a computer.

[0064] In some examples, the simulation results are calculated by assigning real physical properties to objects in the computer. The physics engine has the ability to simulate various objects, such as rigid bodies, elastic bodies, fluids, and other simulation results. For example, the physics engine calculates the movement, rotation, and collision response of objects by assigning real physical properties to objects. Among them, a rigid body is a concept in physics, a solid with a finite size that can be displaced or rotated under the action of external forces, but will not deform. An elastic body (elastic body / soft body) is also called a soft body or a flexible body. It is a concept in physics. It is a solid with a finite size that can be displaced, rotated, or deformed under the action of external forces, and can restore its shape after the external force is removed. Fluid is a concept in physics, a general term for liquids and gases, and an object that will continuously deform under any small shear force.

[0065] The performance of physical simulation depends on the development of physical engines. The mainstream physical engines in the industry have experienced development from simple rigid body scenes to complex fluid scenes, and from small-scale scenes to large-scale scene support. Table 1 below illustrates several mainstream physical engines in the industry, their corresponding back-end platforms, and the types of devices they are mainly applied to. Among them, Havok has a large ecosystem and PhysX has full functions, but these powerful physics engines are mainly used on PCs and hosts. Mobile terminals (such as mobile phones, tablets, etc.) have fewer real-time simulation support functions, and currently focus on rigid bodies or simple cloth simulation scenes. They are almost only based on central processor unit (CPU) calculations, and basically do not support graphics processing unit (GPU) calculations. There is a huge gap in performance optimization for 3D sophisticated physical simulations.

[0066] Table 1

[0067]

[0068] In some examples, the compute unified device architecture (Cuda) and the open computing language (OpenCL) are computing platforms that are currently widely used in the industry. They support the implementation of GPU and CPU versions of various algorithms, enabling GPUs to solve complex computing problems.

[0069] It should be understood that the physical engine and computing platform illustrated in Table 1 are merely illustrative, and other and more physical engines and computing platforms may also be applicable to the display processing method provided in the embodiment of the present application, which will not be described in detail below.

[0070] In some embodiments, the computing power comparison between the CPU and GPU of different devices varies greatly. The CPU single-core computing power is strong and the operation startup is fast, while the GPU has strong parallel computing power and there is additional consumption in the preparation stage of the operation startup.

[0071] In some examples, as shown in Table 1 above, the current mainstream physics engines support both CPU and GPU. Taking the Cuda computing platform as an example, Figure 1 As shown in the figure, the device provides CPU and GPU versions through various algorithms. Programmers need to specify whether the program is compiled into CPU code or GPU code during the development phase. The upper layer is the physics engine implementation layer, which includes the basic components of rigid body physics, fluid physics, and elastic body physics, as well as their corresponding dynamic simulation algorithms. For example, Figure 2 As shown in the figure, programmers estimate the computing requirements according to the target scenario and computing scale, and specify the code to select GPU or CPU compilation in the development state. Then, the physical engine implementation layer calls the corresponding underlying computing algorithm to realize the physical simulation function. For example, if the programmer chooses the GPU backend platform, then the calculation is performed on the GPU during the data transmission and preparation stage; if the programmer chooses the CPU backend platform, then the calculation is performed directly on the CPU during the data transmission and preparation stage.

[0072] In some examples, different physical calculation scales, GPU and CPU calculations will produce different performance results. Optionally, the performance results are measured by frame rate. Frame rate is used to indicate the measurement of the number of frames displayed in the current animation, and the unit is "frames per second" (FPS). The larger the current animation frame rate, the higher the performance can be considered.

[0073] For example, as shown in Table 2 below, when there are 10*10*10 box-shaped objects falling in the scene, the CPU backend platform is faster to calculate. When the scale increases to 11*11*11 box-shaped objects falling, the GPU backend platform is faster to calculate. As the scale of physical calculation increases, the difference between the calculation speed of the GPU backend platform and the CPU backend platform becomes larger.

[0074] However, since the backend platform selection method only supports programmers to specify in the development state, and programmers cannot estimate the specific business scenarios of the subsequent device during user use in the development state, the computing backend platform specified by the programmer may not provide the device with better display performance.

[0075] Table 2

[0076]

[0077] Based on this, the embodiment of the present application provides a display processing method, where the physical engine computing platform can adapt to different computing devices and different business scenarios, and adaptively select the CPU backend platform or the GPU backend platform, thereby realizing complex physical calculations in different configurations and scenarios, and achieving better computing performance.

[0078] In some examples, the high-quality and exquisite effect of the display screen requires a high model resolution. Real-time physical simulation needs to respond to the motion state in each frame of the display screen, and the simulation performance is strongly related to the scale of grid calculation. In addition, while ensuring the image quality, the physical calculation load needs to be reduced. The current common solution in the industry is to separate the model grid into a simulation grid and a rendering grid. Among them, in computer simulation calculations, objects represented continuously need to be converted into simulation grids represented in a discrete manner. When the computer presents an object on the screen, the continuously represented object needs to be converted into a rendering grid represented in a discrete manner. Among them, the simulation grid has a lower resolution and the rendering grid has a higher resolution. By converting the simulation grid into a rendering grid, the resolution is improved, and the simulation grid is output first and then the rendering grid is converted to achieve a reduction in the computing load while meeting the high resolution.

[0079] For example, Figure 3 The method of implementing separation simulation rendering mesh through tessellation shader is shown. Among them, tessellation shader is a computer program used for special effects production, and is also a shader running in GPU. For example, in business scenarios, tessellation shader divides simple mesh into finer meshes through specific function calculation.

[0080] like Figure 3 As shown in the figure, after the application passes the vertex and image data to the vertex buffer in the GPU video memory, the vertex buffer passes the vertex array to the vertex shader, which performs operations such as rotation, scaling, and coordinate conversion. After that, the subdivision shader converts the simulation mesh processed by the vertex shader into a rendering mesh, and passes the rendering mesh to the geometry shader. The geometry shader assembles the primitives and determines the way to draw the graphics according to the connection relationship between the defined vertices. The geometry shader rasterizes the primitives, traverses each screen pixel to determine whether it is covered by the primitive, and passes it to the fragment shader to color each pixel. After that, the result of the fragment shader is output to the frame buffer. Finally, the contents of the frame buffer are displayed on the screen.

[0081] In some examples, the specific working process of the subdivision shader is analyzed by taking the creation engine tool Unity as an example. Among them, there are many implementation methods of the built-in subdivision shader of Unity, but they are all limited to triangular meshes. For example, the subdivision shader subdivides a fixed amount of surface and directly adds a fixed amount of surface subdivision to each mesh unit. Among them, the distance-based surface subdivision changes the surface subdivision level according to the distance between the mesh unit and the virtual camera. For example, the closer the distance, the higher the subdivision level. For example, the distance is determined based on the depth information of the mesh unit. The surface subdivision based on the edge length can calculate the subdivision level according to the edge length of the screen triangle. The longer the edge length, the higher the subdivision level of the mesh unit. Phong surface subdivision can modify the position of the generated subdivision surface, and finally make the generated subdivision surface match the mesh normal. Phong surface subdivision can smooth the surface mesh. Among them, Phong subdivision is a type of subdivision shader, and its operation rule is to modify the interpolated vertices so that the generated surface follows the mesh normal. For example, such as Figure 4 As shown in (a) in the figure, the subdivision shader determines the three vertices and three edges of the triangle mesh, and subdivides the triangle mesh into the following according to the edge length: Figure 4 The four triangular meshes shown in (b).

[0082] In the above process, the higher the subdivision level, the better the final display effect. However, such methods often cannot modify the original mesh vertices; for example, the vertices of the triangle mesh cannot be modified. In addition, the required subdivision level is high, resulting in a large subdivision processing load.

[0083] In addition, the subdivision shader runs on the GPU backend platform and depends on the compiler type, resulting in different effects on different chips. Therefore, the subdivision shader that can achieve high display effects on device 1 may have poor display effects after being calculated on device 2, and the consistency of display effects on different types of devices cannot be guaranteed.

[0084] Another solution to reduce the physical calculation load while ensuring image quality is to directly achieve fine mesh division through loop subdivision or mesh regularization calculation on the CPU side. Among them, loop subdivision is an algorithm for subdividing the mesh in the CPU, and a finer mesh is obtained by modifying the interpolated vertices and the original mesh vertices.

[0085] However, although such methods can achieve good rendering effects, they are computationally very expensive and are generally performed as model post-processing, making them difficult to use for real-time animation.

[0086] Based on this, the embodiment of the present application provides a display processing method, which pre-configures the mapping relationship between the simulation grid and the rendering grid, so that after the computing platform outputs the simulation grid, the corresponding rendering grid can be directly obtained according to the mapping relationship. This ensures that the image quality is exquisite while minimizing the physical calculation load, and achieves an exquisite UI dynamic effect experience.

[0087] Exemplarily, the display processing method provided in the embodiment of the present application is applied to an electronic device 100. The electronic device 100 may be, for example, a mobile phone, a tablet computer, a laptop computer, a smart screen, a wearable device, a vehicle terminal, an augmented reality (AR) / VR, a PS, an Xbox, a PC, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), an artificial intelligence (AI) device, and other terminal devices. The operating system installed in the electronic device 100 includes but is not limited to The embodiments of the present application do not limit the specific type of the electronic device 100 or the installed operating system.

[0088] Figure 5 A schematic structural diagram of an electronic device 100 is shown.

[0089] The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc.

[0090] It is to be understood that the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0091] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0092] The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of instruction fetching and execution.

[0093] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0094] In some embodiments, the processor 110 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0095] The I2C interface is a bidirectional synchronous serial bus, including a serial data line (SDA) and a serial clock line (SCL). In some embodiments, the processor 110 may include multiple groups of I2C buses. The processor 110 may be coupled to the touch sensor, charger, flash, camera 193, etc. through different I2C bus interfaces. For example, the processor 110 may be coupled to the touch sensor through the I2C interface, so that the processor 110 communicates with the touch sensor through the I2C bus interface to realize the touch function of the electronic device 100.

[0096] The MIPI interface can be used to connect the processor 110 with peripheral devices such as the display screen 194 and the camera 193. The MIPI interface includes a camera serial interface (CSI), a display serial interface (DSI), etc. In some embodiments, the processor 110 and the camera 193 communicate via the CSI interface to implement the shooting function of the electronic device 100. The processor 110 and the display screen 194 communicate via the DSI interface to implement the display function of the electronic device 100.

[0097] The USB interface 130 is an interface that complies with the USB standard specification, and specifically can be a Mini USB interface, a Micro USB interface, a USB Type C interface, etc. The USB interface 130 can be used to connect a charger to charge the electronic device 100, and can also be used to transfer data between the electronic device 100 and a peripheral device. It can also be used to connect headphones to play audio through the headphones. The interface can also be used to connect other electronic devices, such as AR devices, etc.

[0098] It is understandable that the interface connection relationship between the modules illustrated in the embodiment of the present application is only a schematic illustration and does not constitute a structural limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0099] The charging management module 140 is used to receive charging input from a charger. The charger may be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 may receive charging input from a wired charger through the USB interface 130. In some wireless charging embodiments, the charging management module 140 may receive wireless charging input through a wireless charging coil of the electronic device 100. While the charging management module 140 is charging the battery 142, it may also power the electronic device through the power management module 141.

[0100] The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, and supplies power to the processor 110, the internal memory 121, the display screen 194, the camera 193, and the wireless communication module 160. The power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle number, battery health status (leakage, impedance), etc. In some other embodiments, the power management module 141 can also be set in the processor 110. In other embodiments, the power management module 141 and the charging management module 140 can also be set in the same device.

[0101] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0102] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0103] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G, etc., applied to the electronic device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the same device as at least some of the modules of the processor 110.

[0104] The modem processor may include a modulator and a demodulator. The modulator is used to modulate the low-frequency baseband signal to be sent into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After the low-frequency baseband signal is processed by the baseband processor, it is passed to the application processor. The application processor outputs a sound signal through an audio device, or displays an image or video through a display screen 194. In some embodiments, the modem processor may be an independent device. In other embodiments, the modem processor may be independent of the processor 110 and be set in the same device as the mobile communication module 150 or other functional modules.

[0105] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the electronic device 100. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, modulates the frequency of the electromagnetic wave signal and performs filtering, and sends the processed signal to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, modulate the frequency of it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0106] In some embodiments, the antenna 1 of the electronic device 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the electronic device 100 can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-CDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology. The GNSS may include a global positioning system (GPS), a global navigation satellite system (GLONASS), a Beidou navigation satellite system (BDS), a quasi-zenith satellite system (QZSS) and / or a satellite based augmentation system (SBAS).

[0107] The electronic device 100 implements the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

[0108] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), for example, an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Mini-led, Micro-led, Micro-oled, quantum dot light emitting diodes (QLED), etc. In some embodiments, the electronic device 100 may include 1 or N display screens 194, where N is a positive integer greater than 1.

[0109] The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then passes the electrical signal to the ISP to be converted into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the electronic device 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.

[0110] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function, such as storing music, video and other files in the external memory card.

[0111] The internal memory 121 can be used to store computer executable program codes, which include instructions. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the electronic device 100 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc. The processor 110 executes various functional applications and data processing of the electronic device 100 by running instructions stored in the internal memory 121, and / or instructions stored in a memory provided in the processor.

[0112] The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signal. The audio module 170 can also be used to encode and decode audio signals. In some embodiments, the audio module 170 can be arranged in the processor 110, or some functional modules of the audio module 170 can be arranged in the processor 110. The electronic device 100 can use the audio module 170, such as music playing, recording, etc. The audio module 170 can include a speaker, a receiver, a microphone, a headphone interface, and an application processor to implement audio functions.

[0113] The sensor module 180 may include a pressure sensor, a gyro sensor, an air pressure sensor, a magnetic sensor, an acceleration sensor, a distance sensor, a proximity light sensor, a fingerprint sensor, a temperature sensor, a touch sensor, an ambient light sensor, a bone conduction sensor, and the like.

[0114] The key 190 includes a power key, a volume key, etc. The key 190 may be a mechanical key or a touch key. The electronic device 100 may receive key input and generate key signal input related to user settings and function control of the electronic device 100.

[0115] Motor 191 can generate vibration prompts. Motor 191 can be used for incoming call vibration prompts, and can also be used for touch vibration feedback. Motor 191 can also correspond to different vibration feedback effects for touch operations on different areas of display screen 194. Different application scenarios (for example: time reminder, receiving information, alarm clock, game, etc.) can also correspond to different vibration feedback effects.

[0116] Indicator 192 may be an indicator light, which may be used to indicate charging status, power changes, messages, missed calls, notifications, etc.

[0117] The SIM card interface 195 is used to connect a SIM card. The SIM card can be connected to or disconnected from the electronic device 100 by inserting or removing the SIM card interface 195. The electronic device 100 can support 1 or N SIM card interfaces, where N is a positive integer greater than 1.

[0118] The following uses Cuda and OpenCL as computing platforms, and the backend platforms supported by the physics engine include CPU and GPU as examples to introduce in detail the display processing method provided in the embodiments of the present application.

[0119] For ease of understanding, several elastic body animation display scenarios are shown below.

[0120] In some scenarios, electronic devices are equipped with a theme function, through which users can set at least one of the display stages of the off-screen interface, the lock-screen interface, and the desktop display at one time, so that users can personalize the electronic device more quickly and conveniently. Depending on the theme, the display of each stage of the theme may include elastic body motion effect display, such as displaying elastic body motion effect on the off-screen interface, displaying elastic body motion effect on the lock-screen interface, and displaying elastic body motion effect in the process of switching from the off-screen interface to the lock-screen interface.

[0121] For example, in response to user operations, the electronic device sets a sports ball theme and displays an elastomeric motion effect display scene of an elastomeric ball moving on the screen-off interface and the lock screen interface.

[0122] For example, Figure 6 As shown in (a) of FIG. 1 , when the electronic device detects that a user clicks the display screen during a black screen, or detects that a user approaches the display screen through a sensor (such as a camera, etc.), the electronic device may display the following: Figure 6 The screen-off interface shown in (b) in FIG. 1 shows a function window 61 on the screen-off interface. When the electronic device displays the screen-off interface, the elements (such as elastic balls) in the screen-off interface move on the screen-off interface based on the sports ball theme. Figure 6 As shown in (b) of FIG. 6 , the electronic device may display element motion effects in the function window 61, such as displaying the movement of multiple elastic balls in the function window. Figure 6 (b) to Figure 6 As shown in (c) in the figure, as time changes, the motion states of multiple elastic balls displayed in the function window change. For example, the size, motion trajectory, shape, etc. will change due to the collision between the elastic balls. In this way, the electronic device displays the scene through the elastic body motion effect, which increases the fun of the off-screen interface display.

[0123] As another example, Figure 6As shown in (c), when the electronic device is displaying the screen-off interface, the lock screen interface may be displayed after the user's operation on the lock screen button 62 or the user's double-click operation on the display screen is detected. When the electronic device is displaying the lock screen interface, the elements in the lock screen interface (such as elastic balls) move on the lock screen interface based on the theme of sports balls. Optionally, in order to maintain the consistency of the theme, the motion effects of different stages can be displayed continuously. For example, the electronic device can link the elastic body motion effect of the screen-off interface with the elastic body motion effect of the lock screen interface. In the process of switching the screen-off interface to the lock screen interface in response to the user's operation, the electronic device uses the elastic body motion effect displayed on the screen-off interface as the motion starting point of the elastic body motion effect displayed on the lock screen interface.

[0124] For example, when an electronic device displays Figure 6 During the process of the screen-off interface shown in (c) of FIG. 1 , if the user instructs to switch to the lock screen interface, the electronic device can control the elastic ball displayed in the function window 61 to move toward the entire display screen. Figure 7 In the lock screen interface shown in (a), the elastic ball has moved from being displayed inside the function window to being displayed outside the function window. Figure 7 As shown in (b), the elastic ball moves to the edge of the display screen, such as the elastic ball touches the edge of the lock screen interface. In this way, the electronic device displays the scene through the elastic body motion effect, which increases the interest of the lock screen interface display.

[0125] As another example, in the process of displaying the dynamic effect of the elastic ball moving from the function window 61 to the entire display screen, the electronic device may Figure 8 As shown in (a) of FIG. 1 , after the electronic device detects the user's operation on the lock screen button 62, it can stop displaying the current animation. For example, the electronic device reversely controls the elastic ball to move into the function window 61. In the process of the electronic device switching from the lock screen interface to the off screen interface in response to the user's operation, as shown in FIG. Figure 8 As shown in (b) in the figure, the elastic ball is displayed moving back to the function window 61. In this way, the electronic device displays the scene through the elastic body dynamic effect, increases the relevance of the dynamic effect between the interface switching of the off screen interface and the lock screen interface, and improves the user experience.

[0126] In other scenarios, the electronic device can also respond to user operations and set the elastic body dynamic effect display scene of each interface separately. For example, the user selects the dynamic wallpaper of the screen-off interface, and the display of the dynamic wallpaper includes the elastic body dynamic effect display. For another example, the user selects the dynamic wallpaper of the lock screen interface, and the display of the lock screen wallpaper includes the elastic body dynamic effect display. Among them, the elastic body dynamic effects between different interfaces can be associated or not associated.

[0127] In some examples, real-time simulation of elastic bodies involves more complex collision detection and collision response, which has high computational requirements.

[0128] It should be understood that the display processing method provided in the embodiment of the present application can be applied not only to the dynamic display scene of elastic bodies, but also to the dynamic display scenes of rigid bodies, fluids, etc. The embodiment of the present application only takes the dynamic display scene of elastic bodies with high computing requirements as an example to illustrate the display processing process.

[0129] In addition, the display processing method provided in the embodiment of the present application can be applied to small-scale dynamic effect scenes or large-scale dynamic effect scenes, which will not be described in detail below. Among them, small-scale dynamic effect scenes and large-scale dynamic effect scenes are only a relative description of the scale of dynamic effects. For example, if the number of objects executing the dynamic effect does not exceed the threshold, it can be regarded as a small-scale dynamic effect scene; if the number of objects executing the dynamic effect exceeds the threshold, it can be regarded as a large-scale dynamic effect scene.

[0130] In some embodiments, a physics engine is configured in the application to process the motion reaction of objects during the dynamic effect display process. The application can be a native system application or a third-party application. For example, the application is a lock screen application, a screen saver application, systemUI, a third-party theme application, a third-party wallpaper application, etc. Or the application is other game applications.

[0131] In some embodiments, Fig. 9 As shown, in response to user operation, the application triggers the elastomer scene (i.e., step 901). For example, in response to the user's operation of the lock screen button, the lock screen application determines that a dynamic effect needs to be displayed, and the dynamic effect to be displayed is the elastomer dynamic effect. Then, the application uses the physical engine to give real physical properties to the elastomers in the interface to be displayed, such as determining the programming code, script, etc. that realizes the movement, rotation, or collision reflection of these elastomers. After that, the application adaptively determines the back-end platform for physical calculation with better performance (i.e., step 902) according to the physical calculation scale in the current business scenario, so as to perform physical simulation through the determined back-end platform and output the simulation grid (i.e., step 903). Among them, elastomers are often represented by grids in computers.

[0132] In this way, for different physical computing scales and different computing platforms, automatic scheduling can be used to adapt to different computing powers and achieve performance optimization on each platform.

[0133] It should be understood that the lock screen interface is not necessarily the lock screen interface in the locked state, but can also be the lock screen interface in the unlocked state. In some examples, the electronic device can also be unlocked by face recognition. In this scenario, the electronic device can display the unlocked lock screen interface after unlocking. In this regard, the electronic device also responds to the user's operation on the unlocked lock screen interface, such as sliding up or sliding down, and the electronic device can enter the desktop from the unlocked lock screen interface. In this process, the electronic device can display the second animation mentioned in the following embodiments, and use the methods provided in each embodiment to process the elastomer of the theme.

[0134] In some examples, the CPU has strong single-core computing power and fast startup. The GPU has strong parallel computing power, and there is additional consumption in the preparation stage of operation startup. Currently, the most commonly used computing platforms in the industry are Cuda and OpenCL, which flexibly support the implementation of various GPUs and CPUs at the compilation level.

[0135] For example, Fig.10 As shown, the physics engine is configured with a hybrid scheduling computing base of CPU and GPU. Among them, as shown in the figure mark 101, in the device development stage or the application development stage, the programmer abstracts the commonly used physical calculation operators (operators), such as abstractly encapsulating each physical calculation operator respectively. And, as shown in the figure mark 102, all the operators after abstract encapsulation can be implemented in both CPU and GPU. For example, the physical calculation backend includes a CPU backend 102a and a GPU backend 102b, and all the operators after abstract encapsulation can be executed by the CPU backend 102a and the GPU backend 102b. Among them, the operator can also be described as an operator. An operator is a mathematical concept that can be used to represent a mapping relationship. An element of a vector space is calculated to obtain another element in the current or another vector space through this mapping.

[0136] For example, physical calculation operators include add, minus, dot, sort, cross-product, gradient, reduce, Fourier transform, integrate, etc.

[0137] Then, as shown in the figure mark 103, when the application determines to trigger the elastic body scenario, the load assessment-automatic scheduling module adaptively schedules the operations used in the physical calculation to the CPU or GPU for execution.

[0138] For example, Fig.11As shown, the process of the application determining the backend platform for physical computing in step 902 includes a load evaluation phase, an automatic scheduling phase, and a distribution execution phase.

[0139] In some examples, in the load assessment phase, the load assessment module analyzes the execution efficiency of different physical simulation scenarios on different devices, thereby dynamically adjusting whether the computing operations are performed on the GPU or the CPU in the subsequent automatic adjustment phase to obtain better physical simulation performance.

[0140] Exemplarily, before performing a specific physical simulation, the load assessment module first counts the CPU calculation time, GPU calculation time, and CPU-to-GPU data transfer time of each operator required in the current elastic body scene. Among them, the grid code of the image frame obtained by the application is stored in the CPU's memory, so if it is determined that the GPU needs to be used for calculation processing, then this part of the data needs to be transferred from the CPU to the GPU. In other words, in the load assessment stage, the data transfer time from the CPU to the GPU also needs to be considered.

[0141] For example, for operator i, the load evaluation module evaluates the CPU execution time cputime i , GPU execution time gputime i , and the data transfer time from CPU to GPU cpu2gputime i .

[0142] It should be understood that the hardware performance in different electronic devices may be different, so the calculation time of the same operator on different back-end platforms may also be different. Therefore, before selecting the back-end platform, the actual calculation time of each operator on different back-end platforms is first determined by the load evaluation module, so that accurate back-end platform scheduling can be achieved based on the actual calculation time. Therefore, the display method provided in the embodiment of the present application can be adaptively applied to the hardware platforms of different devices.

[0143] In some embodiments, after the electronic device downloads the theme, it can obtain the operators included in the elastic body calculation corresponding to the current theme. In the load evaluation stage, the load evaluation module needs to evaluate the execution time and data transmission time of these operators on different backend platforms. Among them, the elastic body calculation corresponding to the theme is used to realize the elastic body dynamic effect display on one or more interfaces of the current theme.

[0144] For example, a default theme is pre-installed in the electronic device. Before the electronic device runs the default theme, such as the first time, it can first obtain the operators included in the elastic body calculation corresponding to the default theme to evaluate the execution time and data transmission time of these operators on different back-end platforms. In this way, when the default theme is displayed later, the back-end platform used for calculation can be determined. For another example, after the electronic device downloads a custom theme in response to user operation, it can obtain the operators included in the elastic body calculation corresponding to the custom theme to evaluate the execution time and data transmission time of these operators on different back-end platforms. In this way, when the custom theme is displayed later, the back-end platform used for calculation can be determined.

[0145] For example, Fig.12 As shown, the elastic body calculation includes gradient calculation, integral calculation, multiple calculation, and Fourier transform calculation as an example. In the load assessment stage, the load assessment module determines that the CPU takes 3 milliseconds (ms) to perform gradient calculations, and the GPU takes 1 ms to perform gradient calculations; the CPU takes 3.5 ms to perform integral calculations, and the GPU takes 1 ms to perform integral calculations; the CPU takes 10 ms to perform multiple calculations, and the GPU takes 1 ms to perform integral calculations. Among them, the data transmission time from the CPU to the GPU is Fig.12 Not shown.

[0146] Afterwards, if Fig.11 As shown, after the load evaluation phase is over, the automatic scheduling module can obtain the load evaluation results output by the load evaluation module. Then, the automatic scheduling module can perform the automatic scheduling phase according to the load evaluation results and determine the scheduling CPU or GPU. For example, the automatic scheduling module uses the automatic scheduling algorithm to analyze the time consumed by each operator using the GPU or the CPU based on the calculation time of each operator, and determines which back-end platform is less time-consuming to calculate for a certain operator, so as to determine which back-end platform needs to be used to process the operator. Thereby improving the overall computing efficiency and improving the physical performance in elastic scenarios. Among them, the calculation time of the operator includes the CPU calculation time, GPU calculation time, and data transmission time from the CPU to the GPU of the operator previously determined by the load evaluation module.

[0147] Exemplarily, the automatic scheduling module determines the backend platform to be scheduled according to the number of vertices of the elastic body included in the image frame and the duration of each operator to be used. The elastic body is often represented by a grid in the computer, and the grid has a corresponding number of vertices. Fig.12As shown, in the current elastic body scene, the elastic body calculations involved include gradient calculation, integral calculation, multiple calculation, and Fourier transform calculation. In the automatic scheduling stage, the automatic scheduling module calculates the calculation time of various operators on the CPU and the calculation time on the GPU respectively according to the number of elastic body vertices to determine the scheduling result. For example, in the current elastic body scene, through the following formula (1), the automatic scheduling module determines that the gradient calculation, integral calculation, multiple calculation, and Fourier transform calculation can be calculated in a shorter time using the GPU. Therefore, the automatic scheduling module can instruct the distribution execution module to call the GPU for calculation.

[0148]

[0149] in,

[0150] For example, in the above formula (1), during the CPU calculation process, the status of the operator before and after will not change, that is, status i =status i+1 Then, the CPU calculation time of operator i is stepTime i =cuptime+0 1 ×cup2gputime=cuptime. During the GPU calculation process, since the operator needs to be transferred from the CPU to the GPU, the status of the operator will change before and after, that is, status i ≠status i+1 Then, the computation time of the GPU of operator i is stepTime i =gputime+0 0 ×cup2gputime=gputime+cup2gputime.

[0151] Afterwards, if Fig.11 As shown in , after the automatic scheduling phase ends, the distribution execution module can obtain the scheduling result output by the automatic scheduling module, and the scheduling result indicates the backend platform that each operator needs to run. Fig.12 As shown, the distribution execution module instructs each operator to run on the GPU according to the scheduling result, and the GPU performs the calculation.

[0152] Thus, compared with the current situation that only supports programmers to specify whether to compile a program into CPU code or GPU code during the development phase, the display processing method provided in the embodiment of the present application can dynamically switch the backend platform used for calculation at runtime, automatically select the backend platform with better performance according to different hardware platforms and different business scenarios, and achieve better physical simulation.

[0153] For example, based on the above-mentioned method of adaptively scheduling the backend platform, the electronic device can adaptively schedule to the CPU backend platform to perform calculations in low-load small-scale scenarios; and schedule to the GPU backend platform to perform calculations in high-load large-scale scenarios.

[0154] The following example illustrates an experimental scenario to illustrate the effect of the above-mentioned method of adaptive scheduling backend platform.

[0155] Exemplarily, based on the elastic body collision physics simulation application, the physical operation operators include integral operators, sampling operators, difference operators, linear equation operators, spatial gradient operators, matrix dot multiplication, and matrix cross multiplication. Through 100 iterations to correspond to the multi-vertex calculation process, the following test verification scenario is designed.

[0156] In some examples, such as test scenario 1 shown in Table 3 below, in a load scenario of the same scale, the tests are run on test machine 1 and test machine 2 respectively to test whether the computing power can be scheduled to run to a better performance. For example, as shown in Table 3 below, the CPU performance of test machine 1 and test machine 2 is the same. Test machine 1 determines to schedule the CPU by comparing the CPU performance and the GPU performance. For example, the CPU calculation time of test machine 1 is shorter. Test machine 2 determines to schedule the GPU by comparing the CPU performance and the GPU performance. For example, the GPU calculation time of test machine 2 is shorter. It can be seen that different test machines can be adaptively scheduled to the back-end platform with better computing power.

[0157] Table 3

[0158] Test scenario 1 CPU Performance GPU Performance Scheduling results Test machine 1 2460ms 3080ms CPU Test machine 2 2460ms 2030ms GPU

[0159] In some examples, such as the test scenario 2 shown in Table 4 below, different load scenarios are run on the test machine 1, such as running a 4x4 low-load elastic body falling collision scenario and a 12x12 high-load elastic body falling collision scenario, respectively, to test whether the computing power can be scheduled to run to a better performance. For example, as shown in Table 4 below, in the 4x4 low-load elastic body falling collision scenario, the test machine 1 determines to schedule the CPU by comparing the CPU performance and the GPU performance. For example, in the low-load elastic body falling collision scenario, the CPU calculation time is shorter. In the 12x12 high-load elastic body falling collision scenario, the test machine 1 determines to schedule the GPU by comparing the CPU performance and the GPU performance. For example, in the high-load elastic body falling collision scenario, the GPU calculation time is shorter. It can be seen that the same test machine can be adaptively scheduled to the back-end platform with better computing power according to different load scenarios.

[0160] Table 4

[0161] Test scenario 2 CPU Performance GPU Performance Scheduling results 4x4 2460ms 3080ms CPU 12x12 3370ms 2954ms GPU

[0162] In some embodiments, the application determines the backend platform through the above method and performs calculations through the backend platform. Among them, the elastomer is often represented by a grid in the computer, so in the elastomer scene, the backend platform can output a simulation grid. Among them, the simulation grid is a coarse force grid and cannot obtain a better display effect. At present, users have a great demand for high-quality and refined effects of electronic devices. Therefore, electronic devices need to process the simulation grid to obtain a finer-grained rendering grid that can achieve better display effects.

[0163] In some cases, the simulation mesh can also be described as a physical topology mesh.

[0164] The following describes the process of converting a simulation mesh into a rendering mesh.

[0165] In some embodiments, multiple subdivisions of a coarser-grained simulation grid can yield a finer-grained rendering grid. Therefore, during the development phase, programmers can pre-establish a mapping relationship between the simulation grid and the rendering grid, and directly pre-set the mapping relationship to the electronic device. In this way, when the electronic device needs to perform mesh subdivision, it can directly determine the rendering grid corresponding to the current simulation grid based on the mapping relationship. This can reduce the subdivision processing load while improving resolution and display effects.

[0166] For example, Fig.10 As shown by reference numeral 104 , in the physical engine functional service layer, a mapping matrix between the simulation grid and the rendering grid calculated offline is preset, and the mapping matrix is ​​used to indicate the mapping relationship from the simulation grid to the rendering grid.

[0167] like Fig.11 As shown in FIG, in the topology construction stage, the offline topology construction module creates the mapping relationship between the simulation mesh and the rendering mesh offline. For example, the offline topology construction module subdivides the simulation mesh to obtain a high-resolution rendering mesh. Fig.13 As shown, after obtaining the simulation mesh, the offline topology construction module will perform multiple loop subdivisions to output a rendering mesh that meets the frame rate requirements.

[0168] In some examples, the offline topology construction module defines x as the original simulation mesh vertex and x' as the mesh vertex after the next subdivision. The mapping relationship between the two is Among them, x j ′ is the jth subdivision mesh vertex, which is related to the vertices of the n original simulation meshes, x i is the i-th related simulation mesh vertex, w i is the corresponding weight coefficient.

[0169] Afterwards, the offline topology construction module sequentially substitutes the coarse mesh vertices of the previous step into the fine mesh vertices of the next step. Fig.13 As shown, the offline topology construction module is based on the vertex coordinates of the original simulation mesh Determine the mesh vertex coordinates obtained after the first loop subdivision: Thus, the first update of vertex coordinates is achieved. After that, in the n-times loop subdivision process, the offline topology construction module updates the mesh vertex coordinates in sequence. Finally, the offline topology construction module can establish the relationship between all original simulation mesh vertices x and all final rendering mesh vertices x" as shown below, thereby obtaining the mapping matrix w.

[0170]

[0171] In some embodiments, after the electronic device triggers the display of the elastic body scene and outputs the simulation grid through the back-end platform, Fig. 9 As shown, the simulation grid can be mapped to the rendering grid through a preset mapping relationship, thereby realizing the dynamic effect display based on the rendering grid (i.e., step 904 and step 905). In this way, based on the mapping relationship, the grid updated by the physical simulation is mapped to the high-resolution rendering grid once per frame to reduce the subdivision processing load.

[0172] For example, Fig.11 As shown in Figure 1, during the online computing process, the distribution execution module schedules the backend platform to perform the calculation and outputs the physical simulation mesh. Afterwards, the online acceleration module can obtain the mapping relationship preset by the offline topology construction module and directly map the physical simulation mesh to a refined rendering mesh based on the mapping relationship. Fig.14 As shown, through simple mapping processing of the mapping relationship, the application does not need to perform multiple loop subdivisions, and can also obtain a rendering grid with a higher resolution to achieve a better display effect.

[0173] Thus, compared with the existing subdivision shader, which can only achieve subdivision by modifying the interpolated vertices so that the generated surface follows the mesh normal, resulting in additional subdivision processing load, the display processing method provided in the embodiment of the present application can directly map the simulation mesh to the rendering mesh at one time according to the offline preset mapping relationship, thereby reducing the subdivision processing load while ensuring the subdivision effect.

[0174] Furthermore, since the mapping relationship between the simulation mesh and the rendering mesh is established in an offline process, a rendering mesh with a more delicate and smooth effect can be obtained by modifying the original vertices of the simulation mesh.

[0175] In addition, the display processing method provided in the embodiment of the present application is independent of the GPU hardware type, thus avoiding the possibility that different compiler types may cause different segmentation effects.

[0176] It should be understood that in some scenarios, the programmer pre-specifies the backend platform. Then, after the simulation grid output by the specified backend platform, the electronic device can also map the simulation grid to the rendering grid according to the mapping relationship. That is, the above-mentioned method of adaptively scheduling the backend platform and the method of obtaining the rendering grid based on the mapping relationship can be executed separately or jointly, and the embodiments of the present application do not limit this.

[0177] The following example illustrates an experimental scenario to illustrate the effect of the above method for obtaining a rendering grid based on a mapping relationship.

[0178] For example, on the same electronic device, in the same elastic body collision scene, three methods of calculating the rendering grid, namely, the GPU-side subdivision shader, the CPU-side classic loop subdivision method, and the mapping method provided in the embodiment of the present application, are used to compare the electronic device performance, the rendering grid resolution, and the subdivision effect. As shown in Table 5 below, the measurement items of the electronic device performance include frame rate, power consumption increment, and double data rate (DDR) frequency.

[0179] Table 5

[0180] result GPU-side tessellation shader Classic loop subdivision method on CPU Mapping Method Frame rate (fps) 56.1 12.0 58.5 Power consumption increase 124 The frame rate difference is large, so no comparison is made 57 DDR frequency 1641.2 2133.0 1167.9

[0181] According to the experimental results shown in Table 5 above, it can be seen that the mapping method provided by the embodiment of the present application has a power consumption benefit of 67 and a DDR frequency reduction of 473 compared to the GPU-side subdivision shader; compared to the CPU-side classic loop subdivision method, the frame rate is increased by 46.5fps and the DDR frequency is reduced by 965.1. In addition, the frame rate of the CPU-side classic loop subdivision method is only 12fps, which seriously drops frames and cannot be used in real-time scenarios of electronic devices. Therefore, the above experimental results do not compare the power consumption increment of the mapping method and the subdivision method.

[0182] also, Fig.15 (a) in the figure uses the classic loop subdivision method on the CPU side to obtain the display effect after rendering the mesh. Fig.15 (b) is the display effect after the rendered grid is obtained by using the mapping method provided in the embodiment of the present application. It can be clearly seen that the mapping method provided in the embodiment of the present application can obtain a better subdivision effect, so as to obtain a better display effect.

[0183] Fig.16 The flowchart of a display processing method provided by the embodiment of the present application is shown in FIG. Fig.16The specific order described below is a limitation, and it should be understood that in other embodiments, the order of some steps in the method can be interchanged according to actual needs, or some steps can be omitted or deleted. The method includes the following steps:

[0184] Step 1601: The electronic device determines a theme used by the electronic device in response to a first operation by the user, where the theme includes an elastic body displayed on a screen-off interface.

[0185] In some examples, the first operation is, for example, a user clicking on a display screen of an electronic device when the screen is black; or a user approaching the electronic device. When the user approaches the electronic device, the sensor of the electronic device can detect the user, thereby determining that the user needs to turn on the phone.

[0186] In some embodiments, after detecting the user's first operation, the electronic device determines that the screen-off interface needs to be displayed. In some examples, the electronic device is configured with a theme function, which can be used to set the display of the screen-off interface, lock screen interface, desktop, and other stages. For example, an elastomer is displayed on the screen-off interface. In some examples, the display content and display effects of different themes may be different. Then, after detecting the user's first operation, the electronic device can determine the elastomer displayed on the screen-off interface indicated in the theme currently used by the electronic device.

[0187] Step 1602: The electronic device obtains a simulation grid corresponding to the elastic body.

[0188] In some embodiments, after acquiring the elastomer, the electronic device may determine to trigger the simulation of the elastomer to output a simulation grid. In some examples, the electronic device may simulate the elastomer through the first platform and / or the second platform. To improve the simulation efficiency and the performance of the electronic device, the electronic device may select a platform that can achieve less time-consuming elastomer simulation.

[0189] In some examples, the following examples take the first platform as a CPU and the second platform as a GPU as an example to introduce the platform selection process in detail during the simulation process. It should be understood that in another example, the first platform may also be a GPU and the second platform may also be a CPU.

[0190] In some embodiments, the electronic device obtains a first simulation duration for simulating the elastic body by a first platform of the electronic device, and a second simulation duration for simulating the elastic body by a second platform of the electronic device. Afterwards, the electronic device calls the first platform and / or the second platform to simulate the elastic body according to the first simulation duration and the second simulation duration.

[0191] For example, when the first simulation duration is less than the second simulation duration, the electronic device calls the first platform to simulate the elastic body. That is, it takes less time when the first platform simulates the elastic body alone, so the electronic device can choose to call the first platform to simulate the elastic body.

[0192] For another example, when the second simulation duration is less than the first simulation duration, the electronic device calls the second platform to simulate the elastic body. That is, it takes less time when the second platform simulates the elastic body alone, so the electronic device can choose to call the second platform to simulate the elastic body.

[0193] For another example, when the third simulation duration is less than the first simulation duration and the second simulation duration, the electronic device calls the first platform and the second platform to simulate the elastic body, and the third simulation duration is the simulation duration required when the first platform and the second platform are called to perform cooperative simulation on the elastic body according to the first simulation duration and the second simulation duration. That is, it takes less time when the first platform and the second platform cooperate to simulate the elastic body, so the electronic device can choose to call the first platform and the second platform to simulate the elastic body.

[0194] For example, Fig.10 As shown, the lower layer of the physics engine includes a hybrid scheduling computing base of the GPU and the CPU, in which the encapsulated operators can be calculated by both the GPU and the CPU. Then, after the GPU executes the operator operation, the simulation grid can be uploaded to the physics engine business layer located in the upper layer for processing. Alternatively, after the CPU executes the operator operation, the simulation grid can be uploaded to the physics engine business layer located in the upper layer for processing. Alternatively, after the GPU and the CPU execute the operator operation respectively, their respective simulation results can be uploaded to the physics engine business layer located in the upper layer respectively, and the physics engine can obtain the simulation grid after processing the obtained GPU simulation results and CPU simulation results.

[0195] In some embodiments, after detecting the user's first operation and determining the theme used by the electronic device, the electronic device can obtain N operators required for the theme, where N is a positive integer. Afterwards, the electronic device can obtain the first calculation duration of each of the N operators executed by the first platform, the second calculation duration of each of the N operators executed by the second platform, and the data transmission duration from the first platform to the second platform.

[0196] For example, Fig.12As shown, the electronic device obtains N operators corresponding to the subject, which are gradient calculation, integral calculation, multiple calculation, and Fourier transform calculation. Then, the electronic device can determine the first calculation duration of the first platform to perform the gradient calculation, integral calculation, multiple calculation, and Fourier transform calculation respectively, the second calculation duration of the second platform to perform the gradient calculation, integral calculation, multiple calculation, and Fourier transform calculation respectively, and determine the data transmission duration of each of the N operators from the first platform to the second platform.

[0197] In this way, before the electronic device selects a platform, it first determines the actual calculation time of different platforms for each operator, so that accurate platform scheduling can be achieved based on the actual calculation time, thereby realizing adaptive hardware platforms applied to different devices.

[0198] In some embodiments, when the electronic device determines that an elastomer needs to be simulated, it can obtain the number of operators of N operators corresponding to the elastomer. For example, simulating an elastomer requires M gradient calculations, and the number of operators for gradient calculation is M, where M is a positive integer. Afterwards, the electronic device obtains the first duration for the first platform to execute the N operators respectively based on the number of operators and the first calculation duration; and obtains the second duration for the second platform to execute the N operators respectively based on the number of operators, the second calculation duration, and the data transmission duration, the second duration includes the cumulative transmission duration of each of the N operators transmitted from the first platform to the second platform. Afterwards, the electronic device determines the first simulation duration based on the first duration. Determine the second simulation duration based on the second duration.

[0199] In some examples, the electronic device determines that the first duration corresponding to the first type of operator among the N types of operators is less than the second duration, and calls the first platform to execute the first type of operator. The electronic device determines that the second duration corresponding to the second type of operator among the N types of operators is less than the first duration, and calls the second platform to execute the second type of operator.

[0200] For example, Fig.12 As shown, the electronic device determines that the first duration for the CPU to perform gradient calculation is 3ms, and the second duration for the GPU to perform gradient calculation is 1ms; the first duration for the CPU to perform integral calculation is 3.5ms, and the second duration for the GPU to perform integral calculation is 1ms; the first duration for the CPU to perform multiple calculation is 10ms, and the second duration for the GPU to perform integral calculation is 1ms. Then, the electronic device can determine that the first simulation duration for calling the CPU to simulate the elastomer is 16.5ms (3ms+3.5ms+10ms), and the first simulation duration for calling the GPU to simulate the elastomer is 3ms (1ms+1ms+1ms). Then, based on the fact that the second simulation duration is less than the first simulation duration, the electronic device can determine to call the GPU to simulate the elastomer to obtain a simulation grid.

[0201] As another example, the electronic device determines that the first duration for the CPU to perform gradient calculation is 3ms, and the second duration for the GPU to perform gradient calculation is 1ms; the first duration for the CPU to perform integral calculation is 3.5ms, and the second duration for the GPU to perform integral calculation is 1ms; the first duration for the CPU to perform multiple calculation is 1ms, and the second duration for the GPU to perform integral calculation is 10ms. Then, the electronic device can determine that the first type of operator is multiple calculation, the second type of operator is gradient calculation and integral calculation, and the sum of the duration for the first platform to execute the first type of operator and the duration for the second platform to execute the second type of operator is 3ms (1ms+1ms+1ms). Then, the electronic device can determine to call the CPU to perform multiple calculation, and call the GPU to perform gradient calculation and integral calculation, so that through the cooperation of the CPU and GPU, the simulation of the elastomer can be completed in a shorter time, and the simulation grid can be output, thereby improving the simulation efficiency.

[0202] Thus, compared with the current situation that only supports programmers to specify whether to compile a program into CPU code or GPU code during the development phase, the display processing method provided in the embodiment of the present application can dynamically switch the platform used for calculation at runtime, automatically select a platform with better performance according to different hardware platforms and different business scenarios, and achieve better physical simulation.

[0203] Step 1603: The electronic device maps the simulation grid to a rendering grid according to the mapping relationship.

[0204] In some embodiments, before the electronic device is used for the first time, the electronic device is able to obtain and save a mapping relationship, where the mapping relationship includes a mapping matrix for converting a simulation mesh into a rendering mesh.

[0205] For example, Fig.11 As shown, during the development stage, programmers can create a mapping relationship between a simulation grid and a rendering grid offline through an offline topology construction module, and directly preset the mapping relationship into the electronic device.

[0206] In this way, based on the mapping relationship, the electronic device can map the simulation mesh updated by the physical simulation to the high-resolution rendering mesh once per frame to reduce the subdivision processing load.

[0207] Thus, compared with the existing subdivision shader, which can only achieve subdivision by modifying the interpolated vertices so that the generated surface follows the mesh normal, resulting in additional subdivision processing load, the display processing method provided in the embodiment of the present application can directly map the simulation mesh to the rendering mesh at one time according to the offline preset mapping relationship, thereby reducing the subdivision processing load while ensuring the subdivision effect.

[0208] Furthermore, since the mapping relationship between the simulation mesh and the rendering mesh is established in an offline process, a rendering mesh with a more delicate and smooth effect can be obtained by modifying the original vertices of the simulation mesh.

[0209] In addition, the display processing method provided in the embodiment of the present application is independent of the GPU hardware type, thus avoiding the possibility that different compiler types may cause different segmentation effects.

[0210] Step 1604: The electronic device displays a screen-off interface including an elastic body according to the rendered grid.

[0211] In some embodiments, after acquiring the rendering grid, the electronic device can execute the subsequent drawing and rendering process of the elastic body, and then send the synthesized image frame to the display screen for display through layer synthesis. In this way, the electronic device can display the off-screen interface including the elastic body.

[0212] For example, Figure 6 As shown in (a), during the black screen process, the electronic device detects the first operation of the user on the display screen, obtains the sports ball theme currently used by the electronic device, obtains the elastic ball displayed on the screen-off interface included in the sports ball theme, simulates the elastic ball through the adaptive selection platform of step 1602 to obtain a simulation grid, and directly maps the simulation grid to a rendering grid based on the mapping relationship through step 1603, and can display the rendering grid based on the rendering grid. Figure 6 As shown in (b), the screen-off interface includes the elastic ball as shown in the function window 61.

[0213] As another example, while the electronic device is displaying the screen-off interface, it detects a second operation of the user on the screen-off interface, and calls the first platform and / or the second platform to simulate the elastomer. The electronic device displays a first animation of the elastomer from the screen-off interface to the lock screen interface. Afterwards, the electronic device may display a lock screen interface including the elastomer. Afterwards, while the electronic device is displaying the lock screen interface, it detects a third operation of the user on the lock screen interface, and calls the first platform and / or the second platform to simulate the elastomer. The electronic device displays a second animation of the elastomer from the lock screen interface to the desktop. Afterwards, the electronic device displays the desktop.

[0214] For example, Figure 6 As shown in (c), when the electronic device is displaying the screen-off interface, if the user's second operation on the display screen is detected, the first platform and / or the second platform may be called to simulate the elastic body to display the Figure 7 As shown in (a), the first animation of the elastic ball from the off screen interface to the lock screen interface. Afterwards, the electronic device may display the following Figure 7 The lock screen interface including elastic balls shown in (b) is shown in FIG.

[0215] In some examples, during displaying a first animation, displaying a lock screen interface including an elastomer, and displaying a second animation, the platform called by the electronic device is the same as the platform called by the electronic device during displaying a screen-off interface including an elastomer.

[0216] In this way, after the electronic device allocates a platform for simulating the elastomer according to the elastomer included in the theme, it can continuously call the same platform to simulate the elastomer displayed in multiple interfaces included in the same theme. Based on the consistency of the display effect on the interface at different stages of the theme, it can avoid unnecessary repeated selection of platforms while ensuring the display effect and display processing efficiency.

[0217] Combination of the above Figure 6-Figure 16 The display processing method provided by the embodiment of the present application is described in detail. Fig.17 The electronic device provided by the embodiments of the present application is described in detail.

[0218] In one possible design, Fig.17 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Fig.17 As shown, the electronic device 1700 may include: a transceiver unit 1701, a processing unit 1702, and a display unit 1703. The electronic device 1700 may be used to implement the functions of the electronic device involved in the above method embodiment.

[0219] Optionally, the transceiver unit 1701 is used to support the electronic device 1700 to execute Fig. 9 Step 901 in; and / or, for supporting the electronic device 1700 to perform Fig.16 Step 1601 in .

[0220] Optionally, the processing unit 1702 is used to support the electronic device 1700 to execute Fig. 9 Step 902, step 903, and step 904 in; and / or, for supporting the electronic device 1700 to perform Fig.16 Step 1602 and step 1603 in .

[0221] Optionally, the display unit 1703 is used to support the electronic device 1700 to execute Fig. 9 Step 905 in; and / or, for supporting the electronic device 1700 to perform Fig.16 Step 1604 in .

[0222] Among them, the transceiver unit may include a receiving unit and a sending unit, and may be implemented by a transceiver or a transceiver-related circuit component, and may be a transceiver or a transceiver module. The operations and / or functions of each unit in the electronic device 1700 are respectively to implement the corresponding process of the display processing method described in the above method embodiment. All relevant contents of each step involved in the above method embodiment can be referred to the functional description of the corresponding functional unit, and for the sake of brevity, they will not be repeated here.

[0223] Optionally, Fig.17 The electronic device 1700 shown may also include a storage unit ( Fig.17 (not shown), the storage unit stores a program or instruction. When the transceiver unit 1701, the processing unit 1702 and the display unit 1703 execute the program or instruction, Fig.17 The electronic device 1700 shown can execute the display processing method described in the above method embodiment.

[0224] Fig.17 The technical effects of the electronic device 1700 shown can refer to the technical effects of the display processing method described in the above method embodiment, and will not be repeated here.

[0225] In addition to being in the form of electronic device 1700, the technical solution provided in the present application may also be a functional unit or chip in the electronic device, or a device used in conjunction with the electronic device.

[0226] An embodiment of the present application also provides a chip system, including: a processor, the processor is coupled to a memory, the memory is used to store programs or instructions, when the program or instructions are executed by the processor, the chip system implements the method in any of the above method embodiments.

[0227] Optionally, the processor in the chip system may be one or more. The processor may be implemented by hardware or by software. When implemented by hardware, the processor may be a logic circuit, an integrated circuit, etc. When implemented by software, the processor may be a general-purpose processor implemented by reading software code stored in a memory.

[0228] Optionally, the memory in the chip system may also be one or more. The memory may be integrated with the processor or may be separately arranged with the processor, which is not limited in the embodiments of the present application. Exemplarily, the memory may be a non-transient processor, such as a read-only memory ROM, which may be integrated with the processor on the same chip or may be arranged on different chips respectively. The embodiments of the present application do not specifically limit the type of memory and the arrangement of the memory and the processor.

[0229] Exemplarily, the chip system can be a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on chip (SoC), a central processor unit (CPU), a network processor (NP), a digital signal processor (DSP), a microcontroller unit (MCU), a programmable logic device (PLD) or other integrated chips.

[0230] It should be understood that each step in the above method embodiment can be completed by an integrated logic circuit of hardware in a processor or by instructions in the form of software. The method steps disclosed in the embodiments of the present application can be directly embodied as being executed by a hardware processor, or by a combination of hardware and software modules in a processor.

[0231] An embodiment of the present application also provides a computer-readable storage medium, in which a computer program is stored. When the computer program is run on a computer, the computer executes the above-mentioned related steps to implement the display processing method in the above-mentioned embodiment.

[0232] The embodiment of the present application also provides a computer program product. When the computer program product is run on a computer, the computer is enabled to execute the above-mentioned related steps to implement the display processing method in the above-mentioned embodiment.

[0233] In addition, an embodiment of the present application further provides a device. The device may be a component or a module, and the device may include one or more processors and a memory connected to each other. The memory is used to store a computer program. When the computer program is executed by one or more processors, the device performs the display processing method in the above-mentioned method embodiments.

[0234] Among them, the device, computer-readable storage medium, computer program product or chip provided in the embodiments of the present application are used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding methods provided above, and will not be repeated here.

[0235] The steps of the method or algorithm described in conjunction with the disclosure of the embodiments of the present application can be implemented in hardware or by executing software instructions by a processor. The software instructions can be composed of corresponding software modules, and the software modules can be stored in random access memory (RAM), flash memory, read only memory (ROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, mobile hard disks, CD-ROMs or any other form of storage medium known in the art. An exemplary storage medium is coupled to a processor so that the processor can read information from the storage medium and can write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an application specific integrated circuit (ASIC).

[0236] Through the description of the above implementation methods, those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above functional modules is used as an example. In practical applications, the above functions can be assigned to different functional modules as needed; that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. The specific working process of the system, device and unit described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0237] In the several embodiments provided in the present application, it should be understood that the disclosed method can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the modules or units is only a logical function division, and there may be other division methods in actual implementation; for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of modules or units, which can be electrical, mechanical or other forms.

[0238] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0239] Computer-readable storage media include, but are not limited to, any of the following: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, and other media that can store program codes.

[0240] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A display processing method, characterized in that: Applied to electronic equipment, the method comprises: In response to a first operation of a user, determining a theme used by the electronic device, the theme comprising an elastic body displayed on a screen-off interface; Obtaining a simulation grid corresponding to the elastic body; According to the mapping relationship, mapping the simulation grid to a rendering grid; According to the rendering grid, a screen-off interface including the elastic body is displayed.

2. The method according to claim 1, characterized in that The obtaining of the simulation grid corresponding to the elastic body comprises: Acquire a first simulation duration for simulating the elastomer by a first platform of the electronic device, and a second simulation duration for simulating the elastomer by a second platform of the electronic device; According to the first simulation duration and the second simulation duration, the first platform and / or the second platform are called to simulate the elastomer.

3. The method according to claim 2, characterized in that The calling the first platform and / or the second platform to simulate the elastic body according to the first simulation duration and the second simulation duration includes: When the first simulation duration is less than the second simulation duration, calling the first platform to simulate the elastic body; or, When the second simulation duration is shorter than the first simulation duration, calling the second platform to simulate the elastic body; or, When the third simulation duration is less than the first simulation duration and the second simulation duration, the first platform and the second platform are called to simulate the elastomer, and the third simulation duration is the simulation duration required for calling the first platform and the second platform to perform cooperative simulation of the elastomer determined according to the first simulation duration and the second simulation duration.

4. The method according to claim 2 or 3, characterized in that: Before obtaining a first simulation duration for simulating the elastomer by the first platform and a second simulation duration for simulating the elastomer by the second platform, the method further includes: Obtain N operators corresponding to the topic, where N is a positive integer; Obtain a first calculation duration for the first platform to execute each of the N operators, a second calculation duration for the second platform to execute each of the N operators, and a data transmission duration for each of the N operators from the first platform to the second platform.

5. The method according to claim 4, characterized in that The obtaining of a first simulation duration of the first platform simulating the elastomer and a second simulation duration of the second platform simulating the elastomer comprises: Obtain the number of operators of N operators corresponding to the elastic body; According to the number of operators and the first calculation duration, obtaining first durations for the first platform to respectively execute the N types of operators; and obtaining, according to the number of operators, the second calculation duration, and the data transmission duration, second durations for the second platform to respectively execute the N types of operators, where the second duration includes a cumulative transmission duration for the first platform to transmit each of the N types of operators to the second platform; Determining the first simulation duration according to the first duration; The second simulation duration is determined according to the second duration.

6. The method according to claim 5, characterized in that The method further comprises: Determine that the first duration corresponding to a first type of operator among the N types of operators is less than the second duration, and call the first platform to execute the first type of operator; It is determined that the second duration corresponding to the second type of operators among the N types of operators is less than the first duration, and the second platform is called to execute the second type of operators.

7. The method according to any one of claims 1 to 6, characterized in that: The method further comprises: Detecting a second operation of the user on the screen-off interface, calling the first platform and / or the second platform to simulate the elastic body; Displaying a first animation of the elastic body from the screen-off interface to the screen-lock interface; The lock screen interface including the elastomer is displayed.

8. The method according to claim 7, characterized in that The method further comprises: Detecting a third operation of the user on the lock screen interface, calling the first platform and / or the second platform to simulate the elastic body; Display a second animation of the elastic body from the lock screen interface to the desktop; The desktop is displayed.

9. The method according to claim 7 or 8, characterized in that: During the process of displaying the first animation, displaying the lock screen interface including the elastomer, and displaying the second animation, the platform called by the electronic device is the same as the platform called by the electronic device during the process of displaying the screen-off interface including the elastomer.

10. The method according to any one of claims 1 to 9, characterized in that: Before determining the theme used by the electronic device in response to the first operation of the user, the method further includes: The mapping relationship is acquired and saved, wherein the mapping relationship includes a mapping matrix used to convert the simulation mesh into the rendering mesh.

11. An electronic device, characterized in that: include: A processor, a memory and a display screen, wherein the memory and the display screen are coupled to the processor, the memory is used to store computer program code, and the computer program code includes computer instructions. When the processor reads the computer instructions from the memory, the electronic device executes the method as described in any one of claims 1 to 10.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium comprises a computer program, and when the computer program is executed on an electronic device, the electronic device is enabled to execute the method according to any one of claims 1 to 10.

13. A computer program product, characterized in that When the computer program product is executed on a computer, the computer is caused to execute the method according to any one of claims 1 to 10.

Images