Image processing method and electronic device

Abstract

The present application provides an image processing method and electronic device, which can integrate GPU rendering in multiple places in the original image processing, reduce the duration of the synthesizer SurfaceFlinger, and send it to the LCD display in time, reducing the performance problems of frame loss and freeze. The method includes: generating a first rendering command according to first rendering information; generating a second rendering command according to second rendering information, the second rendering information includes the dynamic rendering information of the current layer, wherein the generating of the first rendering command according to the first rendering information and the generating of the second rendering command according to the second rendering information are completed in the same stage; sending the first rendering command and the second rendering command to the graphics processor; setting a first indication information, the first indication information instructs the synthesizer to no longer render the current layer according to the second rendering information; performing layer synthesis according to the image data of multiple layers to obtain the current image frame.

Description

Technical Field

[0001] The present application relates to the field of image processing and display technology, and more specifically, to an image processing method and electronic device. Background Art

[0002] Smooth and natural motion effects will make the interface of electronic devices smoother and more comfortable to use. They can enhance the quality of the entire electronic device and improve the user experience. Common motion effect scenes of electronic devices include enlarging the application launch icon, rounded corner cutting of icons, shrinking the application to the desktop after exiting, entering the multi-tasking card after exiting the application, etc. From these motion effect scenes, it is easy to extract common motion effect processes such as rotation, scaling, and rounded corner cutting. In the current image processing process, since the display subsystem (DSS) does not support rounded corner cutting, does not support high-ratio scaling, and does not support multiple channel rotation, it will be rendered by the graphics processing unit (GPU). However, in the image synthesis stage, GPU rendering will cause the synthesis time to become longer, and it cannot be sent to the liquid crystal display panel (LCD) for display in time, and the motion effect is not smooth, affecting the user experience. Summary of the invention

[0003] The present application provides an image processing method and electronic device, which can integrate GPU rendering in multiple places in the original image processing, and simultaneously reduce the duration of the synthesizer SurfaceFlinger, and then send it to the LCD display in time, reducing the performance problems of frame loss and jamming.

[0004] In a first aspect, a method for image processing is provided, the method comprising: generating a first rendering command according to first rendering information, the first rendering information comprising rendering information generated by controls and layout of a current layer; generating a second rendering command according to second rendering information, the second rendering information comprising dynamic rendering information of the current layer, wherein the generating of the first rendering command according to the first rendering information and the generating of the second rendering command according to the second rendering information are completed at the same stage; sending the first rendering command and the second rendering command to a graphics processor; setting first indication information, the first indication information being used to instruct a synthesizer to no longer render the current layer according to the second rendering information;

[0005] Layer synthesis is performed based on image data of multiple layers to obtain a current image frame, wherein the image data of the multiple layers include image data of the current layer, and the image data of the current layer is rendered by the graphics processor based on the first rendering command and the second rendering command.

[0006] In this method, the first rendering command and the second rendering command are generated at the same stage, and the rendering of the current layer's dynamic effect characteristics and the original GPU rendering (the original GPU rendering refers to the GPU rendering performed according to the first rendering command) are executed together. In this way, the GPU renderings in multiple places in the original image processing can be integrated, and the duration of the synthesizer SurfaceFlinger can be reduced simultaneously. SurfaceFlinger can use DSS online superposition and then send it to the LCD display in time, reducing the performance problems of frame loss and jamming, thereby achieving continuous and smooth dynamic effects.

[0007] In combination with the first aspect, in certain implementations of the first aspect, before sending the first rendering command and the second rendering command to the graphics processor, the method further includes: determining a priority level of the method in the operating system based on second indication information.

[0008] Optionally, the second indication information may be a pre-set level, such as setting the priority level of image processing to the highest level, so as to give priority to the user's visual experience.

[0009] Optionally, the second indication information may also be to determine the priority level of image processing according to the current system operation load.

[0010] In combination with the first aspect, in some implementations of the first aspect, the motion effect rendering information includes at least one of the following parameters: rotation, scaling, and fillet radius.

[0011] In combination with the first aspect, in some implementations of the first aspect, the method further includes: determining the first rendering information and the second rendering information according to a motion rendering strategy.

[0012] In combination with the first aspect, in some implementations of the first aspect, the motion rendering strategy includes: when a first parameter included in the first rendering information is different from a first parameter included in the second rendering information, adjusting the first parameter included in the first rendering information according to the first parameter included in the second rendering information.

[0013] In combination with the first aspect, in some implementations of the first aspect, the method further includes: generating a first rendering command according to first rendering information in response to a first vertical synchronization signal, the first rendering information including rendering information generated by controls and layouts of a current layer; generating a second rendering command according to second rendering information, the second rendering information including dynamic rendering information of the current layer, wherein the generating of the first rendering command according to the first rendering information and the generating of the second rendering command according to the second rendering information are completed at the same stage; sending the first rendering command and the second rendering command to a graphics processor; setting first indication information, the first indication information being used to instruct the synthesizer to no longer render the current layer according to the second rendering information;

[0014] In response to a second vertical synchronization signal, layer synthesis is performed based on image data of multiple layers to obtain a current image frame, wherein the image data of the multiple layers include image data of the current layer, and the image data of the current layer is rendered by the graphics processor based on the first rendering command and the second rendering command.

[0015] In a second aspect, an electronic device is provided, comprising: a processing unit, configured to generate a first rendering command according to first rendering information, the first rendering information including rendering information generated by controls and layout of a current layer; the processing unit is also configured to generate a second rendering command according to second rendering information, the second rendering information including dynamic rendering information of the current layer, wherein the generating of the first rendering command according to the first rendering information and the generating of the second rendering command according to the second rendering information are completed at the same stage; a transceiver unit, configured to send the first rendering command and the second rendering command to a graphics processor; the processing unit is also configured to set first indication information, the first indication information being used to instruct a synthesizer to no longer render the current layer according to the second rendering information; the processing unit is also configured to perform layer synthesis according to image data of multiple layers to obtain a current image frame, the image data of the multiple layers including image data of the current layer, the image data of the current layer being rendered by the graphics processor according to the first rendering command and the second rendering command.

[0016] In combination with the second aspect, in some implementations of the second aspect, before sending the first rendering command and the second rendering command to the graphics processor, the processing unit is further configured to:

[0017] According to the second indication information, the priority level of the electronic device in the operating system is determined.

[0018] In combination with the second aspect, in some implementations of the second aspect, the motion effect rendering information includes at least one of the following parameters: rotation, scaling, and fillet radius.

[0019] In combination with the second aspect, in some implementations of the second aspect, the processing unit is further used to: determine the first rendering information and the second rendering information according to a motion rendering strategy.

[0020] In combination with the second aspect, in some implementations of the second aspect, the motion rendering strategy includes: when a first parameter included in the first rendering information is different from a first parameter included in the second rendering information, adjusting the first parameter included in the first rendering information according to the first parameter included in the second rendering information.

[0021] In combination with the second aspect, in some implementations of the second aspect, the processing unit is also used to: generate a first rendering command according to first rendering information in response to a first vertical synchronization signal, the first rendering information including rendering information generated by controls and layout of the current layer; generate a second rendering command according to second rendering information, the second rendering information including dynamic rendering information of the current layer, wherein the generating of the first rendering command according to the first rendering information and the generating of the second rendering command according to the second rendering information are completed at the same stage; send the first rendering command and the second rendering command to the graphics processor; set first indication information, the first indication information is used to instruct the synthesizer to no longer render the current layer according to the second rendering information; in response to the second vertical synchronization signal, perform layer synthesis according to image data of multiple layers to obtain a current image frame, the image data of the multiple layers including image data of the current layer, and the image data of the current layer is rendered by the graphics processor according to the first rendering command and the second rendering command.

[0022] In a third aspect, a terminal device is provided, comprising a processor, wherein the processor is connected to a memory, the memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory, so that the terminal device executes the method in the above-mentioned first aspect or any possible implementation of the first aspect.

[0023] According to a fourth aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed, the method in the above-mentioned first aspect or any possible implementation manner of the first aspect is implemented.

[0024] In a fifth aspect, a chip is provided, characterized in that it includes a processor and an interface; the processor is used to read instructions to execute the method in the above-mentioned first aspect or any possible implementation of the first aspect.

[0025] Optionally, the chip may further include a memory, in which instructions are stored, and the processor is used to execute the instructions stored in the memory or instructions derived from other sources. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 It is a schematic diagram of the software processing flow of the electronic device according to the embodiment of the present application;

[0027] Figure 2 is a schematic diagram of the processing flow of the existing graphics system of the embodiment of the present application;

[0028] Figure 3 This is a schematic diagram of an unsmooth dynamic effect in an embodiment of the present application;

[0029] Figure 4 This is a schematic diagram of a deep-level disassembly of an unsmooth dynamic effect in an embodiment of the present application;

[0030] Figure 5 is a schematic flow chart of an image processing method according to an embodiment of the present application;

[0031] Figure 6 is a schematic block diagram of the graphics system processing flow provided by this application;

[0032] Figure 7 is a schematic diagram of an image processing pipeline provided in an embodiment of the present application;

[0033] Figure 8 is a schematic flow chart of an image processing method provided in an embodiment of the present application;

[0034] Fig. 9 A schematic block diagram of a device according to an embodiment of the present application is shown;

[0035] Fig.10 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0036] Fig.11 A schematic diagram of the structure of a terminal device provided for this application. DETAILED DESCRIPTION

[0037] The technical solution in this application will be described below in conjunction with the accompanying drawings.

[0038] In order to better understand the present application, the terms involved in the embodiments of the present application are introduced as follows:

[0039] Vertical synchronization signal: such as Vsync. The vertical synchronization signal can be used to trigger the drawing of one or more layers. It should be noted that in the embodiment of the present application, "the vertical synchronization signal can be used to trigger the drawing of one or more layers" specifically means: the vertical synchronization signal can be used to trigger the drawing of one or more layers, and trigger the rendering of the one or more layers. That is, in the embodiment of the present application, the one or more layers drawn refer to one or more layers after rendering. In the embodiment of the present application, the electronic device, in response to the vertical synchronization signal, can draw one or more layers for each application respectively through each drawing thread in multiple drawing threads. That is, the electronic device, in response to the vertical synchronization signal, can perform drawing tasks for one or more applications at the same time to draw one or more layers corresponding to each application.

[0040] Vsync can also be used to trigger layer synthesis of one or more drawn layers to obtain image frames, and can also be used to trigger hardware to refresh the display image frames.

[0041] It should be noted that the name of the vertical synchronization signal may be different in different systems or architectures. For example, in some systems or architectures, the name of the vertical synchronization signal (i.e., vertical synchronization signal 1) used to trigger the drawing of one or more layers may not be Vsync. However, no matter what the name of the vertical synchronization signal is, as long as it is a synchronization signal with similar functions and conforms to the technical ideas of the method provided in the embodiment of the present application, it should be covered within the scope of protection of the present application.

[0042] The central processing unit (CPU) is the computing and control core of the computer system and the final execution unit for information processing and program running.

[0043] A graphics processing unit (GPU) is a processor specifically used for image computing. The core component of a computer system, which is usually called a "graphics card", is the GPU.

[0044] The embodiment of the present application provides an image processing method, which can be applied to an electronic device including a touch screen. Specifically, the method can be applied to the electronic device in response to a user's touch operation on the touch screen, during the process of the touch screen displaying an image.

[0045] Please refer to Figure 1 , which takes the user operation as a touch operation as an example, and shows the software processing flow diagram of the electronic device from "the user's finger inputs the touch operation on the touch screen" to "the touch screen displays the image corresponding to the touch operation". Figure 1As shown, the electronic device may include: a touch panel (touch panel, TP) / TP driver (Driver) 10, an Input framework (ie, Input Framework) 20, a UI framework (ie, UI Framework) 30, a Display framework (ie, Display Framework) 40 and a hardware display module 50.

[0046] like Figure 1 As shown, the software processing flow of the electronic device may include the following steps (1) to (5).

[0047] Step (1): After the TP in the TPIC / TP driver 10 collects the touch operation of the user's finger on the TP of the electronic device, the TP driver reports the corresponding touch event to the Event Hub.

[0048] Step (2): The Input Reader thread of the Input framework 20 can read the touch event from the Event Hub, and then send the touch event to the Input Dispatcher thread; the Input Dispatcher thread uploads the touch event to the UI thread (such as DoFrame) in the UI framework 30.

[0049] Step (3): The UI thread in the UI framework 30 draws one or more layers corresponding to the touch event; the rendering thread (such as DrawFrame) performs layer rendering on the one or more layers.

[0050] Step (4): The synthesis thread in the Display framework 40 performs layer synthesis on the drawn one or more layers (ie, the one or more layers after rendering) to obtain an image frame.

[0051] Step (5): The liquid crystal display panel (LCD) of the hardware display module 50 is driven to receive the synthesized image frame, and the synthesized image frame is displayed by the LCD. After the LCD displays the image frame, the image displayed by the LCD can be perceived by human eyes.

[0052] After the user inputs a touch operation on the touch screen, the touch screen displays the image or action corresponding to the touch operation. For example, when the user touches the WeChat application icon on the touch screen, the WeChat application will open. During the WeChat application opening process, the WeChat icon will slowly enlarge until the WeChat interface opens. This process of the WeChat icon slowly enlarging is called motion effect. Motion effect is the result of image display. Figure 2 This is the existing graphics system processing flow. Figure 2As shown, the entire graphics system passes through the drawing main thread (Ueser Interface Thread, UI Thread), and UI Thread mainly calculates display content, such as view creation, measurement, layout, graphics drawing, etc. UI Thread can obtain attribute information of one or more layers included in the current image, and the attribute information includes first rendering information and second rendering information. The first rendering information includes rendering information generated by controls and layouts of one or more layers, and the second rendering information includes dynamic rendering information of one or more layers; the rendering main thread (Render Thread) can obtain the first rendering information of the current image, and convert the first rendering information of the current image into a first rendering command that can be recognized by the GPU. Render Thread passes the first rendering command to the graphics processor (Graphics Processing Unit, GPU), and the GPU renders according to the first rendering command; the synthesizer SurfaceFlinger can obtain the second rendering information of the current image, and SurfaceFlinger passes the second rendering information of the current image to the display subsystem (Display SubSystem, DSS) module. The DSS module decides whether to perform DSS online synthesis or graphics processor (Graphics Processing Unit, GPU) rendering synthesis based on its own capabilities. When there are many channels such as scaling, rounded corner cutting, and rotation, the DSS online synthesis conditions are not met, and GPU rendering synthesis is required. SurfaceFlinger converts the second rendering information of the current image into a second rendering synthesis command that can be recognized by the GPU. SurfaceFlinger passes the second rendering synthesis command to the graphics processing unit (GPU), and the GPU renders and synthesizes according to the second rendering synthesis command.

[0053] Figure 3 This is a schematic diagram of the existing graphics system processing flow. Since DSS does not support rounded corner cutting, multi-channel rotation, or scaling greater than 1.8 times, GPU rendering is required during the synthesis stage. Under the current terminal device system, most synthesis times exceed 16.6ms and cannot be sent to the LCD display immediately, resulting in unsmooth motion effects and affecting user experience.

[0054] Figure 4This is a deep disassembly diagram of the unsmooth dynamic effect. When the first Vsync clock comes, the UI and Render need to perform animation logic, measurement layout, drawing, synchronization (synchronization refers to the synchronization of UI resources to Render, such as synchronizing the first rendering information of the layer generated by the UI to the Render), command submission (command submission refers to the Render converting the first rendering information of the layer into a command that the GPU can recognize and sending it to the GPU) and exchange cache (exchange cache is the cache address where the GPU stores the rendering results after the Render sends the GPU to Surfaceflinger to render the layer). When the second Vsync clock comes, Surfaceflinger needs to perform message processing, synthesis preparation, memory acquisition (acquiring the rendered image data), rendering layer (Surfaceflinger calls the GPU to render the layer), command submission, layer synthesis, and refresh display. However, due to the dynamic effect characteristics such as rotation, scaling, and rounded corner cutting, GPU rendering is required, and GPU rendering needs to complete operations such as memory acquisition, rendering layer, and command submission. The rendering layer operation is time-consuming, and as the number of layers increases and the rendering complexity increases, the time consumption will increase linearly. This brings performance problems, resulting in an unsmooth dynamic effect process, and occasional freezes, which reduces the user experience.

[0055] At present, in order to ensure smooth screen effects, the more common method is to increase the frequency of the CPU and GPU, speed up the processing speed of the CPU and GPU, thereby reducing the execution time and ensuring that it can be sent to the LCD display in time before the next Vsync signal arrives. However, this will cause power consumption loss, reduce the battery life of the mobile phone, and the overall energy efficiency ratio is not high.

[0056] Smooth and natural motion effects will make the interface of electronic devices smoother and very comfortable to use, which will enhance the quality of the entire electronic device and improve the user experience. Currently common motion effect scenes for electronic devices include enlarging the application startup icon, rounded corner cutting of icons, shrinking the application to the desktop, exiting the application and entering the multi-tasking card, etc. From these motion effect scenes, it is easy to extract common motion effect processes such as rotation, scaling, and rounded corner cutting. Since DSS does not support rounded corner cutting, does not support high-ratio scaling, and does not support multiple channel rotations, it will use GPU rendering, but GPU rendering will cause the synthesis time to become longer, bringing a series of performance problems. How to ensure that the specific rendering layers, command submission and other operations of the dynamic effect characteristics such as rotation, scaling, and rounded corner cutting are processed in the appropriate location is a technical problem to be solved in this application.

[0057] An image processing method provided in an embodiment of the present application can perform rendering of window dynamic effects such as rotation, scaling, and rounded corner cutting in the Render stage, that is, the rendering of window dynamic effects such as rotation, scaling, and rounded corner cutting is performed together with the original GPU rendering (the original GPU rendering refers to the GPU rendering performed according to the first rendering command). In this way, the GPU renderings at multiple locations can be integrated and the SurfaceFlinger duration can be reduced simultaneously. SurfaceFlinger can use DSS online superposition and then send it to the LCD display in time, without causing performance problems such as frame loss and stuttering, so that the dynamic effect continues to be smooth.

[0058] Exemplarily, the electronic device in the embodiments of the present application may be a mobile phone, a tablet computer, a desktop, a laptop, a handheld computer, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, as well as a cellular phone, a personal digital assistant (PDA), an augmented reality (AR) device, a virtual reality (VR) device, and the like including a touch screen. The embodiments of the present application do not impose any special restrictions on the specific form of the electronic device.

[0059] The implementation of the embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0060] Combine the following Figure 5 Describe in detail an image processing method provided by the present application, Figure 5 is a schematic flow chart of an image processing method 200 according to an embodiment of the present application. The method 200 can be applied in Figure 1 The scenario shown can of course also be applied to other touch device scenarios, and the embodiments of the present application are not limited here.

[0061] It should also be understood that in the embodiments of the present application, the electronic device is used as an example to illustrate the method. As an example but not a limitation, the execution subject of the method may also be a chip, a chip system, or a processor applied to the electronic device.

[0062] like Figure 5 As shown, Figure 5 The method 200 shown in FIG. 2 may include S201 to S20. Figure 5 Each step in method 200 is described in detail.

[0063] S201, the electronic device generates a first rendering command according to first rendering information, where the first rendering information includes rendering information generated by controls and layout of a current layer.

[0064] S202, the electronic device generates a second rendering command according to second rendering information, where the second rendering information includes dynamic rendering information of the current layer, wherein generating the first rendering command according to the first rendering information and generating the second rendering command according to the second rendering information are completed at the same stage.

[0065] S203: The electronic device sends the first rendering command and the second rendering command to a graphics processor.

[0066] S204, the electronic device sets first indication information, where the first indication information is used to instruct the synthesizer to no longer render the current layer according to the second rendering information.

[0067] S205, the electronic device performs layer synthesis based on image data of multiple layers to obtain a current image frame, wherein the image data of the multiple layers include image data of the current layer, and the image data of the current layer is rendered by the graphics processor according to the first rendering command and the second rendering command.

[0068] It should be understood that, because the second rendering information includes the dynamic effect rendering information of the current layer, the second rendering information can also be referred to as dynamic effect rendering information.

[0069] Therefore, by executing motion rendering together with the original GPU rendering, the GPU rendering in multiple places can be integrated and the SurfaceFlinger duration can be reduced simultaneously. SurfaceFlinger can use DSS online superposition and then send it to the LCD display in time without causing performance issues such as frame loss and stuttering, so that the motion effect continues to be smooth.

[0070] In order to more clearly understand the present application, the steps included in the method 200 are described in detail below.

[0071] The first rendering information and the second rendering information are mainly required by the UI application on the electronic device to perform animation logic, measurement layout, drawing and other operations to obtain the first rendering information and the second rendering information of the current layer.

[0072] In S201, the electronic device generates a first rendering command according to the first rendering information. Mainly, Render generates the first rendering command according to the first rendering information. The first rendering command is a command recognizable by a GPU.

[0073] In S202, the electronic device generates a second rendering command according to the second rendering information, mainly the rendering enhancement device generates the second rendering command according to the second rendering information, the second rendering command is a command recognizable by the GPU, wherein the rendering enhancement device is a thread parallel to Render, and the generating of the first rendering command according to the first rendering information and the generating of the second rendering command according to the second rendering information are completed at the same stage.

[0074] In S203, the electronic device sending the first rendering command and the second rendering command to the graphics processor GPU means that Render sends the first rendering command to the GPU, and the rendering enhancement device sends the second rendering command to the GPU.

[0075] When the GPU receives the first command and the second command, the GPU renders the layer according to the first rendering command and the second rendering command to obtain the image data. The GPU performs the rendering of the motion effect feature and the original GPU rendering at the same time, so that the GPU rendering in multiple places can be integrated to reduce the total GPU rendering time, and there is no need to perform GPU rendering during SurfaceFlinger, which simultaneously reduces the SurfaceFlinger time. SurfaceFlinger can use DSS online superposition and then send it to the LCD display in time, without causing performance problems such as frame loss and jamming, so that the motion effect continues to be smooth.

[0076] In S205, the electronic device performs layer synthesis according to the image data of the multiple layers to obtain the current image frame, which means that SurfaceFlinger reads the image data in the cache area to perform layer synthesis.

[0077] Optionally, before sending the first rendering command and the second rendering command to the graphics processor, the method further includes: determining a priority level of the method in the operating system according to second indication information.

[0078] Optionally, the second indication information may be a pre-set level, such as setting the priority level of image processing to the highest level, so as to give priority to the user's visual experience.

[0079] Optionally, the second indication information may also be to determine the priority level of image processing according to the current system operating load.

[0080] Optionally, the motion effect rendering information includes at least one of the following parameters: rotation, scaling, and fillet radius.

[0081] It should be understood that the motion effect rendering information may also include other parameters, such as the radius of the fillet, etc.

[0082] Optionally, the method further includes: determining the first rendering information and the second rendering information according to a motion rendering strategy.

[0083] Optionally, the motion rendering strategy includes: when a first parameter included in the first rendering information is different from a first parameter included in the second rendering information, adjusting the first parameter included in the first rendering information according to the first parameter included in the second rendering information.

[0084] Optionally, the method further includes: generating a first rendering command according to first rendering information in response to a first vertical synchronization signal, the first rendering information including rendering information generated by controls and layout of the current layer; generating a second rendering command according to second rendering information, the second rendering information including dynamic rendering information of the current layer, wherein the generating of the first rendering command according to the first rendering information and the generating of the second rendering command according to the second rendering information are completed at the same stage; sending the first rendering command and the second rendering command to a graphics processor; setting first indication information, the first indication information being used to instruct the synthesizer to no longer render the current layer according to the second rendering information;

[0085] In response to a second vertical synchronization signal, layer synthesis is performed based on image data of multiple layers to obtain a current image frame, wherein the image data of the multiple layers include image data of the current layer, and the image data of the current layer is rendered by the graphics processor based on the first rendering command and the second rendering command.

[0086] Figure 6 is a schematic block diagram of the graphics system processing flow provided by this application. Figure 6 As shown, the entire graphics system goes through UI Thread, Render Thread, rendering enhancement device, GPU rendering, SurfaceFlinger, and DSS online synthesis and display. In the original Render Thread stage, a rendering enhancement device is added. The rendering enhancement device is used to convert window motion attribute parameters such as rotation, scaling, and rounded corner cutting into a second rendering command that can be recognized by the GPU. Render sends the first rendering command to the GPU, and the rendering enhancement device sends the second rendering command to the GPU. When the GPU receives the first command and the second command, the GPU performs layer rendering according to the first rendering command and the second rendering command to obtain image data. The GPU motion effect rendering in the prior art is moved forward and executed together with the original GPU rendering.

[0087] Figure 7 is a schematic diagram of an image processing pipeline provided in an embodiment of the present application, such as Figure 7As shown, when the first Vsync clock comes, UI and Render need to perform animation logic, measurement layout, drawing, synchronization, command submission and cache exchange, and the rendering enhancement device needs to perform synchronization (synchronization refers to the synchronization of UI resources to the rendering enhancement device, such as synchronizing the dynamic attribute parameters of the layer generated by the UI to the rendering enhancement device), rendering enhancement (the rendering enhancement device makes some judgments based on the rendering parameters and rendering strategies) and command submission (command submission refers to the rendering enhancement device converting the dynamic attribute parameters of the layer into commands that the GPU can recognize and sending them to the GPU); when the second Vsync clock comes, Surfaceflinger needs to perform message processing, synthesis preparation, enhanced layers, layer synthesis and refresh display. The image processing pipeline provided in the embodiment of the present application modifies the traditional rendering pipeline and adds a rendering enhancement device. The rendering enhancement device is used to convert window dynamic attribute parameters such as rotation, scaling, and rounded corner cutting into a second command that the GPU can recognize, so that the GPU dynamic effect rendering in the prior art is moved forward and executed together with the original GPU rendering, so that it is directly sent to the DSS for online synthesis through the enhanced layer, and finally displayed on the LCD.

[0088] Figure 8 is a schematic flow chart of an image processing method 300 provided in an embodiment of the present application. Figure 8 The interaction process between different modules is described in detail. Figure 8 As shown, the processing method 300 may include S301 to S311. Figure 7 Each step in method 300 is described in detail.

[0089] S301: The UI framework draws the current layer corresponding to the touch event.

[0090] S302, the rendering module Render is responsible for converting the controls and layout of the UI framework into a first rendering command.

[0091] S303, the window management is responsible for managing the size, position, rotation, scaling, corner radius, etc. of the current layer. A window is a layer. The window management sends the dynamic effect attribute parameters to the rendering enhancement window management

[0092] S304, the rendering enhancement window management is mainly responsible for forwarding the second rendering information such as rotation, scaling, and rounded corner cutting to the rendering enhancement engine strategy module, and notifying the surfaceflinger to bypass the native process and prepare to accept the rendering enhancement layer without the need for dynamic rendering.

[0093] S305, the rendering enhancement engine strategy is mainly responsible for receiving messages from the rendering enhancement window management and integrating strategies with the native rendering module.

[0094] Rendering enhancement engine strategies include:

[0095] 1. Process the management messages sent by the window such as rotation, scaling, and rounded corner cutting, and return other messages directly to surfaceFlinger, such as transparency and other parameters;

[0096] 2. When the first parameter included in the first rendering information is different from the first parameter included in the second rendering information, the first parameter included in the first rendering information is adjusted according to the first parameter included in the second rendering information. For example, whether the rounded corner size of the first rendering information conflicts with the rounded corner size of the second rendering information, if there is a conflict, the rounded corner size of the second rendering information shall prevail.

[0097] The current layer is rendered according to the rounded corner size of the second rendering information, thereby ensuring user experience.

[0098] S306, the rendering enhancement engine is responsible for converting the dynamic effect parameters such as rotation, scaling, and rounded corner cutting into a second rendering command that can be recognized by the GPU.

[0099] S307, the engine scheduling policy management is responsible for the management of the entire rendering enhancement engine thread scheduling priority. For example, the current operating system includes multiple processes, images, sounds, etc. The rendering enhancement engine thread scheduling priority refers to the difference between the image rendering priority and the priorities of other processes. The image rendering priority can be set high.

[0100] S308: The GPU receives the image rendering priority, the first rendering command, and the second rendering command to perform layer rendering.

[0101] S309, the rendering enhancement layer management receives a message from the rendering enhancement window management, such as receiving the first indication information. If the rendering enhancement layer management receives the first indication information, the native second rendering process does not need to be executed; if the first indication information is not received, the native second rendering process needs to be executed.

[0102] S310, Surfaceflinger obtains the image data rendered by the GPU from the cache and sends it to the DSS for online synthesis.

[0103] S311, LCD display.

[0104] It should be understood that Figure 8 The image processing module shown is only one embodiment. In a specific embodiment, it can be merged or split, such as the window management can be compatible with the content of the rendering enhancement window management.

[0105] It should also be understood that the above-mentioned window management, rendering enhancement window, rendering enhancement engine strategy, rendering enhancement engine, engine scheduling strategy, etc. jointly realize the functions of the rendering enhancement device described above.

[0106] Combination of the above Figures 1 to 8 The communication parameter measurement method of the multi-card terminal device in the embodiment of the present application is described in detail. Figures 9 to 11 The communication device according to the embodiment of the present application is described in detail.

[0107] Fig. 9 A schematic block diagram of an apparatus 400 according to an embodiment of the present application is shown.

[0108] In some embodiments, the apparatus 400 may be a terminal device, or may be a chip or a circuit, such as a chip or a circuit that may be disposed in a terminal device.

[0109] In one possible manner, the apparatus 400 may include a processing unit 410 (ie, an example of a processor) and a transceiver unit 430. In some possible implementations, the processing unit 410 may also be referred to as a determination unit. In some possible implementations, the transceiver unit 430 may include a receiving unit and a sending unit.

[0110] In one implementation, the transceiver unit 430 may be implemented by a transceiver or a transceiver-related circuit or an interface circuit.

[0111] In one implementation, the device may further include a storage unit 420. In one possible implementation, the storage unit 420 is used to store instructions. In one implementation, the storage unit may also be used to store data or information. The storage unit 420 may be implemented by a memory.

[0112] In some possible designs, the processing unit 410 is used to execute the instructions stored in the storage unit 420, so that the apparatus 400 implements the steps performed by the terminal device in the above method. Alternatively, the processing unit 410 can be used to call the data of the storage unit 420, so that the apparatus 400 implements the steps performed by the terminal device in the above method.

[0113] For example, the processing unit 410, the storage unit 420, and the transceiver unit 430 can communicate with each other through the internal connection path to transmit control and / or data signals. For example, the storage unit 420 is used to store a computer program, and the processing unit 410 can be used to call and run the computer program from the storage unit 420 to control the transceiver unit 430 to receive signals and / or send signals, and complete the steps of the terminal device or access network device in the above method. The storage unit 420 can be integrated in the processing unit 410, or it can be set separately from the processing unit 410.

[0114] Optionally, if the apparatus 400 is a communication device (eg, a terminal device), the transceiver unit 430 includes a receiver and a transmitter. The receiver and the transmitter may be the same or different physical entities. When they are the same physical entity, they may be collectively referred to as a transceiver.

[0115] When the device 400 is a terminal device, the transceiver unit 430 may be a sending unit or a transmitter when sending information, and the transceiver unit 430 may be a receiving unit or a receiver when receiving information. The transceiver unit may be a transceiver, and the transceiver, transmitter or receiver may be a radio frequency circuit. When the device includes a storage unit, the storage unit is used to store computer instructions, and the processor is connected to the memory for communication. The processor executes the computer instructions stored in the memory, so that the device can execute method 200 or method 300. The processor may be a general-purpose central processing unit (CPU), a microprocessor, or an application specific integrated circuit (ASIC).

[0116] Optionally, if the device 400 is a chip or a circuit, the transceiver unit 430 includes an input interface and an output interface.

[0117] When the device 400 is a chip, the transceiver unit 430 may be an input and / or output interface, a pin or a circuit, etc. The processing unit 410 may execute the computer-executable instructions stored in the storage unit so that the device may execute the method 200 or the method 300. Optionally, the storage unit is a storage unit in the chip, such as a register, a cache, etc. The storage unit may also be a storage unit located outside the chip in the terminal, such as a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM), etc.

[0118] As an implementation, the function of the transceiver unit 430 may be implemented by a transceiver circuit or a dedicated chip for transceiver. The processing unit 410 may be implemented by a dedicated processing chip, a processing circuit, a processing unit or a general chip.

[0119] As another implementation method, it is possible to use a general-purpose computer to implement the communication device (e.g., terminal device, or access network device) provided in the embodiment of the present application. That is, the program code for implementing the functions of the processing unit 410 and the transceiver unit 430 is stored in the storage unit 420, and the general-purpose processing unit implements the functions of the processing unit 410 and the transceiver unit 430 by executing the code in the storage unit 420.

[0120] In some embodiments, the device 400 may be an electronic device, and the processing unit 410 is used to generate a first rendering command based on first rendering information, wherein the first rendering information includes rendering information generated by controls and layout of the current layer; the processing unit 410 is also used to generate a second rendering command based on second rendering information, wherein the second rendering information includes dynamic rendering information of the current layer, wherein the generation of the first rendering command based on the first rendering information and the generation of the second rendering command based on the second rendering information are completed at the same stage; the transceiver unit 430 is used to send the first rendering command and the second rendering command to the graphics processor; the processing unit 410 is also used to set first indication information, wherein the first indication information is used to instruct the synthesizer to no longer render the current layer according to the second rendering information; the processing unit 410 is also used to perform layer synthesis based on image data of multiple layers to obtain a current image frame, wherein the image data of the multiple layers includes image data of the current layer, and the image data of the current layer is rendered by the graphics processor based on the first rendering command and the second rendering command.

[0121] In one implementation, before sending the first rendering command and the second rendering command to the graphics processor, the processing unit 410 is further used to: determine the priority level of the electronic device in the operating system according to the second indication information.

[0122] In one implementation, the motion effect rendering information includes at least one of the following parameters: rotation, scaling, and fillet radius.

[0123] In one implementation, the processing unit 410 is further configured to: determine the first rendering information and the second rendering information according to a motion rendering strategy.

[0124] In one implementation, the motion rendering strategy includes: when a first parameter included in the first rendering information is different from a first parameter included in the second rendering information, adjusting the first parameter included in the first rendering information according to the first parameter included in the second rendering information.

[0125] In one implementation, the processing unit 410 is also used to: generate a first rendering command according to first rendering information in response to a first vertical synchronization signal, the first rendering information including rendering information generated by controls and layout of the current layer; generate a second rendering command according to second rendering information, the second rendering information including dynamic rendering information of the current layer, wherein the generation of the first rendering command according to the first rendering information and the generation of the second rendering command according to the second rendering information are completed at the same stage; send the first rendering command and the second rendering command to the graphics processor; set first indication information, the first indication information is used to instruct the synthesizer to no longer render the current layer according to the second rendering information; in response to a second vertical synchronization signal, perform layer synthesis according to image data of multiple layers to obtain a current image frame, the image data of the multiple layers including image data of the current layer, the image data of the current layer is rendered by the graphics processor according to the first rendering command and the second rendering command.

[0126] When the device 400 is configured in or is itself an electronic device, each module or unit in the device 400 can be used to execute each action or processing process performed by the electronic device in the above method. Here, in order to avoid redundancy, its detailed description is omitted.

[0127] Fig.10 FIG. 5 is a schematic diagram of the structure of an electronic device 500 provided in an embodiment of the present application. Fig.10 As shown, the electronic device 500 may include a processor 510, an external memory interface 520, an internal memory 521, a universal serial bus (USB) interface 530, a charging management module 540, a power management module 541, a battery 542, an antenna 1, an antenna 2, a mobile communication module 550, a wireless communication module 560, an audio module 570, a speaker 570A, a receiver 570B, a microphone 170C, an earphone interface 570D, a sensor module 580, a button 590, a motor 591, an indicator 592, a camera 593, a display screen 594, and a subscriber identification module (SIM) card interface 595, etc.

[0128] Among them, the sensor module 580 may include a pressure sensor 580A, a gyroscope sensor 580B, an air pressure sensor 580C, a magnetic sensor 580D, an acceleration sensor 580E, a distance sensor 580F, a proximity light sensor 580G, a fingerprint sensor 580H, a temperature sensor 580J, a touch sensor 580K, an ambient light sensor 580L, and a bone conduction sensor 580M, etc.

[0129] It is to be understood that the structure illustrated in this embodiment does not constitute a specific limitation on the electronic device 500. In other embodiments, the electronic device 500 may include more or fewer components than shown in the figure, or combine some components, or separate some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0130] The processor 510 may include one or more processing units, for example, the processor 510 may include an application processor (application processor, AP), a modem processor, a GPU, an image signal processor (image signal processor, ISP), a controller, a memory, a video codec, a digital signal processor (digital signal processor, DSP), a baseband processor, and / or a neural network processor (neural-network processing unit, NPU), etc. Among them, different processing units may be independent devices or integrated in one or more processors.

[0131] The controller may be the nerve center and command center of the electronic device 500. The controller may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.

[0132] The processor 510 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 510 is a cache memory. The memory may store instructions or data that the processor 510 has just used or cyclically used. If the processor 510 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 510, and thus improves the efficiency of the system.

[0133] In some embodiments, the processor 510 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuits sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0134] It is understandable that the interface connection relationship between the modules shown in this embodiment is only a schematic illustration and does not constitute a structural limitation on the electronic device 500. In other embodiments, the electronic device 500 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0135] The charging management module 540 is used to receive charging input from a charger. While the charging management module 540 is charging the battery 542 , it can also power the electronic device through the power management module 541 .

[0136] The power management module 541 is used to connect the battery 542, the charging management module 540 and the processor 510. The power management module 541 receives input from the battery 542 and / or the charging management module 540, and supplies power to the processor 510, the internal memory 521, the external memory, the display screen 594, the camera 593, and the wireless communication module 560. In some other embodiments, the power management module 541 may also be disposed in the processor 510. In other embodiments, the power management module 541 and the charging management module 540 may also be disposed in the same device.

[0137] The wireless communication function of the electronic device 500 can be implemented through the antenna 1, the antenna 2, the mobile communication module 550, the wireless communication module 560, the modem processor and the baseband processor.

[0138] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 500 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network.

[0139] The mobile communication module 550 can provide solutions for wireless communications including 2G / 3G / 4G / 5G, etc., applied to the electronic device 500. The mobile communication module 550 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 550 can receive electromagnetic waves from the antenna 1, and filter, amplify, etc. the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 550 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1.

[0140] The modem processor may include a modulator and a demodulator. The modulator is used to modulate the low-frequency baseband signal to be sent into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After the low-frequency baseband signal is processed by the baseband processor, it is passed to the application processor. The application processor outputs a sound signal through an audio device (not limited to a speaker 570A, a receiver 570B, etc.), or displays an image or video through a display screen 594.

[0141] The wireless communication module 560 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) network), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the electronic device 500. The wireless communication module 560 can be one or more devices integrating at least one communication processing module. The wireless communication module 560 receives electromagnetic waves via the antenna 2, modulates the frequency of the electromagnetic wave signal and filters it, and sends the processed signal to the processor 510. The wireless communication module 560 can also receive the signal to be sent from the processor 510, modulate the frequency of it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0142] In some embodiments, the antenna 1 of the electronic device 500 is coupled to the mobile communication module 550, and the antenna 2 is coupled to the wireless communication module 560, so that the electronic device 500 can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology. The GNSS may include a global positioning system (GPS), a global navigation satellite system (GLONASS), a Beidou navigation satellite system (BDS), a quasi-zenith satellite system (QZSS) and / or a satellite-based augmentation system (SBAS).

[0143] The electronic device 500 implements the display function through a GPU, a display screen 594, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 594 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 510 may include one or more GPUs, which execute program instructions to generate or change display information.

[0144] The display screen 594 is used to display images, videos, etc. The display screen 594 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, a quantum dot light-emitting diode (QLED), etc.

[0145] Among them, the display screen 594 in the embodiment of the present application can be a touch screen. That is, a touch sensor 580K is integrated in the display screen 594. The touch sensor 580K can also be called a "touch panel". In other words, the display screen 594 may include a display panel and a touch panel, and the touch sensor 580K and the display screen 594 form a touch screen, also known as a "touch screen". The touch sensor 580K is used to detect touch operations acting on or near it. After the touch operation detected by the touch sensor 580K, it can be passed to the upper layer by the driver of the kernel layer (such as TP driver) to determine the type of touch event. Visual output related to the touch operation can be provided by the display screen 594. In other embodiments, the touch sensor 580K can also be arranged on the surface of the electronic device 500, which is different from the position where the display screen 594 is located.

[0146] The electronic device 500 can realize the shooting function through an ISP, a camera 593, a video codec, a GPU, a display screen 594, and an application processor. The ISP is used to process the data fed back by the camera 593. The camera 593 is used to capture static images or videos. The digital signal processor is used to process digital signals. In addition to processing digital image signals, it can also process other digital signals. The video codec is used to compress or decompress digital videos. The electronic device 500 can support one or more video codecs. In this way, the electronic device 500 can play or record videos in multiple encoding formats, such as: Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0147] NPU is a neural network (NN) computing processor, which can quickly process input information and continuously self-learn by drawing on the structure of biological neural networks, such as the transmission mode between neurons in the human brain. NPU can realize applications such as intelligent cognition of electronic device 500, such as image recognition, face recognition, voice recognition, text understanding, etc.

[0148] The external memory interface 550 can be used to connect an external memory card, such as a MicroSD card, to expand the storage capacity of the electronic device 500. The external memory card communicates with the processor 510 through the external memory interface 520 to implement a data storage function. For example, files such as music and videos are stored in an external memory card. The internal memory 521 can be used to store computer executable program codes, and the executable program codes include instructions. The processor 510 can execute various functional applications and data processing of the electronic device 500 by running the instructions stored in the internal memory 521. For example, in an embodiment of the present application, the processor 510 can execute instructions stored in the internal memory 521, and the internal memory 521 may include a program storage area and a data storage area. Among them, the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area can store data created during the use of the electronic device 500 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 521 may include a high-speed random access memory and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0149] The electronic device 500 can implement audio functions such as music playing and recording through the audio module 570, the speaker 570A, the receiver 570B, the microphone 170C, the headphone jack 570D, and the application processor.

[0150] The audio module 570 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 570 can also be used to encode and decode audio signals. The speaker 570A, also known as the "speaker", is used to convert audio electrical signals into sound signals. The receiver 570B, also known as the "earpiece", is used to convert audio electrical signals into sound signals. The microphone 170C, also known as the "microphone" or "microphone", is used to convert sound signals into electrical signals. The headphone jack 570D is used to connect wired headphones.

[0151] The pressure sensor 580A is used to sense the pressure signal and can convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor 580A can be disposed on the display screen 594. There are many types of pressure sensors 580A, such as resistive pressure sensors, inductive pressure sensors, capacitive pressure sensors, etc.

[0152] A parallel plate with a conductive material. When a force acts on the pressure sensor 580A, the capacitance between the electrodes changes. The electronic device 500 determines the intensity of the pressure based on the change in capacitance. When a touch operation acts on the display screen 594, the electronic device 500 detects the intensity of the touch operation based on the pressure sensor 580A. The electronic device 500 can also calculate the position of the touch based on the detection signal of the pressure sensor 580A. In some embodiments, touch operations acting on the same touch position but with different touch operation intensities can correspond to different operation instructions. In an embodiment of the present application, the electronic device 500 can obtain the pressing force of the user's touch operation through the pressure sensor 580A.

[0153] The button 590 includes a power button, a volume button, etc. The button 590 can be a mechanical button. It can also be a touch button. The electronic device 500 can receive the button input and generate a key signal input related to the user settings and function control of the electronic device 500. The motor 591 can generate a vibration prompt. The motor 591 can be used for incoming call vibration prompts, and can also be used for touch vibration feedback. The indicator 592 can be an indicator light, which can be used to indicate the charging status, power changes, messages, missed calls, notifications, etc. The SIM card interface 595 is used to connect the SIM card. The SIM card can be inserted into the SIM card interface 595, or pulled out from the SIM card interface 595 to achieve contact and separation with the electronic device 500. The electronic device 500 can support 1 or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 595 can support NanoSIM cards, MicroSIM cards, SIM cards, etc.

[0154] The methods in the following embodiments can all be implemented in the electronic device 500 having the above hardware structure.

[0155] Fig.11 This is a schematic diagram of the structure of a terminal device 600 provided in the present application. The terminal device 600 can execute the actions executed by the electronic device in the above method embodiment.

[0156] For ease of explanation, Fig.11 Only the main components of the terminal device are shown. Fig.11 As shown, the terminal device 600 includes a processor, a memory, a control circuit, an antenna, and an input and output device.

[0157] The processor is mainly used to process the communication protocol and communication data, as well as to control the entire terminal device, execute the software program, and process the data of the software program, for example, to support the terminal device to perform the actions described in the above-mentioned transmission precoding matrix indication method embodiment. The memory is mainly used to store software programs and data, for example, to store the codebook described in the above-mentioned embodiment. The control circuit is mainly used for conversion between baseband signals and radio frequency signals and processing of radio frequency signals. The control circuit and the antenna can also be called a transceiver, which is mainly used to send and receive radio frequency signals in the form of electromagnetic waves. Input and output devices, such as touch screens, display screens, keyboards, etc. are mainly used to receive data input by users and output data to users.

[0158] When the terminal device is turned on, the processor can read the software program in the storage unit, interpret and execute the instructions of the software program, and process the data of the software program. When data needs to be sent wirelessly, the processor performs baseband processing on the data to be sent, and outputs the baseband signal to the RF circuit. The RF circuit performs RF processing on the baseband signal and then sends the RF signal outward in the form of electromagnetic waves through the antenna. When data is sent to the terminal device, the RF circuit receives the RF signal through the antenna, converts the RF signal into a baseband signal, and outputs the baseband signal to the processor. The processor converts the baseband signal into data and processes the data.

[0159] Those skilled in the art will appreciate that for ease of description, Fig.11 Only one memory and processor are shown. In an actual terminal device, there may be multiple processors and memories. The memory may also be referred to as a storage medium or a storage device, etc., which is not limited in the embodiments of the present application.

[0160] For example, the processor may include a baseband processor and a central processing unit. The baseband processor is mainly used to process the communication protocol and communication data, and the central processing unit is mainly used to control the entire terminal device, execute software programs, and process data of the software programs. Fig.10 The processor in integrates the functions of the baseband processor and the central processing unit. Those skilled in the art will understand that the baseband processor and the central processing unit may also be independent processors, which are interconnected through technologies such as buses. Those skilled in the art will understand that the terminal device may include multiple baseband processors to adapt to different network standards, and the terminal device may include multiple central processing units to enhance its processing capabilities. The various components of the terminal device may be connected through various buses. The baseband processor may also be described as a baseband processing circuit or a baseband processing chip. The central processing unit may also be described as a central processing circuit or a central processing chip. The function of processing the communication protocol and communication data may be built into the processor, or may be stored in the storage unit in the form of a software program, and the processor executes the software program to implement the baseband processing function.

[0161] For example, in the embodiment of the present application, the antenna and the control circuit having the transceiver function may be regarded as the transceiver unit 610 of the terminal device 600, and the processor having the processing function may be regarded as the processing unit 650 of the terminal device 600. Fig.10 As shown, the terminal device 600 includes a transceiver unit 610 and a processing unit 650. The transceiver unit may also be referred to as a transceiver, a transceiver, a transceiver device, etc. Optionally, the device used to implement the receiving function in the transceiver unit 610 may be regarded as a receiving unit, and the device used to implement the sending function in the transceiver unit 610 may be regarded as a sending unit, that is, the transceiver unit includes a receiving unit and a sending unit. Exemplarily, the receiving unit may also be referred to as a receiver, a receiver, a receiving circuit, etc., and the sending unit may be referred to as a transmitter, a transmitter, or a transmitting circuit, etc.

[0162] It should be understood that in the embodiments of the present application, the processor may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0163] It should also be understood that the memory in the embodiments of the present application may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM).

[0164] The above embodiments can be implemented in whole or in part by software, hardware, firmware or any other combination. When implemented by software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state hard disk.

[0165] An embodiment of the present application further provides a computer-readable medium on which a computer program is stored. When the computer program is executed by a computer, the steps performed by the electronic device in any of the above embodiments are implemented.

[0166] An embodiment of the present application also provides a computer program product, which, when executed by a computer, implements the steps performed by the electronic device in any of the above embodiments.

[0167] The embodiment of the present application also provides a system chip, which includes: a communication unit and a processing unit. The processing unit may be, for example, a processor. The communication unit may be, for example, a communication interface, an input / output interface, a pin or a circuit. The processing unit may execute computer instructions to enable the chip in the communication device to perform the steps performed by the electronic device provided in the embodiment of the present application.

[0168] Optionally, the computer instructions are stored in a storage unit.

[0169] The various embodiments in this application may be used independently or in combination, which is not limited here.

[0170] In addition, various aspects or features of the present application can be implemented as methods, devices or products using standard programming and / or engineering techniques. The term "product" used in this application covers computer programs that can be accessed from any computer-readable device, carrier or medium. For example, computer-readable media may include, but are not limited to: magnetic storage devices (e.g., hard disks, floppy disks or tapes, etc.), optical disks (e.g., compact discs (CDs), digital versatile discs (DVDs), etc.), smart cards and flash memory devices (e.g., erasable programmable read-only memory (EPROM), cards, sticks or key drives, etc.). In addition, the various storage media described herein may represent one or more devices and / or other machine-readable media for storing information. The term "machine-readable medium" may include, but is not limited to, wireless channels and various other media capable of storing, containing and / or carrying instructions and / or data.

[0171] It should be understood that "and / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent three situations: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the previously associated objects are in an "or" relationship. "At least one" means one or more; "at least one of A and B" is similar to "A and / or B", describing the association relationship of associated objects, indicating that three relationships may exist. For example, at least one of A and B can represent three situations: A exists alone, A and B exist at the same time, and B exists alone.

[0172] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0173] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0174] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0175] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0176] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0177] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0178] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for image processing, characterized in that: include: Generate a first rendering command according to first rendering information, wherein the first rendering information includes rendering information generated by controls and layout of the current layer; Generate a second rendering command according to second rendering information, wherein the second rendering information includes dynamic effect rendering information of the current layer, Wherein, generating the first rendering command according to the first rendering information and generating the second rendering command according to the second rendering information are completed at the same stage; Sending the first rendering command and the second rendering command to a graphics processor; Setting first indication information, where the first indication information is used to instruct the synthesizer to no longer render the current layer according to the second rendering information; Layer synthesis is performed based on image data of multiple layers to obtain a current image frame, wherein the image data of the multiple layers include image data of the current layer, and the image data of the current layer is rendered by the graphics processor based on the first rendering command and the second rendering command.

2. The method according to claim 1, characterized in that Before sending the first rendering command and the second rendering command to the graphics processor, the method further includes: According to the second indication information, a priority level of the method in the operating system is determined.

3. The method according to claim 1 or 2, characterized in that: The dynamic effect rendering information includes at least one of the following parameters: Rotation, scale, and corner radius.

4. The method according to claim 3, characterized in that The method further comprises: The first rendering information and the second rendering information are determined according to a motion effect rendering strategy.

5. The method according to claim 4, characterized in that The dynamic rendering strategy includes: When the first parameter included in the first rendering information is different from the first parameter included in the second rendering information, the first parameter included in the first rendering information is adjusted according to the first parameter included in the second rendering information.

6. The method according to any one of claims 1 to 5, characterized in that The method further comprises: In response to a first vertical synchronization signal, generating a first rendering command according to first rendering information, wherein the first rendering information includes rendering information generated by controls and layout of a current layer; Generate a second rendering command according to second rendering information, wherein the second rendering information includes dynamic effect rendering information of the current layer, Wherein, generating the first rendering command according to the first rendering information and generating the second rendering command according to the second rendering information are completed at the same stage; Sending the first rendering command and the second rendering command to a graphics processor; Setting first indication information, where the first indication information is used to instruct the synthesizer to no longer render the current layer according to the second rendering information; In response to a second vertical synchronization signal, layer synthesis is performed based on image data of multiple layers to obtain a current image frame, wherein the image data of the multiple layers include image data of the current layer, and the image data of the current layer is rendered by the graphics processor based on the first rendering command and the second rendering command.

7. An electronic device, characterized in that: include: A processing unit, configured to generate a first rendering command according to first rendering information, wherein the first rendering information includes rendering information generated by controls and layout of a current layer; The processing unit is further configured to generate a second rendering command according to second rendering information, wherein the second rendering information includes dynamic effect rendering information of the current layer. Wherein, generating the first rendering command according to the first rendering information and generating the second rendering command according to the second rendering information are completed at the same stage; A transceiver unit, configured to send the first rendering command and the second rendering command to a graphics processor; The processing unit is further used to set first indication information, where the first indication information is used to instruct the synthesizer to no longer render the current layer according to the second rendering information; The processing unit is also used to perform layer synthesis based on image data of multiple layers to obtain a current image frame, wherein the image data of the multiple layers include image data of the current layer, and the image data of the current layer is rendered by the graphics processor according to the first rendering command and the second rendering command.

8. The electronic device according to claim 7, characterized in that: Before sending the first rendering command and the second rendering command to the graphics processor, the processing unit is further used for: According to the second indication information, the priority level of the electronic device in the operating system is determined.

9. The electronic device according to claim 7 or 8, characterized in that: The dynamic effect rendering information includes at least one of the following parameters: Rotation, scale, and corner radius.

10. The electronic device according to claim 9, characterized in that: The processing unit is also used for: The first rendering information and the second rendering information are determined according to a motion effect rendering strategy.

11. The electronic device according to claim 10, characterized in that: The dynamic rendering strategy includes: When the first parameter included in the first rendering information is different from the first parameter included in the second rendering information, the first parameter included in the first rendering information is adjusted according to the first parameter included in the second rendering information.

12. The electronic device according to any one of claims 7 to 11, characterized in that: The processing unit is also used for: In response to a first vertical synchronization signal, generating a first rendering command according to first rendering information, wherein the first rendering information includes rendering information generated by controls and layout of a current layer; Generate a second rendering command according to second rendering information, wherein the second rendering information includes dynamic effect rendering information of the current layer, Wherein, generating the first rendering command according to the first rendering information and generating the second rendering command according to the second rendering information are completed at the same stage; Sending the first rendering command and the second rendering command to a graphics processor; Setting first indication information, where the first indication information is used to instruct the synthesizer to no longer render the current layer according to the second rendering information; In response to a second vertical synchronization signal, layer synthesis is performed based on image data of multiple layers to obtain a current image frame, wherein the image data of the multiple layers include image data of the current layer, and the image data of the current layer is rendered by the graphics processor based on the first rendering command and the second rendering command.

13. An electronic device, comprising a processor, wherein the processor is connected to a memory, the memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory, so that the electronic device executes the method as claimed in any one of claims 1 to 6.

14. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed, the method according to any one of claims 1 to 6 is implemented.

15. A chip, characterized in that: Including processor and interface; The processor is configured to read instructions to execute the data transmission method according to any one of claims 1 to 6.

Images