Picture rendering method and device

Abstract

The invention discloses a picture rendering method and device, and belongs to the technical field of electronic equipment.The method comprises the steps that a forward vector of a first image frame in a display picture of a first application is obtained, and the forward vector represents the direction of the first image frame on a world coordinate z axis; according to the forward vector of the first image frame, the forward vector of a previous image frame of the first image frame and the interval duration between the two frames, the rotation angular velocity of the first image frame relative to the previous image frame is determined, and the rotation angular velocity is used for representing the rotation degree of the display picture; and when the rotation angular velocity is smaller than an angular velocity threshold value, frame insertion processing is performed on a display picture of the first application, and picture rendering is performed on an image frame of the first application after frame insertion processing.

Description

Technical Field

[0001] The present application belongs to the technical field of electronic equipment, and specifically relates to a picture rendering method and device thereof. Background Art

[0002] Interpolation technology refers to a technical method that improves the smoothness and stability of the picture by adding additional intermediate image frames during the rendering process of video or game pictures. At present, the algorithm of interpolation technology is usually integrated in the picture rendering process. However, during the picture rendering process, when faced with a scene where the picture rotates rapidly, the pixels of the next image frame of the picture do not appear in the previous image frame due to the rapid movement of the pixels of the picture. The difference between the two frames of the previous and next images is large, and the intermediate image frame cannot be accurately generated. If the interpolation technology is still used, the picture will have problems such as ghosting and fragmentation, resulting in poor picture quality. Therefore, when to use interpolation technology in the picture rendering process to avoid problems such as ghosting and fragmentation has become a technical problem that needs to be solved urgently by those skilled in the art.

[0003] In the related technology, for each mobile application involving interpolation technology, the application developer is required to provide the screen scene information of the application. If the application developer detects that the current screen scene is not a fast-rotating scene, the interpolation technology is used. This processing method requires the application developer to deeply participate in the entire research and development process and provide the application's screen scene information to the device operating system platform in the form of a software tool development kit. On the one hand, the application developer needs to invest a lot of manpower and financial resources in secondary development and integrate the software tool development kit for each application, resulting in a relatively high application development cost. On the other hand, due to the close coupling with the application, the device operating system platform cannot complete the development of this solution alone, resulting in low research and development efficiency of the device operating system platform. Summary of the invention

[0004] The purpose of the embodiments of the present application is to provide a screen rendering method and device thereof, which can reduce the R&D cost of applications and improve the R&D efficiency of the device operating system platform.

[0005] In a first aspect, an embodiment of the present application provides a screen rendering method, the method comprising:

[0006] Acquire a forward vector of a first image frame in a display screen of a first application, wherein the forward vector represents a direction of the first image frame on a world coordinate z-axis;

[0007] Determining a rotational angular velocity of the first image frame compared with the previous image frame according to a forward vector of the first image frame, a forward vector of an image frame previous to the first image frame, and an interval between the two frames, wherein the rotational angular velocity is used to characterize a degree of rotation of the display screen;

[0008] When the rotation angular velocity is less than an angular velocity threshold, interpolation processing is performed on the display screen of the first application, and screen rendering is performed on the image frame of the first application after the interpolation processing.

[0009] In a second aspect, an embodiment of the present application provides a screen rendering device, the device comprising:

[0010] An acquisition module, configured to acquire a forward vector of a first image frame in a display screen of a first application, wherein the forward vector indicates a direction of the first image frame on a world coordinate z-axis;

[0011] a determination module, configured to determine a rotation angular velocity of the first image frame compared with the previous image frame according to a forward vector of the first image frame, a forward vector of an image frame previous to the first image frame, and an interval between the two frames, wherein the rotation angular velocity is used to characterize a degree of rotation of the display screen;

[0012] a processing module, configured to perform frame insertion processing on a display picture of the first application when the rotation angular velocity is less than an angular velocity threshold;

[0013] A rendering module is used to render the image frames of the first application after the interpolation processing.

[0014] In a third aspect, an embodiment of the present application provides an electronic device, comprising a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the steps of the screen rendering method described in the first aspect.

[0015] In a fourth aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the steps of the screen rendering method described in the first aspect are implemented.

[0016] In a fifth aspect, an embodiment of the present application provides a chip, comprising a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run a program or instruction to implement the steps of the screen rendering method described in the first aspect.

[0017] In an embodiment of the present application, during the real-time rendering process of the display screen of the first application, the forward vector of the first image frame in the display screen is obtained, wherein the forward vector indicates the direction of the first image frame on the z-axis of the world coordinate system; according to the forward vector of the first image frame, the forward vector of the previous image frame of the first image frame, and the interval between the two frames, the rotational angular velocity of the first image frame compared with the previous image frame is determined. Since the rotational angular velocity is used to characterize the degree of rotation of the display screen, when the rotational angular velocity is less than the angular velocity threshold, it indicates that the scene of the screen is not a fast rotating scene. The display screen of the first application is interpolated by the interpolation technology, and the image frame of the first application after the interpolation is rendered, which can avoid the problems of ghosting and fragmentation of the screen and improve the picture quality. The above processes are all completed independently by the electronic device operating system platform, without the need for deep cooperation from the application developer, the need for secondary development by the application developer, and the need for a software tool development kit for integrated applications. It can be dynamically effective when the application is running, reducing the R&D cost of the application and improving the R&D efficiency of the device operating system platform. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 is an example diagram of a camera provided in an embodiment of the present application in a world coordinate system;

[0019] Figure 2 It is an architectural diagram of a screen rendering method in the related art;

[0020] Figure 3 This is one of the flow charts of a screen rendering method provided in an embodiment of the present application;

[0021] Figure 4 It is an architecture diagram of the screen rendering method provided in the embodiment of the present application;

[0022] Figure 5 This is the second flowchart of a screen rendering method provided in an embodiment of the present application;

[0023] Figure 6 It is a structural block diagram of a picture rendering device provided in an embodiment of the present application;

[0024] Figure 7 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application;

[0025] Figure 8 It is a schematic diagram of the hardware structure of an electronic device implementing an embodiment of the present application. DETAILED DESCRIPTION

[0026] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.

[0027] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the terms used in this way are interchangeable where appropriate, so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0028] To facilitate understanding, the concepts and application scenarios involved in the embodiments of the present application are first introduced.

[0029] 1. Concept

[0030] Image rendering refers to the technology of generating two-dimensional images or videos from models in three-dimensional scenes based on the set environment, lighting, materials and rendering parameters.

[0031] The rotating scene of the picture refers to the scene or elements in the picture that can be rotated or have the characteristic of rotation. This design can increase the interactivity of the picture and the user's immersion.

[0032] The Unity engine is a cross-platform 2D and 3D development engine that dominates the mobile game market.

[0033] Interpolation technology is a technology that uses an independent display chip or post-processing technology to share the rendering of half of the screen for the graphics processor. The specific process is to calculate the position information of the pixel points of the latter image frame in the previous image frame based on the previous and next image frames, obtain the motion trajectory of the pixel points based on the position information, generate an intermediate image frame at an intermediate point of the motion trajectory, and insert the intermediate image frame between the previous and next image frames to increase the frame rate of the picture, thereby improving the smoothness and viewing experience of the picture.

[0034] The graphics application programming interface (OpenGL ES / EGL API) is a widely used graphics programming interface that provides many powerful APIs for implementing various graphics rendering and graphics processing tasks.

[0035] The three directions of the camera in the world coordinate system, such as Figure 1 As shown, the x-axis of the world coordinate system represents the left direction of the camera, the y-axis of the world coordinate system represents the upward direction of the camera, and the z-axis of the world coordinate system represents the direction in which the camera is facing.

[0036] The glGetUniformLocation function is a function in OpenGL that is used to obtain the position number of a uniform variable in a shader program. The function of this function is to return the position number of a uniform variable. Since the position should be a vector vec3 rather than an integer value, it is just a position number. Through this function, the value of the uniform variable can be set. If there is no such variable, -1 is returned. The use of this function can help developers better manage and operate uniform variables in shader programs, thereby improving the effect and efficiency of graphics rendering.

[0037] The glGetUniformfv function is a function in OpenGL ES that is used to obtain the value of a uniform variable in a shader program. This function belongs to the C language specification of OpenGL ES, which allows developers to query the floating-point value of a specific uniform variable in a shader program.

[0038] The glDrawElements function is a function in OpenGL that is used to draw three-dimensional graphics. It uses an index array to specify the order of vertices and normals to achieve the drawing of three-dimensional models.

[0039] 2. Application Scenarios

[0040] Interpolation technology refers to a technical method that improves the smoothness and stability of the picture by adding additional intermediate image frames during the rendering process of video or game pictures. At present, the algorithm of interpolation technology is usually integrated in the picture rendering process. However, during the picture rendering process, when faced with a scene where the picture rotates rapidly, the pixels of the next image frame of the picture do not appear in the previous image frame because the pixels of the picture move too fast. The difference between the two frames is large, and the intermediate image frame cannot be accurately generated. At this time, if the interpolation technology is still used, the picture will be accompanied by ghosting, fragmentation and other problems. Therefore, when to use interpolation technology in the picture rendering process to avoid ghosting, fragmentation and other problems has become a technical problem that needs to be solved urgently by those skilled in the art.

[0041] In the related art, for each mobile application involving interpolation technology, the application developer is required to provide the screen scene information of the application. If the application developer detects that the current screen scene is not a fast-rotating scene, the interpolation technology is used, otherwise the interpolation technology is disabled. Figure 2As shown, the application developer provides mobile applications, graphics application programming interfaces and server-side software tool development kits in the code space, and the device operating system platform provides interpolation modules, system native window renderers and client-side software tool development kits in the system code space. Among them, the server-side software tool development kit and the client-side software tool development kit contain the application's screen scene information, which requires the joint participation of the application developer and the device operating system platform. The interpolation module is used to provide an interpolation algorithm, and the system native window renderer is used to provide rendering functions.

[0042] It can be seen that the above processing method requires the application developer to deeply participate in the entire research and development process, and provide the application's screen scene information to the device operating system platform in the form of a software tool development kit. On the one hand, the application developer needs to invest a lot of manpower and financial resources in secondary development, and integrate the software tool development kit for each application, resulting in a relatively high application development cost. On the other hand, it is tightly coupled with the application, and the device operating system platform cannot complete the development of this solution alone, resulting in low research and development efficiency of the device operating system platform. In order to solve the above technical problems, the embodiments of the present application provide a screen rendering method and a device thereof.

[0043] Next, in conjunction with the accompanying drawings, the screen rendering method provided by the embodiment of the present application is described in detail through specific embodiments and their application scenarios.

[0044] It should be noted that the screen rendering method provided in the embodiment of the present application is applicable to electronic devices. In practical applications, the electronic devices may include: mobile terminals such as smart phones, tablet computers, personal digital assistants, etc., and the embodiment of the present application does not limit this.

[0045] Figure 3 This is one of the flowcharts of a screen rendering method provided in an embodiment of the present application, such as Figure 3 As shown, the method may include the following steps: S301, S302, S303 and S304;

[0046] In S301, a forward vector of a first image frame in a display screen of a first application is obtained, wherein the forward vector represents a direction of the first image frame on a world coordinate z-axis.

[0047] In the embodiment of the present application, the first application may be any application developed by a third party, for example, a video application, a game application, or other application involving interpolation technology.

[0048] In the embodiment of the present application, the first image frame generally refers to the current image frame of the display screen of the first application.

[0049] For the sake of easy distinction, the first image frame and its preceding and following image frames are referred to as “original image frames”; the intermediate image frames generated by the interpolation technology are referred to as “interpolated image frames”.

[0050] In the embodiment of the present application, during the real-time rendering process of the display screen of the first application, a forward vector of the first image frame in the display screen is obtained.

[0051] In the embodiment of the present application, the forward vector refers to the camera orientation vector, which is one of the image drawing parameters. The forward vector can be used to calculate the rotation angular velocity of the first image frame relative to the image content of the previous image frame.

[0052] Related technologies Figure 2 In contrast, in the present application embodiment, Figure 4 As shown, the application developer only needs to provide mobile application and graphics application programming interfaces in the code space, and the device operating system platform provides a scene recognition module, an interpolation module and a system native window renderer in the system code space, wherein the scene recognition module is a functional module that can execute steps S301 to S302, the interpolation module is a functional module that can execute step S303, and the system native window renderer is a functional module that can execute step S304. The scene recognition module belongs to the system code space and does not need to be compiled with the application, and takes effect automatically during the operation of the application.

[0053] In some embodiments, the above step S301 may include the following steps: S3011;

[0054] In S3011, a forward vector of a first image frame in a display screen of the first application is obtained from a calling parameter of a graphics application programming interface of the first application.

[0055] In an embodiment of the present application, considering that in the real-time rendering process of the display screen of the first application, it is necessary to call the graphics application programming interface of the first application, and draw and render each image frame of the display screen through the graphics application programming interface, and the calling parameters of the graphics application programming interface include the screen drawing parameters of the first image frame (i.e., the current image frame) of the display screen, therefore, in an embodiment of the present application, before the first application is about to call the graphics application programming interface to render the first image frame, the scene recognition module intercepts the call of the graphics application programming interface in the system, and obtains the screen drawing parameters of the first image frame, i.e., the forward vector of the first image frame, from the calling parameters of the graphics application programming interface.

[0056] It can be seen that in the embodiment of the present application, since the calling parameters of the graphics application programming interface of the first application include the screen drawing parameters of the first image frame of the display screen, the forward vector of the first image frame in the display screen of the first application can be accurately obtained from the calling parameters of the graphics application programming interface of the first application.

[0057] In some embodiments, when the first application is an application written based on the first development engine, the above step S3011 may include the following steps: S30111, S30112 and S30113;

[0058] In S30111, through the first function, the position value of the first built-in variable of the first development engine is found from the calling parameters of the graphical application programming interface of the first application.

[0059] In the embodiment of the present application, the first development engine can be any current application development engine.

[0060] In the embodiment of the present application, the first function is a function in the operating system running on the electronic device. When the operating system running on the electronic device is an Android system, the first function may be a glGetUniformLocation function.

[0061] In the embodiment of the present application, the first built-in variable stores the forward vector of the first image frame. In practical applications, different development engines may have different names for the first built-in variables.

[0062] In the embodiment of the present application, the position value of the first built-in variable of the first development engine can be found from the calling parameters of the graphical application programming interface according to the names of the first function and the first built-in variable.

[0063] In S30112, when the position value is greater than zero, the variable value of the first built-in variable is obtained from the calling parameters of the graphical application programming interface through the second function.

[0064] In the embodiment of the present application, the second function is a function in the operating system running on the electronic device. When the operating system running on the electronic device is an Android system, the second function may be a glGetUniformfv function.

[0065] In an embodiment of the present application, if the position value of the first built-in variable is greater than zero, it means that the first built-in variable exists in the calling parameters of the graphical application programming interface, and the variable value of the first built-in variable can be obtained from the calling parameters of the graphical application programming interface based on the second function and the position value of the first built-in variable.

[0066] In S30113, a forward vector of a first image frame in a display screen of a first application is obtained from a variable value of a first built-in variable.

[0067] In an embodiment of the present application, the variable value of the first built-in variable usually contains more information, and the forward vector of the first image frame usually exists in a specified field in the variable value of the first built-in variable, so the forward vector of the first image frame is obtained from the specified field of the variable value of the first built-in variable.

[0068] It can be seen that in the embodiments of the present application, for an application developed by any application development engine, the existing functions in the operating system running the electronic device can be used to obtain the forward vector of the first image frame from the calling parameters of the graphics application programming interface, which has high versatility and a wide range of applicable scenarios.

[0069] In the embodiment of the present application, after the forward vector of the first image frame is obtained, the forward vector of the first image frame may be stored in a database so that the forward vector of the first image frame may be referenced when rendering the next image frame.

[0070] In S302, the rotational angular velocity of the first image frame compared to the previous image frame is determined according to the forward vector of the first image frame, the forward vector of the previous image frame of the first image frame and the interval between the two frames, wherein the rotational angular velocity is used to characterize the degree of rotation of the display screen.

[0071] In the embodiment of the present application, the greater the rotation angular velocity, the higher the degree of rotation of the displayed image, and the higher the probability that the image scene is a rapidly rotating scene; the smaller the rotation angular velocity, the lower the degree of rotation of the displayed image, and the lower the probability that the image scene is a rapidly rotating scene.

[0072] In an embodiment of the present application, the forward vector of the previous image frame of the first image frame can be stored in a database, and the method for obtaining the forward vector of the previous image frame is similar to the method for obtaining the forward vector of the first image frame. Specifically, before the first application is about to call the graphics application programming interface to render the previous image frame, the scene recognition module intercepts the call of the graphics application programming interface in the system and obtains the screen drawing parameters of the previous image frame, that is, the forward vector of the previous image frame, from the calling parameters of the graphics application programming interface.

[0073] Exemplarily, the forward vector of the first image frame is recorded as L1, the forward vector of the image frame before the first image frame is recorded as L2, and the interval between the two frames is recorded as t.

[0074] First, the angle α between the forward vector of the first image frame, denoted as L1, and the forward vector of the previous image frame, denoted as L2, is calculated, where α = {acos[dot(L1,L2)] / (|L1|*|L2|)} / π*180, and the unit of α is radians. Then, the rotation angular velocity ω is calculated according to the angle α and the interval time t, where ω = α / t.

[0075] In S303, when the rotation angular velocity is less than the angular velocity threshold, interpolation processing is performed on the display screen of the first application.

[0076] In an embodiment of the present application, when the rotation angular velocity is less than the angular velocity threshold, it indicates that the scene of the display screen of the first application is not a fast rotation scene. At this time, the interpolation technology can generate accurate intermediate image frames, instructing the interpolation module to perform interpolation processing on the display screen of the first application, and there will be no problems of ghosting or fragmentation of the screen.

[0077] In the embodiment of the present application, when the rotation angular velocity is greater than or equal to the angular velocity threshold, it indicates that the scene of the display screen of the first application is a fast rotation scene. At this time, the interpolation technology cannot generate accurate intermediate image frames, and it is not suitable to use the interpolation technology. The interpolation function of the interpolation module is disabled, and the display screen of the first application is not interpolated, thereby avoiding the problems of image ghosting and fragmentation.

[0078] Taking into account that the performance of different electronic devices is different, for the same application, the performance of the interpolation algorithm used is also different. In order to ensure processing accuracy, in an embodiment of the present application, when setting the value of the angular velocity threshold, reference is made to the performance of the interpolation algorithm corresponding to the interpolation processing. Specifically, the size of the angular velocity threshold is positively correlated with the performance of the interpolation algorithm corresponding to the interpolation processing. The higher the performance of the interpolation algorithm corresponding to the interpolation processing, the larger the value of the angular velocity threshold. Conversely, the lower the performance of the interpolation algorithm corresponding to the interpolation processing, the smaller the value of the angular velocity threshold.

[0079] In S304, screen rendering is performed on the image frame of the first application after the interpolation process.

[0080] In the embodiment of the present application, when performing interpolation processing on the display screen of the first application, one intermediate image frame or multiple intermediate image frames can be inserted, and screen rendering is performed on both the original image frame and the interpolated image frame of the first application.

[0081] In the embodiment of the present application, when the display screen of the first application is not interpolated, only the original image frame of the first application is rendered.

[0082] As can be seen from the above embodiment, in this embodiment, during the real-time rendering process of the display screen of the first application, the forward vector of the first image frame in the display screen is obtained, wherein the forward vector indicates the direction of the first image frame on the z-axis of the world coordinate system; according to the forward vector of the first image frame, the forward vector of the previous image frame of the first image frame and the interval between the two frames, the rotation angular velocity of the first image frame compared with the previous image frame is determined. Since the rotation angular velocity is used to characterize the degree of rotation of the display screen, when the rotation angular velocity is less than the angular velocity threshold, it indicates that the scene of the screen is not a fast rotation scene. The interpolation technology is used to interpolate the display screen of the first application, and the image frame of the first application after the interpolation is rendered, which can avoid the problems of ghosting and fragmentation in the screen, and improve the picture quality. The above processes are all completed independently by the electronic device operating system platform, without the need for deep cooperation from the application developer, the need for secondary development by the application developer, and the need for the software tool development kit of the integrated application. It can be dynamically effective when the application is running, reducing the R&D cost of the application and improving the R&D efficiency of the device operating system platform.

[0083] Figure 5 This is the second flowchart of a screen rendering method provided in an embodiment of the present application. Considering that the first application is a game application, for game applications, most of the current mainstream mobile game market share is developed by the Unity engine. In the embodiment of the present application, the screen rendering process of the Unity engine application is optimized, such as Figure 5 As shown, the method may include the following steps: S501, S502, S503, S504, S505 and S506;

[0084] In S501, when the first application is written based on the Unity engine, the position value of a first built-in variable named hlslcc_mtx4x4unity_WorldToCamera of the Unity engine is searched from the calling parameters of the glDrawElements function of the first application through the first function.

[0085] Since the first application is written based on the Unity engine, the graphics application programming interface "glDrawElements function" of the first application will be called during the real-time rendering process of the display screen of the first application, and each image frame of the display screen will be drawn and rendered through the glDrawElements function. The calling parameters of the glDrawElements function include the screen drawing parameters of the first image frame of the display screen. Therefore, in an embodiment of the present application, before the first application is about to call the glDrawElements function to render the first image frame, the scene recognition module intercepts the call of the glDrawElements function in the system and obtains the screen drawing parameters of the first image frame, that is, the forward vector of the first image frame, from the calling parameters of the glDrawElements function.

[0086] In the embodiment of the present application, the first function is a function in the operating system running on the electronic device. When the operating system running on the electronic device is an Android system, the first function may be a glGetUniformLocation function.

[0087] In the embodiment of the present application, when the first application is written based on the Unity engine, the forward vector of the first image frame is stored in the first built-in variable named hlslcc_mtx4x4unity_WorldToCamera.

[0088] In S502, when the position value is greater than zero, the view matrix of the first built-in variable is obtained from the calling parameters of the glDrawElements function through the second function.

[0089] In the embodiment of the present application, the second function is a function in the operating system running on the electronic device. When the operating system running on the electronic device is an Android system, the second function may be a glGetUniformfv function.

[0090] In the embodiment of the present application, the variable value of the first built-in variable named hlslcc_mtx4x4unity_WorldToCamera is a view matrix, and the view matrix is ​​shown in the following formula (1):

[0091]

[0092] Among them, Right represents Figure 1 The opposite direction of the x-axis shown, (Right x ,Right y ,Right z ) refers to the camera right direction vector; Up represents Figure 1The y-axis direction shown, (Up x , Up y , Up z ) refers to the camera's up vector; Look represents Figure 1 The z-axis direction shown in the figure is x , Look y , Look z ) refers to the camera heading vector, i.e., the forward vector of the first image frame.

[0093] In S503 , the third row element of the view matrix is ​​determined as the forward vector of the first image frame.

[0094] In the embodiment of the present application, the forward vector of the first image frame is (Look x , Look y , Look z ).

[0095] In S504, the rotational angular velocity of the first image frame compared to the previous image frame is determined based on the forward vector of the first image frame, the forward vector of the previous image frame of the first image frame and the interval between the two frames; wherein the rotational angular velocity is used to characterize the degree of rotation of the display screen.

[0096] In S505 , when the rotation angular velocity is less than the angular velocity threshold, interpolation processing is performed on the display screen of the first application.

[0097] In S506 , screen rendering is performed on the image frame of the first application after the interpolation process.

[0098] The contents of S504 to S506 in the embodiment of the present application are Figure 3 The contents of S302 to S304 in the illustrated embodiment are similar and will not be described in detail here.

[0099] It can be seen that in the embodiment of the present application, it is only necessary to integrate the functional modules capable of executing the above S501 to S506 into the system of the electronic device and compile it once, and all applications developed based on the Unity engine can be dynamically effective, thereby improving the efficiency of research and development. In addition, it does not require in-depth cooperation from the application developer, does not require secondary development, and does not require the integration of any software tool development kit. It can be dynamically effective when the application is running, thereby reducing the cost of research and development.

[0100] In some other embodiments provided in the present application, the provided screen rendering method can also be Figure 3 or Figure 5 On the basis of the embodiment shown, the following steps are added:

[0101] When the rotation angular velocity is greater than or equal to the angular velocity threshold, the clock frequency of the target processor is increased, wherein the target processor is used to run the first application, and the target processor includes: a central processing unit and / or a graphics processor.

[0102] In an embodiment of the present application, when the rotation angular velocity is greater than or equal to the angular velocity threshold, it indicates that the scene of the display screen of the first application is a rapidly rotating scene, and the screen tends to change rapidly and significantly. At this time, the smoothness requirements for the display screen of the first application will be higher. Therefore, when rapid rotation of the scene is detected, the clock frequency of the central processing unit and / or graphics processor can be appropriately increased to improve the overall performance of the first application, so that the first application runs more smoothly.

[0103] The screen rendering method provided in the embodiment of the present application can be executed by a screen rendering device. In the embodiment of the present application, the screen rendering device provided in the embodiment of the present application is described by taking the screen rendering method executed by the screen rendering device as an example.

[0104] Figure 6 is a structural block diagram of a picture rendering device provided in an embodiment of the present application, such as Figure 6 As shown, the picture rendering device 600 may include: an acquisition module 601, a determination module 602, a processing module 603 and a rendering module 604;

[0105] An acquisition module 601 is used to acquire a forward vector of a first image frame in a display screen of a first application, wherein the forward vector represents a direction of the first image frame on a world coordinate z-axis;

[0106] A determination module 602 is used to determine a rotation angular velocity of the first image frame compared with the previous image frame according to a forward vector of the first image frame, a forward vector of an image frame previous to the first image frame, and an interval between the two frames, wherein the rotation angular velocity is used to represent a degree of rotation of the display screen;

[0107] A processing module 603 is used to perform frame insertion processing on the display picture of the first application when the rotation angular velocity is less than an angular velocity threshold;

[0108] The rendering module 604 is used to render the image frames of the first application after the interpolation process.

[0109] As can be seen from the above embodiment, in this embodiment, during the real-time rendering process of the display screen of the first application, the forward vector of the first image frame in the display screen is obtained, wherein the forward vector indicates the direction of the first image frame on the z-axis of the world coordinate system; according to the forward vector of the first image frame, the forward vector of the previous image frame of the first image frame and the interval between the two frames, the rotation angular velocity of the first image frame compared with the previous image frame is determined. Since the rotation angular velocity is used to characterize the degree of rotation of the display screen, when the rotation angular velocity is less than the angular velocity threshold, it indicates that the scene of the screen is not a fast rotation scene. The interpolation technology is used to interpolate the display screen of the first application, and the image frame of the first application after the interpolation is rendered, which can avoid the problems of ghosting and fragmentation in the screen, and improve the picture quality. The above processes are all completed independently by the electronic device operating system platform, without the need for deep cooperation from the application developer, the need for secondary development by the application developer, and the need for the software tool development kit of the integrated application. It can be dynamically effective when the application is running, reducing the R&D cost of the application and improving the R&D efficiency of the device operating system platform.

[0110] Optionally, as an embodiment, the acquisition module 601 may include:

[0111] The acquisition submodule is used to acquire the forward vector of the first image frame in the display screen of the first application from the calling parameters of the graphics application programming interface of the first application.

[0112] Optionally, as an embodiment, the first application is written based on a first development engine;

[0113] The acquisition submodule may include:

[0114] A search unit, configured to search, through a first function, for a position value of a first built-in variable of the first development engine from a call parameter of a graphical application programming interface of a first application;

[0115] A first acquisition unit is used for acquiring a variable value of the first built-in variable from a call parameter of the graphical application programming interface through a second function when the position value is greater than zero, wherein the first function and the second function are functions in an operating system running on the electronic device;

[0116] The second acquisition unit is used to acquire a forward vector of a first image frame in a display screen of the first application from a variable value of the first built-in variable.

[0117] Optionally, as an embodiment, when the first development engine is the Unity engine, the variable value of the first built-in variable is a view matrix, and the elements of the third row of the view matrix constitute a forward vector of the first image frame.

[0118] Optionally, as an embodiment, the size of the angular velocity threshold is positively correlated with the performance of the frame interpolation algorithm corresponding to the frame interpolation processing.

[0119] The screen rendering device in the embodiment of the present application can be a device, or a component, integrated circuit, or chip in a terminal. The device can be a mobile electronic device or a non-mobile electronic device. Exemplarily, the mobile electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, an in-vehicle electronic device, a wearable device, an ultra-mobile personal computer (Ultra-Mobile Personal Computer, UMPC), a netbook or a personal digital assistant (Personal Digital Assistant, PDA), etc., and the non-mobile electronic device can be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (Personal Computer, PC), a television (Television, TV), a teller machine or a self-service machine, etc., which is not specifically limited in the embodiment of the present application.

[0120] The screen rendering device in the embodiment of the present application may be a device having a device operating system. The device operating system may be an Android device operating system, an IOS device operating system, or other possible device operating systems, which are not specifically limited in the embodiment of the present application.

[0121] The screen rendering device provided in the embodiment of the present application can achieve Figure 3 or Figure 5 To avoid repetition, the various processes implemented in the illustrated method embodiment will not be described again here.

[0122] Alternatively, if Figure 7 As shown, an embodiment of the present application also provides an electronic device 700, including a processor 701, a memory 702, and a program or instruction stored in the memory 702 and executable on the processor 701. When the program or instruction is executed by the processor 701, each process of the above-mentioned screen rendering method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.

[0123] It should be noted that the electronic devices in the embodiments of the present application include the mobile electronic devices and non-mobile electronic devices mentioned above.

[0124] Figure 8 It is a schematic diagram of the hardware structure of an electronic device implementing an embodiment of the present application.

[0125] The electronic device 800 includes but is not limited to: a radio frequency unit 801, a network module 802, an audio output unit 803, an input unit 804, a sensor 805, a display unit 806, a user input unit 807, an interface unit 808, a memory 809, and a processor 810 and other components.

[0126] Those skilled in the art will appreciate that the electronic device 800 may also include a power source (such as a battery) for supplying power to each component, and the power source may be logically connected to the processor 810 through a power management system, thereby implementing functions such as managing charging, discharging, and power consumption management through the power management system. Figure 8 The electronic device structure shown in the figure does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently, which will not be described in detail here.

[0127] Among them, the processor 810 is used to obtain a forward vector of a first image frame in a display screen of a first application, wherein the forward vector represents the direction of the first image frame on the world coordinate z-axis; determine a rotation angular velocity of the first image frame compared with the previous image frame according to the forward vector of the first image frame, the forward vector of an image frame previous to the first image frame, and the interval between the two frames, wherein the rotation angular velocity is used to characterize the degree of rotation of the display screen; when the rotation angular velocity is less than an angular velocity threshold, perform interpolation processing on the display screen of the first application, and perform screen rendering on the image frame of the first application after the interpolation processing.

[0128] It can be seen that in the embodiment of the present application, during the real-time rendering process of the display screen of the first application, the forward vector of the first image frame in the display screen is obtained, wherein the forward vector indicates the direction of the first image frame on the z-axis of the world coordinate system; according to the forward vector of the first image frame, the forward vector of the previous image frame of the first image frame and the interval between the two frames, the rotation angular velocity of the first image frame compared with the previous image frame is determined. Since the rotation angular velocity is used to characterize the degree of rotation of the display screen, when the rotation angular velocity is less than the angular velocity threshold, it indicates that the scene of the screen is not a fast rotation scene. The interpolation technology is used to interpolate the display screen of the first application, and the image frame of the first application after the interpolation is rendered, which can avoid the problems of ghosting and fragmentation in the screen and improve the picture quality. The above processes are all completed independently by the electronic device operating system platform, without the need for deep cooperation from the application developer, the need for secondary development by the application developer, and the need for the software tool development kit of the integrated application. It can be dynamically effective when the application is running, reducing the R&D cost of the application and improving the R&D efficiency of the device operating system platform.

[0129] Optionally, as an embodiment, the processor 810 is specifically configured to obtain a forward vector of a first image frame in a display screen of the first application from a calling parameter of a graphics application programming interface of the first application.

[0130] Optionally, as an embodiment, the first application is written based on a first development engine;

[0131] The processor 810 is specifically configured to search, through a first function, for a position value of a first built-in variable of the first development engine from a calling parameter of a graphical application programming interface of a first application;

[0132] When the position value is greater than zero, obtaining the variable value of the first built-in variable from the calling parameter of the graphical application programming interface through a second function, wherein the first function and the second function are functions in an operating system running on the electronic device;

[0133] A forward vector of a first image frame in a display screen of the first application is obtained from a variable value of the first built-in variable.

[0134] Optionally, as an embodiment, when the first development engine is the Unity engine, the variable value of the first built-in variable is a view matrix, and the elements of the third row of the view matrix constitute a forward vector of the first image frame.

[0135] Optionally, as an embodiment, the size of the angular velocity threshold is positively correlated with the performance of the frame interpolation algorithm corresponding to the frame interpolation processing.

[0136] It should be understood that in the embodiment of the present application, the input unit 804 may include a graphics processor (Graphics Processing Unit, GPU) 8041 and a microphone 8042, and the graphics processor 8041 processes the image data of the static picture or video obtained by the image capture device (such as a camera) in the video capture mode or the image capture mode. The display unit 806 may include a display panel 8061, and the display panel 8061 may be configured in the form of a liquid crystal display, an organic light emitting diode, etc. The user input unit 807 includes a touch panel 8071 and other input devices 8072. The touch panel 8071 is also called a touch screen. The touch panel 8071 may include two parts: a touch detection device and a touch controller. Other input devices 8072 may include but are not limited to a physical keyboard, a function key (such as a volume control button, a switch button, etc.), a trackball, a mouse, and a joystick, which will not be repeated here. The memory 809 can be used to store software programs and various data, including but not limited to applications and operating systems. The processor 810 can integrate an application processor and a modem processor, wherein the application processor mainly processes an operating system, a user interface, and applications, etc., and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 810.

[0137] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the various processes of the above-mentioned screen rendering method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0138] The processor is a processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0139] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned screen rendering method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0140] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0141] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0142] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), including instructions for enabling a terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0143] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. A picture rendering method, characterized in that: The method comprises: Acquire a forward vector of a first image frame in a display screen of a first application, wherein the forward vector represents a direction of the first image frame on a world coordinate z-axis; Determining a rotational angular velocity of the first image frame compared with the previous image frame according to a forward vector of the first image frame, a forward vector of an image frame previous to the first image frame, and an interval between the two frames, wherein the rotational angular velocity is used to characterize a degree of rotation of the display screen; When the rotation angular velocity is less than an angular velocity threshold, interpolation processing is performed on the display screen of the first application, and screen rendering is performed on the image frame of the first application after the interpolation processing.

2. The method according to claim 1, characterized in that The obtaining of a forward vector of a first image frame in a display screen of a first application includes: A forward vector of a first image frame in a display screen of the first application is obtained from a calling parameter of a graphics application programming interface of the first application.

3. The method according to claim 2, characterized in that The first application is written based on a first development engine; The obtaining, from the calling parameters of the graphics application programming interface of the first application, the forward vector of the first image frame in the display screen of the first application comprises: Using a first function, searching for a position value of a first built-in variable of the first development engine from a calling parameter of a graphical application programming interface of a first application; When the position value is greater than zero, obtaining the variable value of the first built-in variable from the calling parameter of the graphical application programming interface through a second function, wherein the first function and the second function are functions in an operating system running on the electronic device; A forward vector of a first image frame in a display screen of the first application is obtained from a variable value of the first built-in variable.

4. The method according to claim 3, characterized in that When the first development engine is the Unity engine, the variable value of the first built-in variable is a view matrix, and the elements of the third row of the view matrix constitute a forward vector of the first image frame.

5. The method according to claim 1, characterized in that The magnitude of the angular velocity threshold is positively correlated with the performance of the frame interpolation algorithm corresponding to the frame interpolation process.

6. A picture rendering device, characterized in that: The device comprises: An acquisition module, configured to acquire a forward vector of a first image frame in a display screen of a first application, wherein the forward vector indicates a direction of the first image frame on a world coordinate z-axis; a determination module, configured to determine a rotation angular velocity of the first image frame compared with the previous image frame according to a forward vector of the first image frame, a forward vector of an image frame previous to the first image frame, and an interval between the two frames, wherein the rotation angular velocity is used to characterize a degree of rotation of the display screen; a processing module, configured to perform frame insertion processing on a display picture of the first application when the rotation angular velocity is less than an angular velocity threshold; A rendering module is used to render the image frames of the first application after the interpolation processing.

7. The device according to claim 6, characterized in that The acquisition module comprises: The acquisition submodule is used to acquire the forward vector of the first image frame in the display screen of the first application from the calling parameters of the graphics application programming interface of the first application.

8. The device according to claim 7, characterized in that The first application is written based on a first development engine; The acquisition submodule includes: A search unit, configured to search, through a first function, for a position value of a first built-in variable of the first development engine from a call parameter of a graphical application programming interface of a first application; A first acquisition unit is used for acquiring a variable value of the first built-in variable from a call parameter of the graphical application programming interface through a second function when the position value is greater than zero, wherein the first function and the second function are functions in an operating system running on the electronic device; The second acquisition unit is used to acquire a forward vector of a first image frame in a display screen of the first application from a variable value of the first built-in variable.

9. The device according to claim 8, characterized in that When the first development engine is the Unity engine, the variable value of the first built-in variable is a view matrix, and the elements of the third row of the view matrix constitute a forward vector of the first image frame.

10. The device according to claim 6, characterized in that The magnitude of the angular velocity threshold is positively correlated with the performance of the frame interpolation algorithm corresponding to the frame interpolation process.

Images