Rendering processing method and rendering equipment

Abstract

The invention discloses a rendering processing method and rendering equipment, relates to the technical field of image processing, and aims to reduce heating of terminal equipment and save power consumption. The rendering processing method comprises the steps of firstly intercepting a first rendering instruction sent by a target application and used for rendering a to-be-rendered image of the target application; then obtaining a second rendering instruction for rendering a previous frame of image of the to-be-rendered image; determining view conversion information of the to-be-rendered image relative to the previous frame image according to the first rendering instruction and the second rendering instruction; adjusting an original resolution parameter of at least one rendering channel in the first rendering instruction to a lower preset resolution parameter under the condition that the view conversion information conforms to the preset dynamic scene; and finally, sending the adjusted first rendering instruction to render the to-be-rendered image.

Description

[0002] The present application relates to the field of image processing technology, and in particular to a rendering processing method and a rendering device. Background Art

[0003] As users' requirements for the visual effects of applications become higher and higher, the image resolution of applications is also getting higher and higher. The image resolution of an application depends on how the image is rendered. Specifically, the process of rendering a scene is usually that the central processing unit (CPU) of the terminal device sends rendering data to the graphics processing unit (GPU) for the GPU to render the image based on the rendering data. Since a dynamic scene is generally composed of a large number of sequential frame images, the GPU needs to render each frame of the dynamic scene, which is very time-consuming. In addition, the higher the image resolution of the application, the more detailed the GPU needs to render each frame of the image, which means that the GPU needs to bear a huge amount of computing power.

[0004] In summary, when rendering images of higher-quality applications, the GPU load is high, causing the terminal device to heat up and consume power quickly. Summary of the invention

[0005] The present application provides a rendering processing method and a rendering device, which reduce the heat generation of terminal equipment and save power consumption.

[0006] In order to achieve the above objectives, this application adopts the following technical solutions:

[0007] In a first aspect, a rendering processing method is provided, which first intercepts a first rendering instruction sent by a target application for rendering an image to be rendered of the target application; then obtains a second rendering instruction for rendering a previous frame of the image to be rendered; then determines field of view transformation information of the image to be rendered relative to the previous frame of the image based on the first rendering instruction and the second rendering instruction; and when the field of view transformation information conforms to a preset dynamic scene, adjusts the original resolution parameter of at least one rendering channel in the first rendering instruction to a lower preset resolution parameter; finally, sends the adjusted first rendering instruction to render the image to be rendered.

[0008] In the first aspect, a first rendering instruction indicating a higher resolution sent by a target application is intercepted, and when the field of view change information conforms to a preset dynamic scene, the original resolution parameters of some rendering channels in the first rendering instruction are lowered before being sent to the graphics processor for rendering. This reduces the amount of rendering calculations while relatively ensuring the rendering quality, thereby improving the rendering efficiency, increasing the frame rate of the target application, and reducing the heat and power consumption of the terminal device.

[0009] In a possible implementation, the method also includes: when the field of view transformation information conforms to a preset static scene, generating a third rendering instruction, the third rendering instruction including a first sub-instruction and a second sub-instruction, specifically, the first sub-instruction is used to indicate the use of the rendering result of at least one rendering channel of the previous frame image as the first rendering result of the image to be rendered, and the second sub-instruction includes the original resolution parameters of the remaining rendering channels except at least one rendering channel in the first rendering instruction; sending the third rendering instruction for rendering the image to be rendered.

[0010] In this implementation, when the field of view transformation information conforms to the preset static scene, a third rendering instruction is generated to instruct to reuse the rendering result of a partial rendering channel of the previous frame image as the rendering result of the image to be rendered, thereby reducing the required rendering operation amount.

[0011] In one possible implementation, adjusting the original resolution parameter of at least one rendering channel in the first rendering instruction to a preset resolution parameter includes: determining multiple rendering channels in the first rendering instruction, and specification parameters of the multiple rendering channels; the specification parameters include at least one of the following: rendering channel size, frame buffer attachments, and number of drawing call interfaces; adjusting the original resolution parameter of a target rendering channel among the multiple rendering channels to the preset resolution parameter, wherein the specification parameters of the target rendering channel meet the preset specification conditions.

[0012] In this implementation, only the original resolution parameters of the target rendering channel are adjusted to the preset resolution parameters. In this way, when rendering is performed based on the adjusted first rendering instruction, the rendering calculation amount can be reduced while relatively ensuring the rendering quality, thereby improving the rendering efficiency to increase the frame rate of the target application and reduce the heat and power consumption of the terminal device.

[0013] In a possible implementation, different sizes of field of view change information correspond to different preset dynamic scenes; the preset resolution parameters under different preset dynamic scenes are different, or the preset resolution parameters under different preset dynamic scenes are the same.

[0014] In this implementation, preset resolution parameters under different preset dynamic scenes are flexibly defined.

[0015] In one possible implementation, the field of view transformation information includes displacement transformation information and angle transformation information; the first rendering instruction includes a first field of view matrix and a first coordinate, and the second rendering instruction includes a second field of view matrix and a second coordinate; the field of view transformation information of the image to be rendered relative to the previous frame image is determined according to the first rendering instruction and the second rendering instruction, including: calculating the difference between the first coordinate and the second coordinate, and outputting the displacement transformation information; calculating the first plane normal vector according to the first field of view matrix, and calculating the second plane normal vector according to the second field of view matrix; calculating the angle between the first plane normal vector and the second plane normal vector, and outputting the angle transformation information.

[0016] In this implementation, the field of view transformation information can be accurately calculated, and then based on the field of view transformation information, it can be determined whether the transformation scene of the Camera is a preset dynamic scene or a preset static scene, so as to execute different rendering instruction adjustment strategies based on different scenes.

[0017] In a second aspect, a rendering processing method is provided, the method comprising: receiving a third rendering instruction; wherein the third rendering instruction comprises a first sub-instruction and a second sub-instruction, the first sub-instruction being used to indicate that the rendering result of at least one rendering channel of the previous frame image of the image to be rendered by the target application is used as the first rendering result of the image to be rendered, the second sub-instruction comprising original resolution parameters of the remaining rendering channels except at least one rendering channel in the first rendering instruction, the first rendering instruction being an instruction sent by the target application for rendering the image to be rendered; rendering the image to be rendered according to the third rendering instruction to obtain a full rendering result of the image to be rendered, wherein the full rendering result comprises the first rendering result and the second rendering result, the second rendering result being obtained by rendering the image to be rendered by the remaining rendering channels based on the original resolution parameters.

[0018] In the second aspect, the image to be rendered is rendered based on the third rendering instruction. Since the rendering channel corresponding to the reused rendering result does not need to be rendered again, the rendering calculation amount is reduced, the rendering efficiency is improved, the frame rate of the target application is improved, and the heat and power consumption of the terminal device are reduced.

[0019] In a third aspect, a rendering processing device is provided, including: an interception module, used to intercept a first rendering instruction sent by a target application, wherein the first rendering instruction is used to render an image to be rendered of the target application; an acquisition module, used to acquire a second rendering instruction, wherein the second rendering instruction is used to render a previous frame image of the image to be rendered; a determination module, used to determine field of view transformation information of the image to be rendered relative to the previous frame image according to the first rendering instruction and the second rendering instruction; an adjustment module, used to adjust the original resolution parameter of at least one rendering channel in the first rendering instruction to a preset resolution parameter when the field of view transformation information conforms to a preset dynamic scene, and the preset resolution parameter is lower than the original resolution parameter; a sending module, used to send the adjusted first rendering instruction; the adjusted first rendering instruction is used to render the image to be rendered.

[0020] In one possible implementation, the device also includes: a generating module, used to generate a third rendering instruction when the field of view transformation information conforms to a preset static scene, wherein the third rendering instruction includes a first sub-instruction and a second sub-instruction, the first sub-instruction is used to indicate the use of the rendering result of a partial rendering channel of the previous frame image as the partial rendering result of the image to be rendered, and the second sub-instruction includes the original resolution parameters of the remaining rendering channels except the partial rendering channels in the first rendering instruction; the sending module is also used to send the third rendering instruction, and the third rendering instruction is used to render the image to be rendered.

[0021] In one possible implementation, the adjustment module is specifically used to: determine multiple rendering channels in the first rendering instruction, and specification parameters of the multiple rendering channels; adjust the original resolution parameters of the target rendering channel among the multiple rendering channels to preset resolution parameters; wherein the specification parameters include at least one of the following: rendering channel size, frame buffer attachments, and the number of drawing call interfaces, and the specification parameters of the target rendering channel meet the preset specification conditions.

[0022] In a possible implementation, different sizes of field of view change information correspond to different preset dynamic scenes; the preset resolution parameters under different preset dynamic scenes are different, or the preset resolution parameters under different preset dynamic scenes are the same.

[0023] In one possible implementation, the field of view transformation information includes displacement transformation information and angle transformation information; the first rendering instruction includes a first field of view matrix and a first coordinate, and the second rendering instruction includes a second field of view matrix and a second coordinate; a determination module is specifically used to: calculate the difference between the first coordinate and the second coordinate, and output the displacement transformation information; calculate the first plane normal vector according to the first field of view matrix, and calculate the second plane normal vector according to the second field of view matrix; calculate the angle between the first plane normal vector and the second plane normal vector, and output the angle transformation information.

[0024] In a fourth aspect, a rendering processing device is provided, including: a receiving module, used to receive a third rendering instruction; wherein the third rendering instruction includes a first sub-instruction and a second sub-instruction, the first sub-instruction is used to indicate that the rendering result of at least one rendering channel of the previous frame image of the target application to be rendered is used as the first rendering result of the image to be rendered, and the second sub-instruction includes the original resolution parameters of the remaining rendering channels except at least one rendering channel in the first rendering instruction, and the first rendering instruction is an instruction sent by the target application for rendering the image to be rendered; a rendering module, used to render the image to be rendered according to the third rendering instruction to obtain a full rendering result of the image to be rendered, wherein the full rendering result includes the first rendering result and the second rendering result, and the second rendering result is obtained by rendering the image to be rendered by the remaining rendering channels based on the original resolution parameters.

[0025] In a fifth aspect, the present application provides a rendering device, the rendering device comprising a processor and a transceiver, the processor and the transceiver are used to support the rendering device to execute the method of the first aspect or the second aspect. Further, the rendering device may also include a memory, the memory stores computer instructions, and the processor can execute the computer instructions to execute the method of the first aspect or the second aspect.

[0026] In a sixth aspect, the present application provides a computer-readable storage medium, which stores computer instructions. When the computer instructions are executed, the method of the first aspect or the second aspect is executed.

[0027] In a seventh aspect, the present application provides a computer program product comprising instructions, which, when executed on a computer, enables the computer to execute the method of the first aspect or the second aspect described above.

[0028] In an eighth aspect, the present application provides a chip, comprising a processor and a transceiver, wherein the processor and the transceiver are used to support a rendering processing device to execute the method of the first aspect or the second aspect.

[0029] Among them, the beneficial effects described in the third to sixth aspects of the present application can refer to the analysis of the beneficial effects of the first or second aspect, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 A schematic diagram of the structure of a rendering device provided in an embodiment of the present application;

[0031] Figure 2 A schematic diagram of the architecture of a rendering system provided in an embodiment of the present application;

[0032] Figure 3 A flowchart of a rendering processing method provided in an embodiment of the present application;

[0033] Figure 4 A flowchart of another rendering processing method provided in an embodiment of the present application;

[0034] Figure 5 A flowchart of another rendering processing method provided in an embodiment of the present application;

[0035] Figure 6 A schematic diagram of a rendering channel provided in an embodiment of the present application;

[0036] Figure 7 A flowchart of another rendering processing method provided in an embodiment of the present application;

[0037] Figure 8 A schematic diagram of the structure of a rendering processing device provided in an embodiment of the present application;

[0038] Fig. 9 A schematic diagram of the structure of another rendering processing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0039] The network architecture and business scenarios described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. A person of ordinary skill in the art can appreciate that with the evolution of the network architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0040] It should be noted that the terms "first" and "second" in the specification, claims and drawings of the present application are used to distinguish different objects rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units that are not listed, or may optionally include other steps or units that are inherent to these processes, methods, products or devices.

[0041] It should be understood that in the embodiments of the present application, "at least one (item)" refers to one or more, "multiple" refers to two or more, "at least two (items)" refers to two or three and more than three, and "and / or" is used to describe the association relationship of the associated objects, indicating that there can be three relationships. For example, "A and / or B" can represent: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the associated objects before and after are in an "or" relationship. "At least one of the following (items)" or similar expressions refers to any combination of these items, including any combination of single items (items) or plural items (items). For example, at least one of a, b or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple. It should be understood that in the embodiments of the present application, "B corresponding to A" means that B is associated with A. For example, B can be determined based on A. It should also be understood that determining B based on A does not mean determining B based solely on A. B can also be determined based on A and / or other information.

[0042] Before introducing the embodiments of the present application, some terms involved in the embodiments of the present application are explained.

[0043] GPU: also known as display core, visual processor, display chip, is a microprocessor that specializes in performing image and graphics related calculations on personal computers, workstations, game consoles and some mobile devices (such as tablets, smart phones, etc.).

[0044] Model View Projection Matrix (MVP Matrix): It can be referred to as MVP matrix. Specifically, MVP matrix refers to the combination of model, view and projection matrices. The model matrix is ​​used to transform the object from the local coordinate system to the world coordinate system, the view matrix is ​​used to transform the world coordinate system to the camera coordinate system, and the projection matrix is ​​used to transform the camera coordinate system to the clipping coordinate system.

[0045] Draw Call Interface (Draw Call): Draw Call refers to the process of the CPU sending a drawing instruction to the GPU. In order to draw an object, the CPU needs to send a Draw Call to the GPU. The more Draw Calls there are, the greater the load on the CPU and GPU, which affects the performance of the terminal device.

[0046] Rendering Pass: Pass for short, is an important part of the rendering pipeline. Every detail seen in the final rendered frame (also called the rendering result) is calculated through a series of Passes. Pass can generate the textures, buffers, and render targets required for the image. Each image generated by Pass contains specific information about the scene, such as color, normals, and depth information. These image information are combined to generate more complex effects, such as shadows, lighting, blur, glow, and other post-processing effects. Pass is a logical grouping of rendering work with defined input slots and output slots. It receives the input data passed to it through the input slot, calculates it, and then generates an output data, which is passed to other Passes through the output slot.

[0047] In one embodiment, a pass may be a render pass, and the rendering instructions of an application include a group of associated render passes.

[0048] Camera: It can be called a virtual camera, which is very similar to a real camera. The Camera component in is responsible for capturing the game screen and projecting it onto the screen. The screen captured by the Camera determines the screen of the Game panel. When creating a scene, A Camera will be created by default, so the screen will appear only when the user clicks on the Game panel. In game applications, the Camera is generally mobile. For example, in a first-person game, the Camera will always move with our operations. Specifically, the coordinates of the Camera can correspond to a certain point in the world coordinate system. The visible range determined by the Camera determines the screen range of the game. In the various embodiments of the present application, the Camera is also referred to as the field of view.

[0049] In order to meet users' increasing demand for application image quality, games, movies, design and other applications are increasingly moving towards high resolution and high frame rate. The higher the image resolution, the greater the amount of GPU computing required, which causes the terminal device to heat up and consume power quickly.

[0050] In order to solve the above technical problems, an embodiment of the present application provides a rendering processing method. The method provided by the embodiment of the present application is described below in conjunction with the drawings in the specification.

[0051] The communication method provided in the embodiment of the present application can be applied to a rendering device. Exemplarily, the rendering device can be a terminal device (terminal equipment), and the terminal device can be a user equipment (user equipment, UE), a mobile station (mobile station, MS) or a mobile terminal (mobile terminal, MT), etc. Specifically, the terminal device can be a mobile phone, a tablet computer or a computer with a wireless transceiver function, and can also be a virtual reality (virtual reality, VR) terminal, an augmented reality (augmented reality, AR) terminal, etc. In the embodiment of the present application, the device for realizing the function of the terminal device can be a terminal, or it can be a device that can support the terminal to realize the function, such as a chip system, which can be installed in the terminal device or used in combination with the terminal device.

[0052] Exemplarily, the present application embodiment provides a rendering device 100, the structure of which is as follows: Figure 1 As shown, the rendering device 100 includes: a processor 101 , a graphics processor 102 , and a memory 103 .

[0053] Among them, the processor 101 can be a CPU, and the processor 101 can also be other general-purpose processors, digital signal processors (digital signal processor, DSP), application specific integrated circuits (application specific integrated circuit, ASIC), field-programmable gate arrays (field-programmable gate array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components and other chips, or a combination of the above-mentioned types of chips.

[0054] The processor 101 is used to generate a first rendering instruction according to a rendering request of the application, and adjust the first rendering instruction using the rendering processing method of the embodiment of the present application, and then convert the adjusted rendering instruction into hardware code, and send it to the graphics processor 102 for execution.

[0055] The graphics processor 102 is used to render an image of the application by executing hardware codes corresponding to the rendering instructions.

[0056] The memory 103 may be a volatile memory, such as a random-access memory (RAM); or a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD); or a combination of the above types of memories, for storing rendering instructions, rendering resources, shader parameters, rendering status, applications, configuration files, data information or other content that can implement the method of the present application.

[0057] The present application also provides a rendering system, which can be deployed in Figure 1 on the rendering device shown. Figure 2 As shown, Figure 2 The schematic rendering system includes an application layer, a software framework layer, a driver layer, and a hardware layer.

[0058] Specifically, the application layer, the software framework layer, and the driver layer may run on the processor 101 of the rendering device 100 , and the hardware layer may run on the graphics processor 102 of the rendering device 100 .

[0059] Among them, the application layer mainly carries graphics rendering programs, which include games, graphics applications (application, APP), display user interfaces (user interface, UI), cameras (Camera), virtual and augmented reality technologies (virtual reality / augmented reality, VR / AR), etc. The graphics rendering program of the application layer can run on the CPU, receive a rendering request issued by the user, and based on the rendering request, call the target application to generate a first rendering instruction, and send the generated first rendering instruction to the software framework layer.

[0060] The software framework layer mainly carries the rendering engine, which includes the game engine Unreal Engine, Graphics Display System HWUI), game development engine (Cocos), etc. The software framework layer can run on the CPU, receive the first rendering instruction of the application layer, and execute the rendering processing method of the embodiment of the present application based on the first rendering instruction to obtain the adjusted first rendering instruction, and improve the adjusted first rendering instruction based on the rendering engine.

[0061] The driver layer mainly carries the graphics driver program. The driver layer can run on the CPU and convert the adjusted first rendering instruction into hardware code and send it to the hardware layer.

[0062] The hardware layer mainly carries graphics hardware, and illustratively, the hardware may be a GPU, etc. The hardware layer may run on the GPU, and execute the hardware code corresponding to the adjusted first rendering instruction to render an image of the application.

[0063] For example, when a user runs a game application on a mobile phone, a scene is rendered based on a certain operation of the user. The game application running on the application layer on the CPU responds to the user's operation, generates a rendering request and initially generates a first rendering instruction. The game engine running on the software framework layer on the CPU then adjusts the rendering instruction. The driver running on the driver layer on the CPU converts the adjusted rendering instruction into hardware code and sends it to the hardware layer. The hardware layer can run the hardware code on the GPU to render the image of the application.

[0064] Specifically, the process of the software framework layer in the rendering system executing the rendering processing method of the embodiment of the present application will be described in detail in the following method content and will not be repeated here.

[0065] It should be noted that the above Figure 1 Schematic rendering equipment, Figure 2 The illustrated rendering system is only an example of the application scenario of the present application solution, and is not a limitation of the application scenario of the present application solution.

[0066] Based on the above-mentioned rendering device introduced in the embodiment of the present application, the embodiment of the present application provides a rendering processing method. Figure 3 FIG. 1 is a flow chart of a rendering processing method provided by an embodiment of the present application. Figure 3 As shown, the method may include the following steps:

[0067] S310: The processor intercepts a first rendering instruction sent by a target application.

[0068] Among them, when processing rendering instructions sent by various applications, the processor can determine whether it is a target application based on the application information. And intercept when it is determined that the rendering instruction is sent by the target application. The target application is a type of application with higher resolution requirements, and can be exemplarily a game application, VR application, AR application, etc. It is precisely because of its higher high-resolution requirements that if the image to be rendered is rendered based on the first rendering instruction generated by the target application, the resolution of the rendering result is also higher, and the rendering process also has a higher load on the graphics processor.

[0069] Referring to the introduction of the application layer in the rendering system provided in the embodiment of the present application, the application layer receives the rendering request issued by the user and, based on the rendering request, calls the target application to generate a first rendering instruction, and sends the generated first rendering instruction to the software framework layer. The software framework layer usually directly improves the first rendering instruction, while the embodiment of the present application intercepts the first rendering instruction, adjusts the first rendering instruction, and then the rendering engine of the software framework layer improves the adjusted first rendering instruction. In other words, after intercepting the first rendering instruction, the terminal device cannot render based on the first rendering instruction.

[0070] Installed with terminal device Taking the operating system as an example, you can use The reserved interface of the operating system OpenGL and Vulkan intercepts the first rendering instruction, and before the software framework layer calls the driver layer, the first rendering instruction is intercepted and cached into an instruction stream.

[0071] S320: The processor obtains a second rendering instruction.

[0072] The second rendering instruction is used to render the previous frame of the image to be rendered. When the target application renders the scene, the image is rendered frame by frame. When the image to be rendered is rendered, the previous frame has been rendered. The rendering of the previous frame is also triggered by the rendering instruction (i.e., the second rendering instruction) generated by the target application. The second rendering instruction is generally stored in the cache as historical data, and the second rendering instruction can be obtained from the cache.

[0073] S330: The processor determines field of view transformation information of the image to be rendered relative to the previous frame image according to the first rendering instruction and the second rendering instruction.

[0074] The first rendering instruction and the second rendering instruction respectively record the field of view information of the image to be rendered and the previous frame image. By comparing the two sets of field of view information, the field of view transformation information of the image to be rendered and the previous frame image can be determined.

[0075] S340: When the field of view transformation information conforms to a preset dynamic scene, the processor adjusts an original resolution parameter of at least one rendering channel in the first rendering instruction to a preset resolution parameter.

[0076] Among them, field of view change information of different sizes corresponds to different preset dynamic scenes; the preset resolution parameters under different preset dynamic scenes may be the same or different, or may be the same. The preset dynamic scene may be a type of scene in which the field of view change information represents a large degree of field of view change of the image to be rendered relative to the previous frame of the image. Exemplarily, if the target application is a game application, the preset dynamic scene may be a scene in which a character in the game walks, a character runs, or the user's perspective rotates. In this case, the original resolution parameter of at least one rendering channel in the first rendering instruction is lowered (i.e., adjusted to the preset resolution parameter) to reduce the amount of rendering operations.

[0077] S350: The processor sends the adjusted first rendering instruction to the graphics processor, and correspondingly, the graphics processor receives the adjusted first rendering instruction.

[0078] The processor sends the adjusted first rendering instruction to the graphics processor, so that the graphics processor renders the image to be rendered based on the first rendering instruction.

[0079] In an embodiment of the present application, a first rendering instruction indicating a higher resolution sent by a target application is intercepted, and when the field of view change information conforms to a preset dynamic scene, the original resolution parameters of some rendering channels in the first rendering instruction are lowered and then sent to the graphics processor for rendering. This reduces the amount of rendering operations while relatively ensuring the rendering quality, thereby improving the rendering efficiency, thereby increasing the frame rate of the target application, and reducing the heat and power consumption of the terminal device.

[0080] In addition, there is no need for code adaptation between applications and application engines, and it does not rely on any specific hardware platform. Based on the fact that rendering instructions are transmitted in the form of data streams, the scene of field of view change between the previous and next frames is determined, and the resolution of the rendering channel is adjusted in a targeted manner to reduce the target application load and improve the target application frame rate. This allows the target application to reduce the power consumption of the terminal device without the user noticing, and delay the time when the target application is limited by the system frequency and frame, thereby extending the target application running time and improving the user experience.

[0081] In one embodiment, if Figure 4 As shown, the method may also include:

[0082] S410: When the field of view transformation information conforms to a preset static scene, the processor generates a third rendering instruction.

[0083] Among them, the preset static scene can be a type of scene in which the field of view change information characterizes that the field of view change degree of the image to be rendered is relatively small relative to the previous frame image. Exemplarily, if the target application is a game application, the preset static scene can be a scene in which the Camera is stationary in the game. In this case, reusing the rendering results of part of the rendering channels of the previous frame image as the rendering results of the image to be rendered can still ensure the quality of the target application image. At this time, a third rendering instruction is generated, and the third rendering instruction includes a first sub-instruction and a second sub-instruction. The first sub-instruction is used to indicate that the rendering results of at least one rendering channel of the previous frame image are used as the first rendering result of the image to be rendered, and the second sub-instruction includes the original resolution parameters of the remaining rendering channels except at least one rendering channel in the first rendering instruction.

[0084] In another example, if the user does not operate on the game character and the game UI, the rendering results of all rendering channels of the previous frame image can be reused. In this case, the first sub-instruction is used to instruct the use of the rendering results of all rendering channels of the previous frame image as the first rendering result of the image to be rendered. Then the second sub-instruction does not exist, that is, the third rendering instruction includes the first sub-instruction. In this case, the third rendering instruction simply instructs the reuse of rendering results without the need for specific rendering.

[0085] S420: The processor sends a third rendering instruction to the graphics processor. Correspondingly, the graphics processor receives the third rendering instruction.

[0086] S430: The graphics processor renders the image to be rendered according to the third rendering instruction to obtain a full rendering result of the image to be rendered.

[0087] Among them, when the graphics processor renders the image to be rendered according to the third rendering instruction, it is divided into two rendering processes. One is to use the rendering result of at least one rendering channel of the previous frame image of the image to be rendered as the first rendering result of the image to be rendered, and the other is that the remaining rendering channels render the image to be rendered based on the original resolution parameters to obtain the second rendering result, and finally obtain the full rendering result.

[0088] In an embodiment of the present application, when the field of view transformation information conforms to a preset static scene, a third rendering instruction is generated to instruct the reuse of the rendering result of part of the rendering channel of the previous frame image as the rendering result of the image to be rendered, and the image to be rendered is rendered based on the third rendering instruction. Since the rendering channel corresponding to the reused rendering result does not need to be rendered again, the rendering calculation amount is reduced, the rendering efficiency is improved, the frame rate of the target application is improved, and the heat generation and power consumption of the terminal device are reduced.

[0089] In one possible implementation, Figure 5 As shown, S340 (adjusting the original resolution parameter of at least one rendering channel in the first rendering instruction to the preset resolution parameter) may include:

[0090] S510, determining multiple rendering channels in a first rendering instruction and specification parameters of the multiple rendering channels.

[0091] Among them, referring to the description of the rendering channel in the embodiment of the present application, each detail seen in the final rendered frame is calculated through a series of passes, for example, Figure 6 As shown, the rendering channels may include: main scene rendering channel, special effect rendering channel, UI rendering channel, motion blur rendering channel, floodlight rendering channel, etc. The first rendering instruction includes specification parameters of multiple rendering channels, and different rendering channels are responsible for rendering different parts of the image. For example, rendering channel A is responsible for rendering the main scene, rendering channel B is responsible for rendering special effects, and rendering channel C is responsible for rendering UI, etc. The specification parameters include at least one of the following: rendering channel size Size, frame buffering attachment Frame buffering Attachment, and the number of draw call interfaces DrawCall. Based on the specification parameters of the rendering channel, it is possible to distinguish which part of the image the rendering channel is responsible for rendering.

[0092] S520: Adjust the original resolution parameter of the target rendering channel among the multiple rendering channels to a preset resolution parameter.

[0093] Among them, the specification parameters of the target rendering channel meet the preset specification conditions. The preset specification conditions can be set flexibly, taking into account the balance between image quality and improved terminal performance. If too many rendering channels with reduced resolution parameters are used, although the terminal performance is improved, the image quality will be poor, which may be perceived by the user and reduce the user experience. Reducing the rendering channel for rendering the main scene can find a balance between image quality and improved terminal performance. The target rendering channel determined by the preset specification conditions should be able to find a balance between image quality and improved terminal performance.

[0094] Exemplarily, the target rendering channel determined by the preset specification conditions can be the rendering channel for rendering the main scene. In this case, the preset specification conditions are that the Size of the target rendering channel needs to be greater than a certain threshold, the frame buffer attachments need to include three attachments: Color, Depth, and Stencil, and the number of Draw Calls is the largest among multiple rendering channels. Only adjusting the original resolution parameters of the rendering channel for rendering the main scene to the preset resolution parameters can significantly reduce the amount of rendering operations while reducing the rendering quality slightly.

[0095] In another example, the target rendering channel determined by the preset specification conditions can be the rendering channel for rendering the main scene and the associated rendering channel (the associated rendering channel and the rendering channel of the main scene share a depth texture). Only adjusting the original resolution parameters of the rendering channel for rendering the main scene and the associated rendering channel to the preset resolution parameters can greatly reduce the rendering computation amount while slightly reducing the rendering quality.

[0096] In one embodiment, the specification parameters of the rendering channel can also be applied to the generation of the third rendering instruction in S410. The third rendering instruction includes a first sub-instruction and a second sub-instruction, the first sub-instruction is used to indicate that the rendering result of at least one rendering channel of the previous frame image is used as the first rendering result of the image to be rendered, and the second sub-instruction includes the original resolution parameters of the remaining rendering channels except at least one rendering channel in the first rendering instruction. Specifically, the specification parameters of the rendering channel can be used to demarcate which rendering channels are indicated by the first sub-instruction and the second sub-instruction respectively. For example, the rendering channel of the main scene and the remaining rendering channels except the rendering channel of the main scene are determined by the specification parameters of the rendering channel. Then the first sub-instruction can indicate that the rendering result of the rendering channel of the main scene of the previous frame image is used as the first rendering result of the image to be rendered, and the second sub-instruction includes the original resolution parameters of the remaining rendering channels except the rendering channel of the main scene in the first rendering instruction. Rendering based on the third rendering instruction defined in this way does not require rendering the main scene, which greatly reduces the amount of rendering operations.

[0097] In an embodiment of the present application, only the original resolution parameters of the target rendering channel are adjusted to preset resolution parameters. In this way, when rendering is performed based on the adjusted first rendering instruction, the rendering calculation amount can be reduced while relatively ensuring the rendering quality, thereby improving the rendering efficiency to increase the frame rate of the target application and reduce the heat and power consumption of the terminal device.

[0098] In a possible implementation, the field of view transformation information includes displacement transformation information and angle transformation information, such as Figure 7 As shown, S330 (determining the field of view transformation information of the image to be rendered relative to the previous frame image according to the first rendering instruction and the second rendering instruction) may include:

[0099] S710, calculating the difference between the first coordinate and the second coordinate, and outputting displacement transformation information.

[0100] The first rendering instruction includes the first coordinate of the Camera, and the second rendering instruction includes the second coordinate of the Camera. By calculating the difference between the two, the displacement transformation information of the Camera can be obtained.

[0101] S720, calculating a first plane normal vector according to the first field of view matrix, and calculating a second plane normal vector according to the second field of view matrix.

[0102] The first rendering instruction includes a first field of view matrix, and the second rendering instruction includes a second field of view matrix. Based on the field of view matrix, the plane normal vector of the field of view matrix can be calculated. Exemplarily, the first field of view matrix is:

[0103]

[0104] Then the calculated first plane normal vector is (a, b, c), (a, b, c) is calculated by the following function ax+by+cz+d=0, in which:

[0105] a=m 41 +m 31

[0106] b=m 42 +m 32

[0107] c=m 43 +m 33

[0108] d=m 44 +m 34

[0109] S730, calculating the angle between the first plane normal vector and the second plane normal vector, and outputting angle transformation information.

[0110] After determining the first plane normal vector and the second plane normal vector, the angle transformation information can be calculated. For example, the first plane normal vector is The normal vector of the second plane is The angle transformation information θ can be calculated by the following formula:

[0111]

[0112] where θ∈[0°, 180°].

[0113] After determining the field of view change information, the field of view change scene can be determined based on the field of view change information. Exemplarily, the target application is a game application, and the preset dynamic scenes include the character walking, the character running, or the user's perspective rotating in the game. The preset dynamic scenes include static scenes in the game (the character has no major movements, for example, the character meditates, the character stands, the character fishes, etc.). At this time, it can be determined based on the field of view change information which of the above scenes it is. For example, the following scenes can be defined based on the field of view change information:

[0114] Still scene: The displacement transformation information is less than the threshold 1; the angle transformation information is less than the threshold 3.

[0115] Character walking scene: displacement transformation information is greater than threshold 1 and less than threshold 2; angle transformation information is less than threshold 3.

[0116] Character running scene: the displacement transformation information is greater than threshold 2; the angle transformation information is less than threshold 3.

[0117] User perspective rotation scene: The angle change information is greater than the threshold 3.

[0118] In the embodiment of the present application, the field of view transformation information can be accurately calculated, and then based on the field of view transformation information, it can be determined whether the transformation scene of the Camera is a preset dynamic scene or a preset static scene, so as to execute different rendering instruction adjustment strategies based on different scenes.

[0119] In one embodiment, experiments were conducted on various game apps based on the rendering processing method provided in the embodiment of the present application as shown in Table 1. It can be seen that the performance is improved by 10%-20% or the power consumption is reduced by 10%-20% on different game apps.

[0120] Table 1

[0121]

[0122] In the embodiment of the present application, there is no need for code adaptation between the application and the application engine, and it does not rely on any specific hardware platform. Based on the fact that the rendering instruction is transmitted in the form of a data stream, the scene of the field of view change between the previous and next frames is determined, and the resolution of the rendering channel is adjusted in a targeted manner to reduce the target application load and improve the target application frame rate. This allows the target application to reduce the power consumption of the terminal device without the user noticing, and delay the time when the target application is limited by the system frequency and frame, thereby extending the running time of the target application and improving the user experience.

[0123] The above mainly introduces the scheme provided by the embodiment of the present application from the perspective of the execution logic of each step. It is understandable that each node, such as a processor, includes a hardware structure and / or software module corresponding to each function in order to realize the above functions. Those skilled in the art should be easily aware that, in combination with the algorithm steps of each example described in the embodiment disclosed herein, the method of the embodiment of the present application can be implemented in the form of hardware, software, or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of this application.

[0124] The embodiment of the present application can divide the processor or graphics processor into functional modules according to the above method example. For example, each functional module can be divided according to each function, or two or more functions can be integrated into one processing module. The above integrated module can be implemented in the form of hardware or in the form of software functional modules. It should be noted that the division of modules in the embodiment of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation.

[0125] Figure 8 A structural diagram of a rendering processing device 800 is shown. Figure 8 Each module in the device shown has the function of realizing Figure 3-Figure 7 The functions of the corresponding steps in the module can achieve the corresponding technical effects. The corresponding beneficial effects of each module execution step can be referred to Figure 3-Figure 7 The description of the corresponding steps is not repeated here. The functions can be implemented by hardware or by executing corresponding software by hardware. The hardware or software includes one or more modules corresponding to the above functions. The rendering processing device can be a terminal device or a chip or system on chip in the terminal device. For example, the rendering processing device includes:

[0126] The interception module 810 is used to intercept the first rendering instruction sent by the target application.

[0127] The first rendering instruction is used to render the image to be rendered of the target application.

[0128] The acquisition module 820 is used to acquire a second rendering instruction, wherein the second rendering instruction is used to render a previous frame image of the image to be rendered.

[0129] The determination module 830 is used to determine the field of view transformation information of the image to be rendered relative to the previous frame image according to the first rendering instruction and the second rendering instruction.

[0130] The adjustment module 840 is used to adjust the original resolution parameter of at least one rendering channel in the first rendering instruction to a preset resolution parameter when the field of view change information conforms to the preset dynamic scene.

[0131] Among them, the preset resolution parameter is lower than the original resolution parameter.

[0132] The sending module 850 is used to send the adjusted first rendering instruction.

[0133] The adjusted first rendering instruction is used to render the image to be rendered.

[0134] In one embodiment, the device further includes: a generating module 860, which is used to generate a third rendering instruction when the field of view transformation information conforms to a preset static scene, wherein the third rendering instruction includes a first sub-instruction and a second sub-instruction, the first sub-instruction is used to indicate that the rendering result of a part of the rendering channel of the previous frame image is used as the part of the rendering result of the image to be rendered, and the second sub-instruction includes the original resolution parameters of the remaining rendering channels except the part of the rendering channel in the first rendering instruction. The sending module 850 is also used to send the third rendering instruction, and the third rendering instruction is used to render the image to be rendered.

[0135] In one embodiment, the adjustment module 840 is specifically used to: determine multiple rendering channels in the first rendering instruction, and specification parameters of the multiple rendering channels. Adjust the original resolution parameter of the target rendering channel in the multiple rendering channels to the preset resolution parameter. The specification parameter includes at least one of the following: rendering channel size, frame buffer attachment, and number of draw call interfaces, and the specification parameter of the target rendering channel meets the preset specification condition.

[0136] In one embodiment, different sizes of field of view change information correspond to different preset dynamic scenes. The preset resolution parameters under different preset dynamic scenes are different, or the preset resolution parameters under different preset dynamic scenes are the same.

[0137] In one embodiment, the field of view transformation information includes displacement transformation information and angle transformation information. The first rendering instruction includes a first field of view matrix and a first coordinate, and the second rendering instruction includes a second field of view matrix and a second coordinate. The determination module 830 is specifically used to: calculate the difference between the first coordinate and the second coordinate, and output the displacement transformation information. Calculate the first plane normal vector according to the first field of view matrix, and calculate the second plane normal vector according to the second field of view matrix. Calculate the angle between the first plane normal vector and the second plane normal vector, and output the angle transformation information.

[0138] Fig. 9 A structural diagram of a rendering processing device 900 is shown. Fig. 9 Each module in the device shown has the function of realizing Figure 3-Figure 7The functions of the corresponding steps in the module can achieve the corresponding technical effects. The corresponding beneficial effects of each module execution step can be referred to Figure 3-Figure 7 The description of the corresponding steps is not repeated here. The functions can be implemented by hardware or by executing corresponding software by hardware. The hardware or software includes one or more modules corresponding to the above functions. The rendering processing device can be a terminal device or a chip or system on chip in the terminal device. For example, the rendering processing device includes:

[0139] The receiving module 910 is used to receive a third rendering instruction.

[0140] Among them, the third rendering instruction includes a first sub-instruction and a second sub-instruction, the first sub-instruction is used to indicate that the rendering result of at least one rendering channel of the previous frame image of the image to be rendered by the target application is used as the first rendering result of the image to be rendered, and the second sub-instruction includes the original resolution parameters of the remaining rendering channels except at least one rendering channel in the first rendering instruction. The first rendering instruction is an instruction sent by the target application for rendering the image to be rendered.

[0141] The rendering module 920 is used to render the image to be rendered according to the third rendering instruction to obtain a full rendering result of the image to be rendered.

[0142] The full rendering result includes a first rendering result and a second rendering result, and the second rendering result is obtained by rendering the image to be rendered based on the original resolution parameters by the remaining rendering channels.

[0143] The embodiment of the present application also provides a computer-readable storage medium. All or part of the processes in the above method embodiments can be completed by a computer program to instruct the relevant hardware, and the program can be stored in the above computer-readable storage medium. When the program is executed, it can include the processes of the above method embodiments. The computer-readable storage medium can be a terminal device of any of the above embodiments, such as: an internal storage unit including a data sending end and / or a data receiving end, such as a hard disk or memory of a terminal device. The above computer-readable storage medium can also be an external storage device of the above terminal device, such as a plug-in hard disk equipped on the above terminal device, a smart memory card (smart media card, SMC), a secure digital (secure digital, SD) card, a flash card (flash card), etc. Further, the above computer-readable storage medium can also include both an internal storage unit of the above terminal device and an external storage device. The above computer-readable storage medium is used to store the above computer program and other programs and data required by the above terminal device. The above computer-readable storage medium can also be used to temporarily store data that has been output or is to be output.

[0144] The present application also provides a computer instruction. All or part of the process in the above method embodiment can be completed by computer instructions to instruct related hardware (such as computers, processors, network devices, and terminals, etc.). The program can be stored in the above computer-readable storage medium.

[0145] The embodiment of the present application also provides a chip system. The chip system can be composed of a chip, or can include a chip and other discrete devices, without limitation. The chip system includes a processor and a transceiver, and all or part of the processes in the above method embodiment can be completed by the chip system, such as the chip system can be used to implement the functions performed by the processor in the above method embodiment, or to implement the functions performed by the graphics processor in the above method embodiment.

[0146] In one possible design, the chip system further includes a memory, which is used to store program instructions and / or data. When the chip system is running, the processor executes the program instructions stored in the memory so that the chip system performs the functions performed by the processor in the method embodiment or performs the functions performed by the graphics processor in the method embodiment.

[0147] In the embodiments of the present application, the processor may be a general-purpose processor, a digital signal processor, an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component, and may implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the method disclosed in the embodiments of the present application may be directly embodied as being executed by a hardware processor, or may be executed by a combination of hardware and software modules in the processor.

[0148] In the embodiment of the present application, the memory may be a non-volatile memory, such as a hard disk drive (HDD) or a solid-state drive (SSD), etc., or a volatile memory (volatile memory), such as a random-access memory (RAM). The memory is any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory in the embodiment of the present application may also be a circuit or any other device that can implement a storage function, for storing instructions and / or data.

[0149] Unless otherwise specified, the "transmission" (transmit / transmission) appearing in the embodiments of the present application refers to bidirectional transmission, including sending and / or receiving actions. Specifically, the "transmission" in the embodiments of the present application includes the sending of data, the receiving of data, or the sending of data and the receiving of data. In other words, the data transmission here includes uplink and / or downlink data transmission. Data may include channels and / or signals, uplink data transmission is uplink channel and / or uplink signal transmission, and downlink data transmission is downlink channel and / or downlink signal transmission. The "network" and "system" appearing in the embodiments of the present application express the same concept, and the communication system is the communication network.

[0150] Through the description of the above implementation methods, technical personnel in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0151] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0152] The units described as separate components may or may not be physically separated, and the components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0153] In addition, each functional unit in each embodiment of the present application can be integrated into a processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of a software functional unit. If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on such an understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium, including several instructions to enable a device, such as: a single-chip microcomputer, a chip, etc., or a processor (processor) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a U disk, a mobile hard disk, a ROM, a RAM, a disk, or an optical disk.

[0154] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A rendering processing method, characterized in that: include: intercepting a first rendering instruction sent by a target application, wherein the first rendering instruction is used to render an image to be rendered of the target application; Obtaining a second rendering instruction, wherein the second rendering instruction is used to render a previous frame image of the image to be rendered; Determine, according to the first rendering instruction and the second rendering instruction, field of view transformation information of the image to be rendered relative to the previous frame image; When the field of view change information conforms to a preset dynamic scene, adjusting an original resolution parameter of at least one rendering channel in the first rendering instruction to a preset resolution parameter, wherein the preset resolution parameter is lower than the original resolution parameter; Sending the adjusted first rendering instruction; the adjusted first rendering instruction is used to render the image to be rendered.

2. The rendering processing method according to claim 1, characterized in that: The method further comprises: In a case where the field of view change information conforms to a preset static scene, a third rendering instruction is generated, wherein the third rendering instruction includes a first sub-instruction and a second sub-instruction, the first sub-instruction is used to instruct to use a rendering result of at least one rendering channel of the previous frame image as a first rendering result of the image to be rendered, and the second sub-instruction includes original resolution parameters of the remaining rendering channels except the at least one rendering channel in the first rendering instruction; The third rendering instruction is sent, where the third rendering instruction is used to render the image to be rendered.

3. The rendering processing method according to claim 1, characterized in that: Adjusting the original resolution parameter of at least one rendering channel in the first rendering instruction to a preset resolution parameter includes: Determine a plurality of rendering channels in the first rendering instruction, and specification parameters of the plurality of rendering channels; the specification parameters include at least one of the following: a rendering channel size, a frame buffer attachment, and a number of draw call interfaces; Adjusting original resolution parameters of a target rendering channel among the multiple rendering channels to preset resolution parameters, wherein specification parameters of the target rendering channel meet preset specification conditions.

4. The rendering method according to claim 1 or 3, characterized in that: The field of view change information of different sizes corresponds to different preset dynamic scenes; the preset resolution parameters under different preset dynamic scenes are different, or the preset resolution parameters under different preset dynamic scenes are the same.

5. The rendering method according to any one of claims 1 to 4, characterized in that: The field of view transformation information includes displacement transformation information and angle transformation information; the first rendering instruction includes a first field of view matrix and a first coordinate, and the second rendering instruction includes a second field of view matrix and a second coordinate; The determining, according to the first rendering instruction and the second rendering instruction, field of view transformation information of the image to be rendered relative to the previous frame image includes: Calculating the difference between the first coordinate and the second coordinate, and outputting the displacement transformation information; Calculating a first plane normal vector according to the first field of view matrix, and calculating a second plane normal vector according to the second field of view matrix; Calculate the angle between the first plane normal vector and the second plane normal vector, and output the angle transformation information.

6. A rendering processing method, characterized in that: include: Receive a third rendering instruction; wherein the third rendering instruction includes a first sub-instruction and a second sub-instruction, the first sub-instruction is used to instruct to use the rendering result of at least one rendering channel of the previous frame image of the image to be rendered by the target application as the first rendering result of the image to be rendered, the second sub-instruction includes the original resolution parameters of the remaining rendering channels except the at least one rendering channel in the first rendering instruction, and the first rendering instruction is an instruction sent by the target application and used to render the image to be rendered; Render the image to be rendered according to the third rendering instruction to obtain a full rendering result of the image to be rendered, wherein the full rendering result includes the first rendering result and the second rendering result, and the second rendering result is obtained by rendering the image to be rendered by the remaining rendering channels based on the original resolution parameters.

7. A rendering device, characterized in that: The rendering device comprises a processor and a transceiver, and the processor and the transceiver are used to support the rendering device to execute the method according to any one of claims 1-6.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and when the computer instructions are executed, the method according to any one of claims 1 to 6 is executed.

Images