Graphics rendering method, device and storage medium

Abstract

The embodiments of the present application provide a graphics rendering method, device and storage medium, which relate to the field of image processing technology. The method includes a terminal device using a target shader to perform graphics rendering of an application; when the terminal device determines that the application is running in a complex scene, the terminal device obtains a target simplified shader; the complex scene is a scene in which the number of instructions required for the application to run is greater than a first preset threshold, and the target simplified shader occupies less computing resources of the terminal device than the target shader; the terminal device uses the target simplified shader to perform graphics rendering of the application in the complex scene, and since the target simplified shader occupies less computing resources of the terminal device than the target shader, it helps to ensure that the application in the terminal device can still run smoothly when running in a complex scene.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a graphics rendering method, device and storage medium. Background Art

[0002] With the development of computer graphics and computer animation technology, in order to draw application scenes more realistically and finely, application manufacturers will use complex shaders to improve image quality. Shaders are editable programs that can be used to achieve graphics rendering. Graphics rendering is the process by which a computer converts shapes stored in memory into images displayed on the screen.

[0003] When applications in some terminal devices run in complex scenarios, the shader may consume too many computing resources of the shader computing units (e.g., shader cores) in the graphics processing unit (GPU), making it impossible for the GPU to complete rendering within one frame time, causing the terminal device to freeze. Summary of the invention

[0004] The embodiments of the present application provide a graphics rendering method, device and storage medium, which relate to the field of image processing technology and help to ensure that applications in terminal devices can continue to run smoothly when running in complex scenarios.

[0005] In a first aspect, an embodiment of the present application provides a graphics rendering method, the method comprising: a terminal device uses a target shader to perform graphics rendering of an application; when the terminal device determines that the application is running into a complex scene, the terminal device obtains a target simplified shader; the complex scene is a scene in which the number of instructions required for the application to run is greater than a first preset threshold, and the target simplified shader occupies less computing resources of the terminal device than the target shader does; the terminal device uses the target simplified shader to perform graphics rendering of the application in the complex scene.

[0006] In an embodiment of the present application, when the terminal device determines that the application is running in a complex scene, it obtains a target simplified shader, and uses the target simplified shader to render the graphics of the application in the complex scene. Since the target simplified shader occupies less computing resources of the terminal device than the target shader, this helps to ensure that the application in the terminal device can still run smoothly when running in a complex scene.

[0007] In a possible implementation, the terminal device obtains the target simplified shader, including: the terminal device obtains the target simplified shader according to multiple pre-stored correspondences; the multiple correspondences include shader information, device hardware information, and correspondences between simplified shaders; the shader information includes any one of the shader or the shader identifier.

[0008] In another possible implementation, the terminal device obtains the target shader information according to the pre-stored multiple corresponding relationships, including: the terminal device obtains the target shader information and the target simplified shader corresponding to the hardware information of the terminal device in the multiple corresponding relationships. In this way, the terminal device can obtain a simplified shader that helps improve the operating performance of the terminal device when performing graphics rendering according to the shader information and the hardware information of the device.

[0009] In another possible implementation, the above correspondence also includes a load level; the load level is a quantized value corresponding to the load of the terminal device; the above terminal device obtains the information of the target shader according to multiple pre-stored correspondences, including: the terminal device obtains the number of rendering instructions in the complex scene; the terminal device obtains the target load level according to the number of rendering instructions; the terminal device obtains the target simplified shader corresponding to the target shader information, the target load level, and the hardware information of the terminal device in multiple correspondences. In this way, the terminal device can obtain a simplified shader that helps improve the operating performance of the terminal device when rendering graphics in the complex scene according to the shader information, the target load level, and the hardware information of the device.

[0010] In another possible implementation, the method also includes: the terminal device simplifies the target shader according to a first preset rule to obtain multiple simplified shaders; the first preset rule includes at least one of code movement, expression deletion or syntax simplification; the multiple simplified shaders include a target simplified shader; the terminal device collects status information of the terminal device when the terminal device uses multiple simplified shaders to perform graphics rendering of an application in a complex scene; the status information of the terminal device includes the load of the terminal device, the power consumption of the terminal device, the temperature of the terminal device, the rendering time, and the image quality of the picture rendered by each simplified shader; the rendering time is the time required for each simplified shader to perform graphics rendering of the application in a complex scene; the terminal device obtains information about the target shader, hardware information of the terminal device, a target load level, and a correspondence between target simplified shaders based on the status information.

[0011] In this way, the terminal device can analyze in advance the hardware information of the terminal device corresponding to the target shader and the status information of the terminal device when the multiple simplified shaders perform graphics rendering of the application in a complex scene, and obtain a target simplified shader with good performance based on the status information of the terminal device.

[0012] In another possible implementation, the terminal device obtains the target shader information, the hardware information of the terminal device, the target load level, and the correspondence between the target simplified shader based on the status information, including: the terminal device obtains multiple target rendering times from the status information, and the multiple target rendering times are all less than the second preset threshold; the terminal device obtains the target status information from the status information corresponding to the multiple target rendering times; the image quality of the target status information is greater than the third preset threshold; the target status information is the status information of the terminal device when the target simplified shader performs graphics rendering of the application in a complex scene; the terminal device obtains the target load level based on the load of the terminal device in the target status information; the terminal device establishes the correspondence between the target shader information, the hardware information of the terminal device, the target load level, and the target simplified shader.

[0013] In this way, the terminal device can first filter according to the rendering time, and then filter according to the rendered image quality to obtain the target simplified shader, and establish the correspondence between the target shader information, the hardware information of the terminal device, the target load level and the target simplified shader.

[0014] In a possible implementation, the terminal device obtains the target shader information, the hardware information of the terminal device, the target load level, and the correspondence between the target simplified shader according to the status information, including: the terminal device obtains multiple target image quality qualities from the status information, and the multiple target image quality qualities are all greater than a fourth preset threshold; the terminal device obtains the target state information from the status information corresponding to the multiple target image quality qualities; the rendering time of the target state information is less than the fifth preset threshold; the target state information is the state information of the terminal device when the target simplified shader performs graphics rendering of an application in a complex scene; the terminal device obtains the target load level according to the load of the terminal device in the target state information; the terminal device establishes the correspondence between the target shader information, the hardware information of the terminal device, the target load level, and the target simplified shader.

[0015] In this way, the terminal device can first filter according to the quality of the rendered image, and then filter according to the rendering time to obtain the target simplified shader, and establish a correspondence between the target shader information, the hardware information of the terminal device, the target load level, and the target simplified shader.

[0016] In another possible implementation manner, the method further includes: the terminal device receives multiple corresponding relationships sent by other devices.

[0017] In another possible implementation, the method also includes: other devices simplify the target shader according to a first preset rule to obtain multiple simplified shaders; the first preset rule includes at least one of code movement, expression deletion or syntax simplification; the multiple simplified shaders include a target simplified shader; other devices collect status information of other devices when other devices use multiple simplified shaders to perform graphics rendering of applications in complex scenes; the status information of other devices includes the load of other devices, the power consumption of other devices, the temperature of other devices, the rendering time, and the image quality of the pictures rendered by each simplified shader; the rendering time is the time required for each simplified shader to perform graphics rendering of the application in a complex scene; other devices obtain information about the target shader, hardware information of other devices, target load level and the correspondence between the target simplified shaders according to the status information, and the other devices send the obtained correspondence to the terminal device.

[0018] In another possible implementation, the other devices obtain the target shader information, the hardware information of the other devices, the target load level, and the correspondence between the target simplified shader based on the status information, including: the other devices obtain multiple target rendering times from the status information, and the multiple target rendering times are all less than the second preset threshold; the other devices obtain the target status information from the status information corresponding to the multiple target rendering times; the image quality of the target status information is greater than the third preset threshold; the target status information is the status information of other devices when the target simplified shader performs graphics rendering of the application in a complex scene; the other devices obtain the target load level based on the load of the other devices in the target status information; the other devices establish the correspondence between the target shader information, the hardware information of the other devices, the target load level, and the target simplified shader.

[0019] In this way, other devices can first be filtered according to the rendering time, and then filtered according to the rendered image quality to obtain the target simplified shader, and establish a correspondence between the target shader information, the hardware information of other devices, the target load level and the target simplified shader.

[0020] In another possible implementation, the other devices obtain the target shader information, the hardware information of the other devices, the target load level, and the correspondence between the target simplified shader based on the status information, including: the other devices obtain multiple target image quality qualities from the status information, and the multiple target image quality qualities are all greater than the fourth preset threshold; the other devices obtain the target state information from the status information corresponding to the multiple target image quality qualities; the rendering time of the target state information is less than the fifth preset threshold; the target state information is the state information of other devices when the target simplified shader performs graphics rendering of the application in a complex scene; the other devices obtain the target load level based on the load of the other devices in the target state information; the other devices establish the correspondence between the target shader information, the hardware information of the other devices, the target load level, and the target simplified shader.

[0021] In another possible implementation, the terminal device obtains the target simplified shader, including: the terminal device simplifies the target shader according to a first preset rule to obtain the target simplified shader; the first preset rule includes at least one of code movement, expression deletion or syntax simplification.

[0022] In another possible implementation, when the terminal device determines that the application is running in a complex scene, the terminal device obtains a target simplified shader, including: when the terminal device determines that the application is running in a complex scene, and the code complexity of the target shader is greater than a sixth preset threshold, or the number of calls of the target shader is greater than a seventh preset threshold, the terminal device obtains the target simplified shader.

[0023] In a second aspect, an embodiment of the present application provides a terminal device, including a processing module. The processing module is used to use a target shader to perform graphics rendering of an application; when it is determined that the application is running in a complex scene, a target simplified shader is obtained; the complex scene is a scene in which the number of instructions required for the application to run is greater than a first preset threshold, and the target simplified shader occupies less computing resources of the terminal device than the target shader does; and the graphics rendering of the application is performed in the complex scene using the target simplified shader.

[0024] Optionally, the processing module is specifically used to: obtain a target simplified shader based on multiple pre-stored correspondences; the multiple correspondences include shader information, device hardware information, and correspondences between simplified shaders; the shader information includes any one of a shader or a shader identifier.

[0025] Optionally, the processing module is specifically used to: obtain information of the target shader and a target simplified shader corresponding to the hardware information of the terminal device in multiple corresponding relationships.

[0026] Optionally, the above correspondence also includes a load level; the load level is a quantitative value corresponding to the load of the device; the processing module is specifically used to: obtain the number of rendering instructions in a complex scene; obtain the target load level according to the number of rendering instructions; in multiple correspondences, obtain the target shader information, the target load level and the target simplified shader corresponding to the hardware information of the terminal device.

[0027] Optionally, the processing module is also used to: simplify the target shader according to a first preset rule to obtain multiple simplified shaders; the first preset rule includes at least one of code movement, expression deletion or syntax simplification; the multiple simplified shaders include a target simplified shader; collect status information of the terminal device when the terminal device uses multiple simplified shaders to perform graphics rendering of an application in a complex scene; the status information of the terminal device includes the load of the terminal device, the power consumption of the terminal device, the temperature of the terminal device, the rendering time, and the image quality of the picture rendered by each simplified shader; the rendering time is the time required for each simplified shader to perform graphics rendering of the application in a complex scene; obtain information about the target shader, hardware information of the terminal device, target load level, and the correspondence between the target simplified shader based on the status information.

[0028] Optionally, the processing module is specifically used to obtain multiple target rendering times from the status information, and the multiple target rendering times are all less than a second preset threshold; obtain target status information from the status information corresponding to the multiple target rendering times; the image quality of the target status information is greater than a third preset threshold; the target status information is the status information of the terminal device when the target simplified shader performs graphics rendering of the application in a complex scene; obtain the target load level according to the load of the terminal device in the target status information; establish a correspondence between the target shader information, the terminal device hardware information, the target load level, and the target simplified shader.

[0029] Optionally, the processing module is specifically used to: obtain multiple target image quality qualities from the status information, and the multiple target image quality qualities are all greater than a fourth preset threshold; obtain target state information from the status information corresponding to the multiple target image quality qualities; the rendering time of the target state information is less than a fifth preset threshold; the target state information is the state information of the terminal device when the target simplified shader performs graphics rendering of the application in a complex scene; obtain the target load level according to the load of the terminal device in the target state information; establish a correspondence between the target shader information, the hardware information of the terminal device, the target load level, and the target simplified shader.

[0030] Optionally, the terminal device also includes a communication module: used to receive multiple corresponding relationships sent by other devices.

[0031] Optionally, the processing module is specifically used to: simplify the target shader according to a first preset rule to obtain a target simplified shader; the first preset rule includes at least one of code movement, expression deletion or syntax simplification.

[0032] Optionally, when the terminal device determines that the application is running in a complex scene, the processing module is specifically used to: determine that the application is running in a complex scene, and the code complexity of the target shader is greater than a sixth preset threshold, or the number of times the target shader is called is greater than a seventh preset threshold, obtain the target simplified shader.

[0033] In a third aspect, an embodiment of the present application provides a server, including a processing module. The processing module is used to: simplify a target shader according to a first preset rule to obtain multiple simplified shaders; the first preset rule includes at least one of code movement, expression deletion, or syntax simplification; the multiple simplified shaders include a target simplified shader; collect the status information of the terminal device when the terminal device uses multiple simplified shaders to perform graphics rendering of an application in a complex scene; the status information of the terminal device includes the load of the terminal device, the power consumption of the terminal device, the temperature of the terminal device, the rendering time, and the image quality of the picture rendered by each simplified shader; the rendering time is the time required for each simplified shader to perform graphics rendering of the application in a complex scene; obtain the information of the target shader, the hardware information of the terminal device, the target load level, and the correspondence between the target simplified shader according to the status information.

[0034] Optionally, the processing module is specifically used to obtain multiple target rendering times from the status information, and the multiple target rendering times are all less than a second preset threshold; obtain target status information from the status information corresponding to the multiple target rendering times; the image quality of the target status information is greater than a third preset threshold; the target status information is the status information of the terminal device when the target simplified shader performs graphics rendering of the application in a complex scene; obtain the target load level according to the load of the terminal device in the target status information; establish a correspondence between the target shader information, the terminal device hardware information, the target load level, and the target simplified shader.

[0035] Optionally, the processing module is specifically used to: obtain multiple target image quality qualities from the status information, and the multiple target image quality qualities are all greater than a fourth preset threshold; obtain target state information from the status information corresponding to the multiple target image quality qualities; the rendering time of the target state information is less than a fifth preset threshold; the target state information is the state information of the terminal device when the target simplified shader performs graphics rendering of the application in a complex scene; obtain the target load level according to the load of the terminal device in the target state information; establish a correspondence between the target shader information, the hardware information of the terminal device, the target load level, and the target simplified shader.

[0036] Optionally, the terminal device further includes a communication module: configured to send multiple corresponding relationships to the server.

[0037] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which a computer program or instruction is stored. When the computer program or instruction is run on a computer, the computer executes the graphics rendering method described in the first aspect or any possible implementation of the first aspect.

[0038] In a fifth aspect, an embodiment of the present application provides a computer program product including a computer program. When the computer program runs on a computer, the computer executes the graphics rendering method described in the first aspect or any possible implementation of the first aspect.

[0039] In a sixth aspect, the present application provides a chip or a chip system, the chip or chip system comprising at least one processor and a communication interface, the communication interface and the at least one processor are interconnected by a line, and the at least one processor is used to run a computer program or instruction to execute the graphics rendering method described in the first aspect or any possible implementation of the first aspect. The communication interface in the chip can be an input / output interface, a pin or a circuit, etc.

[0040] In a possible implementation, the chip or chip system described above in the present application further includes at least one memory, in which instructions are stored. The memory may be a storage unit inside the chip, such as a register, a cache, etc., or a storage unit of the chip (e.g., a read-only memory, a random access memory, etc.).

[0041] It should be understood that the second to sixth aspects of the present application correspond to the technical solutions of the first aspect of the present application, and the beneficial effects achieved by each aspect and the corresponding feasible implementation methods are similar and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 A schematic diagram of the corresponding relationship between the frame rate and the running time applicable to the embodiment of the present application;

[0043] Figure 2 A schematic diagram of the structure of a simplified platform applicable to the graphics rendering method provided in the embodiment of the present application;

[0044] Figure 3 A schematic diagram of the structure of a graphics rendering system applicable to the graphics rendering method provided in an embodiment of the present application;

[0045] Figure 4 A schematic diagram of the structure of an electronic device to which the graphics rendering method provided in an embodiment of the present application is applicable;

[0046] Figure 5 A software structure block diagram of an electronic device to which the embodiments of the present application are applicable;

[0047] Figure 6 A flowchart of a learning process of multiple target correspondence relationships in a graphics rendering method provided in an embodiment of the present application;

[0048] Figure 7 A schematic diagram of a scenario of an application program to which an embodiment of the present application is applicable;

[0049] Fig. 8A A schematic diagram of a shader and its corresponding simplified shader provided in an embodiment of the present application;

[0050] Figure 8B A flowchart of another method for obtaining a simplified shader provided in an embodiment of the present application;

[0051] Fig. 9 A flowchart of a learning process for describing a plurality of target correspondence relationships from another perspective provided in an embodiment of the present application;

[0052] Fig.10 A flowchart of using multiple target correspondences to simplify shaders during the running of an application program in a graphics rendering method provided in an embodiment of the present application;

[0053] Fig.11 A flowchart of another graphics rendering method provided in an embodiment of the present application;

[0054] Fig.12 A flowchart of another graphics rendering method provided in an embodiment of the present application;

[0055] Fig.13 A schematic diagram of the structure of a terminal device provided in an embodiment of the present application;

[0056] Fig.14 A schematic diagram of the structure of a server provided in an embodiment of the present application;

[0057] Fig.15 A schematic diagram of the structure of a chip provided in an embodiment of the present application. DETAILED DESCRIPTION

[0058] In order to clearly describe the technical solutions of the embodiments of the present application, some terms and technologies involved in the embodiments of the present application are briefly introduced below:

[0059] 1) Frames and frame rate

[0060] A frame is the smallest unit of a single picture in the interface display. A frame can be understood as a still picture. Displaying multiple connected frames in rapid succession can create the illusion of motion.

[0061] Frame rate refers to the number of frames that refresh an image in 1 second, which can also be understood as the number of times the GPU in the terminal device refreshes the screen per second. A high frame rate can produce smoother and more realistic animations. The more frames per second, the smoother the displayed action will be.

[0062] It should be noted that before the interface displays a frame, it usually needs to go through processes such as drawing, rendering, and synthesis.

[0063] 2) Frame drawing

[0064] Frame drawing refers to the picture drawing of the display interface. The display interface can be composed of one or more views, each view can be drawn by the visual control of the view system, each view is composed of subviews, and a subview corresponds to a small widget in the view, for example, one of the subviews corresponds to a symbol in the picture view.

[0065] 3) Frame rendering (also known as graphics rendering)

[0066] Frame rendering is to color the drawn view or add 3D effects, etc. For example, 3D effects can be lighting effects, shadow effects, and texture effects.

[0067] 4) Shaders

[0068] A shader is an editable program that can run on a GPU to achieve graphics rendering. Shaders include vertex shaders, pixel shaders, fragment shaders, geometry shaders, and compute shaders.

[0069] 5) Hardware information and scene complexity

[0070] In the embodiment of the present application, the hardware information may include at least one of the chip type and chip performance such as central processing unit (CPU) frequency or GPU frequency.

[0071] The complexity of a scene can be represented by the number of draw calls in the scene. Draw calls are commands that the CPU uses to call the underlying graphics drawing interface. Draw calls are used by terminal devices to perform tasks such as detecting rendering status, submitting rendering data, and submitting rendering status.

[0072] 6) Other terms

[0073] In the embodiments of the present application, words such as "first" and "second" are used to distinguish the same or similar items with substantially the same functions and effects. For example, the first chip and the second chip are only used to distinguish different chips, and their order is not limited. Those skilled in the art can understand that words such as "first" and "second" do not limit the quantity and execution order, and words such as "first" and "second" do not necessarily limit them to be different.

[0074] It should be noted that in the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0075] In the embodiments of the present application, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, ab, a--c, bc, or abc, where a, b, c can be single or multiple.

[0076] When users use terminal devices to run applications, when the application runs to complex scenes, the terminal device may experience freezes (also known as frame drops). Figure 1 The figure shows the corresponding relationship between the frame rate of the application and the running time of the application when the application is running in the terminal device. Figure 1 The horizontal axis in the middle is the running time of the application, and the vertical axis is the frame rate when the application is running. The frame rate of the application is set to 60fps. Figure 1 The application has a frame rate of only 31.34fps when it runs to 4 minutes and 7 seconds. At this time, the terminal device freezes when the application is in the scene corresponding to 4 minutes and 7 seconds. Among them, a complex scene can refer to a scene in which the number of rendering instructions in the scene is greater than the first preset threshold. The rendering instruction is a call command from the CPU to the underlying graphics drawing interface. The rendering instruction is used by the terminal device to perform tasks such as detecting the rendering status, submitting rendering data, and submitting the rendering status.

[0077] This is because, when the application runs into a complex scene, the shader consumes a large amount of work from the computing units in the GPU, possibly reaching more than 90%. When the shader consumes more than 90% of the workload of the shader computing units in the GPU, the rendering time of a frame of image is too long, resulting in the GPU being unable to complete the rendering within one frame time, causing the application to freeze.

[0078] In addition, factors affecting rendering time may also include: scenes, shaders used in applications, and hardware information of the terminal device on which the application runs. Scenes may include simple scenes and complex scenes. A simple scene may refer to a scene in which the number of rendering instructions in the scene is less than or equal to a first preset threshold. The hardware information of the terminal device may include: the type of chip used by the terminal device or the performance of the chip in the terminal device, such as at least one of the central processing unit (CPU) frequency or GPU frequency.

[0079] When the application is running on a terminal device, the hardware information of the terminal device is fixed, and the scene is not suitable for change. Therefore, the shader used to render the scene in the terminal device is reasonably simplified to reduce the load on the GPU, which helps to ensure that the application in the terminal device can still run smoothly when running in complex scenes.

[0080] Based on this, an embodiment of the present application provides a graphics rendering method, which can simplify the shader into a simplified shader when the terminal device runs an application. Because the simplified shader occupies less workload of the computing unit in the GPU when running, it can effectively improve the terminal device lag caused by the shader occupying the GPU.

[0081] The terminal device may simplify the shader into a simplified shader, which may include two implementations:

[0082] Implementation 1: When the shader is running, the terminal device can simplify the shader in real time based on certain rules to obtain a simplified shader.

[0083] Implementation 2: You can first learn the simplified shaders corresponding to the shaders in different applications in terminal devices with different hardware information, and obtain multiple target correspondences, which include the relationship between shader information, simplified shader information, the hardware information of the terminal device, and the scene complexity of the scene. In this way, when the application is subsequently run in the terminal device, the terminal device can obtain the target simplified shader information in the target correspondence based on the scene complexity of the scene in the application, the shader information of the shader used by the application, and its own hardware information, and use the simplified shader to run the application. Because the simplified shader occupies less workload on the computing unit in the GPU when running, it can effectively improve the terminal device freeze caused by the shader occupying the GPU.

[0084] For example, for implementation 2, the graphics rendering method provided in the embodiment of the present application can be applied to a simplified platform. The simplified platform can be used to provide a unified shader simplification function for different applications and terminal devices with different hardware information. Figure 2 The figure shows a schematic diagram of the structure of a simplified platform applicable to the graphics rendering method provided in the embodiment of the present application. The simplified platform may include a shader simplification module and a shader replacement module.

[0085] Among them, the shader simplification module is used to obtain the above-mentioned multiple target correspondences. The shader replacement module is used for the terminal device to replace the shader with a simplified shader during the application program running process. It is understandable that the shader simplification module and the shader replacement module can run on different devices, and can also realize their respective functions at different time periods, which is not limited in the embodiments of the present application.

[0086] For example, Figure 2 As shown, the shader simplification module may include a shader simplification unit, an application and hardware information collection unit, and a shader screening unit.

[0087] Among them, the shader simplification unit is used to: collect shaders used during the operation of the application (for example: vertex shader and / or fragment shader, etc.), and simplify the collected shaders according to a preset first preset rule to obtain multiple simplified shaders, and store the multiple simplified shaders.

[0088] The application and hardware information acquisition unit is used to: collect scene data (also known as status information) during the operation of the application (such as the simplified application mentioned above) and collect hardware information, such as at least one of the CPU frequency and / or GPU frequency of the terminal device.

[0089] The shader screening unit is used to screen the simplified shaders according to the collected scene data and hardware information to obtain multiple target correspondences. For example, the rendering time (also known as rendering duration) in the scene data is grouped from high to low, and a simplified shader whose image quality of the rendered image is greater than a second preset threshold is selected in each grouping category. For another example, the image quality of the rendered image in the scene data is segmented, and a simplified shader whose rendering duration is less than a third preset threshold is selected in each segmentation category.

[0090] Exemplarily, the shader replacement module may include: a shader list management unit, an instruction interception unit, a shader comparison unit, a scene recognition unit, a decision selection unit and a shader replacement unit.

[0091] The shader list (ie, the above-mentioned multiple target correspondence relationships) management unit is used to organize and store the above-mentioned multiple target correspondence relationships according to a preset data structure, wherein the preset data structure may be preset in the code.

[0092] It is understandable that all or part of the functions in the shader list management unit can be integrated into the shader screening unit. Alternatively, all or part of the functions in the shader list management unit can be integrated into the decision selection unit. The embodiments of the present application are not limited to this. Exemplarily, based on the scene complexity, scene data, and hardware information, the function of obtaining the shader list can be integrated into the shader screening unit.

[0093] The instruction interception unit is used to intercept the calling instructions of the shader in the application.

[0094] The shader comparison unit is used to search the shader list for the shader whose call instruction is intercepted. If the search fails (i.e. the intercepted shader does not exist in the shader list), the shader in the shader call instruction is not replaced, and the original process in the application is continued to be executed to the GPU for the next step of rendering.

[0095] The scene recognition unit is used to: if the shader search is successful (ie, the intercepted shader exists in the shader list), then identify the scene complexity. For example, identify the scene complexity of the scene corresponding to the shader called by the intercepted call instruction.

[0096] The decision selection unit is used to obtain target simplified shader information in multiple target correspondences based on the scene complexity of the scene in the application, the shader information used by the application, and the hardware information of the terminal device.

[0097] The shader replacement unit is used to replace the shader in the intercepted instruction, compile the replaced shader, and then send it to the GPU for rendering.

[0098] Figure 2 The term application (APP) is used to represent an application running in a terminal device.

[0099] It is understandable that the functions of the simplified platform in the embodiment of the present application can be implemented by a terminal device, or by a graphics rendering system composed of a terminal device and a server. Figure 3 The figure is a schematic diagram of the structure of a graphics rendering system applicable to the graphics rendering method provided in the embodiment of the present application. Figure 3 The terminal device 301 and the server 302 are included. The terminal device 301 may be a smart phone, a desktop computer, a laptop computer, a tablet computer, a virtual reality (VR) device, an augmented reality (AR) device, or a smart wearable device. The server 302 may be a server, a server cluster, or a cloud server. This embodiment of the application does not limit this.

[0100] Figure 3 The terminal device 301 can be used to implement the above Figure 2 The functions of the remaining units are implemented by the server 302. For example, the terminal device 301 can be used to implement the above Figure 2 In the example of FIG. 3 , the server 302 can be used to implement the function of the shader replacement module in the example, and the server 302 can be used to implement the function of the shader simplification module. Alternatively, the terminal device 301 can be used to implement the function of the instruction interception unit, the function of the scene recognition unit, and the function of the shader replacement unit. The server 302 can be used to implement the function of the shader simplification module, the function of the shader list management unit, the function of the shader comparison unit, and the function of the decision selection unit, etc.

[0101] The functions of the terminal device 301 or the server 302 can be Figure 4 The electronic device shown is implemented. Figure 4 FIG. 1 is a schematic diagram of the structure of an electronic device to which the graphics rendering method provided in an embodiment of the present application is applicable. Figure 4 The electronic device 40 shown may include: a processor 401, a memory 402, a display screen 403, a communication module 404, etc.

[0102] It is to be understood that the structure illustrated in this embodiment does not constitute a specific limitation on the electronic device 40. In other embodiments of the present application, the electronic device 40 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0103] The processor 401 may include one or more processing units, for example, the processor 401 may include an application processor (AP), a modem processor, a GPU, an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, a display processing unit (DPU), and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0104] In some embodiments, the electronic device 40 may also include one or more processors 401. Among them, the processor may be the nerve center and command center of the electronic device 40. The processor may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions. A memory may also be provided in the processor 401 for storing instructions and data. In some embodiments, the memory in the processor 401 is a cache memory. The memory may store instructions or data used or recycled by the processor 401. If the processor 401 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 401, and thus improves the efficiency of the electronic device 40.

[0105] The communication module 404 of the electronic device 40 can use any transceiver or other device to communicate with other devices or communication networks, such as wide area networks (WAN), local area networks (LAN), etc.

[0106] The electronic device 40 can realize the display function through the GPU, the display screen 403, and the application processor. The application processor may include an NPU and a DPU. The GPU is a microprocessor for image processing, which connects the display screen 403 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 401 may include one or more GPUs, which execute instructions to generate or change display information. The NPU is a neural network (NN) computing processor, which quickly processes input information by drawing on the biological neural network structure, such as the transmission mode between neurons in the human brain, and can also continuously self-learn. The NPU can realize applications such as intelligent cognition of the electronic device 40, such as image recognition, face recognition, voice recognition, text understanding, etc. The DPU is also called a display subsystem (DSS). The DPU is used to adjust the color of the display screen 403. The DPU can adjust the color of the display screen 403 through a three-dimensional lookup table (3D LUT). The DPU can also perform processing such as scaling, noise reduction, contrast enhancement, backlight brightness management, HDR processing, and display screen parameter Gamma adjustment on the picture.

[0107] The display screen 403 is used to display images, videos, etc. The display screen 403 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device 40 may include 1 or N display screens 403, where N is a positive integer greater than 1.

[0108] The memory 402 may be used to store one or more computer programs, which include instructions. The processor 401 may enable the electronic device 40 to perform various functional applications and data processing by running the above instructions stored in the memory 402. The memory 402 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system; the program storage area may also store one or more applications (such as a gallery, contacts, etc.). The data storage area may store data (such as photos, contacts, etc.) created during the use of the electronic device 40. In addition, the memory 402 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc. In some embodiments, the processor 401 may enable the electronic device 40 to perform various functional applications and data processing by running instructions stored in the memory 402, and / or instructions stored in a memory provided in the processor 401.

[0109] The memory 402 is used to store multiple target correspondences in the embodiment of the present application.

[0110] The display screen 403 is used to display images, videos, etc. The display screen 403 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device 40 may include 1 or N display screens 403, where N is a positive integer greater than 1.

[0111] The software system of the electronic device 40 may adopt a layered architecture, an event-driven architecture, a micro-core architecture, a micro-service architecture, or a cloud architecture. The embodiment of the present application takes the Android system of the layered architecture as an example to exemplify the software structure of the electronic device 40. Figure 5A software structure block diagram of an electronic device applicable to the embodiment of the present application. The layered architecture divides the software system of the electronic device 40 into several layers, each layer has a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system can be divided into five layers, namely, the application layer (applications), the application framework layer (application framework), the Android runtime (Android runtime) and the system library, the hardware abstraction layer (hardware abstraction layer, HAL) and the kernel layer (kernel).

[0112] The application layer can include a series of application packages. The application layer runs applications by calling the application programming interface (API) provided by the application framework layer. Figure 5 As shown, the application package may include applications such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, game, etc.

[0113] The application framework layer provides API and programming framework for the applications in the application layer. The application framework layer includes some predefined functions. Figure 5 As shown, the application framework layer may include a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager, a database, and the like.

[0114] The window manager is used to manage window programs. The window manager can obtain the size of the display screen, determine whether there is a status bar, lock the screen, capture the screen, etc. The content provider is used to store and obtain data and make the data accessible to applications. The data may include video, images, audio, calls made and received, browsing history and bookmarks, phone books, etc. The view system includes visual controls, such as controls for displaying text, controls for displaying pictures, etc. The view system can be used to build applications. The display interface can be composed of one or more views. For example, a display interface including a text notification icon may include a view for displaying text and a view for displaying pictures. The phone manager is used to provide communication functions for the electronic device 40. For example, the management of call status (including connected, hung up, etc.). The resource manager provides various resources for applications, such as localized strings, icons, pictures, layout files, video files, etc. The notification manager enables applications to display notification information in the status bar, which can be used to convey notification-type messages and can disappear automatically after a short stay without user interaction. For example, the notification manager is used to notify download completion, message reminders, etc. The notification manager can also provide notifications that appear in the system top status bar in the form of icons or scroll bar text, such as notifications of applications running in the background, and can also provide notifications that appear on the screen in the form of dialog windows. For example, a text message is prompted in the status bar, a prompt sound is emitted, the electronic device 40 vibrates, an indicator light flashes, etc. The database can be used to organize, store and manage data according to the data structure.

[0115] The Android runtime consists of a core library and a virtual machine. The Android runtime is responsible for scheduling and management of the Android system. The core library consists of two parts: one is the function that the Java language needs to call, and the other is the Android core library. The application layer and the application framework layer run in the virtual machine. The virtual machine executes the Java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform object life cycle management, stack management, thread management, security and exception management, and garbage collection.

[0116] The system library may include multiple functional modules, such as a surface manager, media libraries, a 3D graphics processing library, such as an open graphics library (OpenGL), a 2D graphics engine (such as SGL), etc.

[0117] The surface manager is used to manage the display subsystem and provide fusion of 2D and 3D layers for multiple applications. The media library supports playback and recording of multiple common audio and video formats, as well as static image files. The media library can support multiple audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc. The 3D graphics processing library is used to implement 3D graphics drawing, graphics rendering, synthesis and layer processing. The 2D graphics engine is a drawing engine for 2D drawing.

[0118] The hardware abstraction layer may include multiple library modules, such as sensor library modules. The Android system may load corresponding library modules for the device hardware, thereby enabling the application framework layer to access the device hardware. The device hardware may include, for example, a temperature sensor in the electronic device 40.

[0119] The kernel layer is a layer between hardware and software. The kernel layer is used to drive the hardware to make the hardware work. The kernel layer at least includes display drivers, etc., which are not limited in the embodiments of the present application.

[0120] A graphics rendering method provided by an embodiment of the present application includes a learning process of multiple target correspondences and a process of using the multiple target correspondences to simplify shaders during the running of an application.

[0121] Figure 6 A flowchart of a learning process of multiple target correspondence relationships in a graphics rendering method provided in an embodiment of the present application is shown as follows: Figure 6 As shown, the following steps may be included:

[0122] S600: The terminal device collects the shaders used in the running process of the application.

[0123] In a possible implementation, the terminal device determines a shader call instruction based on a preset keyword and intercepts the shader call instruction, wherein the shader call instruction includes at least one of a shader identifier or a shader. When the shader is not included in the call instruction, the terminal device obtains the shader based on the shader identifier. The preset keyword may be specified by code.

[0124] Exemplarily, the preset keyword is ShaderSource, and when the terminal device determines that the instruction includes ShaderSource, the shader call instruction is obtained. The shader call instruction obtained by the terminal device is glShaderSource(vertexShader, 1, &vertexShaderSource, NULL). Among them, vertexShaderSource is a shader identifier. The terminal device obtains the shader indicated by the identifier according to vertexShaderSource. The shader indicated by vertexShaderSource is used for Figure 7 The target object in the target scene is shown as rendered in color. The target object is Figure 7 In the scene shown is soldier 701, where soldier 701 is drawn by a visual control of a viewing system in a terminal device.

[0125] S601: The terminal device obtains a heavy-load shader from the collected shaders according to at least one of the code complexity of the shader or the number of times the shader is called.

[0126] Code complexity includes time complexity or space complexity. Time complexity is the trend of code execution time as the data size grows, also known as asymptotic time complexity. Space complexity is asymptotic space complexity, which indicates the growth relationship between the algorithm's storage space and data size.

[0127] The number of shader calls is the number of times the shader is called in the scene rendered by the shader. The more times the shader is called, the greater the load of the shader.

[0128] In a possible implementation, the terminal device determines a shader whose code complexity is greater than a fourth preset threshold as a heavy-load shader. Alternatively, the terminal device determines a shader whose number of calls is greater than a fifth preset threshold as a heavy-load shader, or the terminal device determines a shader whose code complexity is greater than the fourth preset threshold and whose number of calls is greater than the fifth preset threshold as a heavy-load shader.

[0129] It is understandable that the greater the code complexity of the shader, the longer the rendering time required to call the shader for graphics rendering, so the terminal device can obtain the rendering time of the shader. The terminal device can use the shaders with a preset weight as heavy-load shaders in the order of the rendering time.

[0130] Exemplarily, the terminal device sorts the acquired rendering times of the shaders in descending order, and uses the shaders corresponding to the top 80% of the rendering times as heavy-load shaders.

[0131] S602: The terminal device simplifies the heavy-load shader according to a first preset rule to obtain a plurality of simplified shaders, wherein the first preset rule includes at least one of code movement, expression deletion, or syntax simplification.

[0132] Code movement is to move the fragment in the fragment shader to the corresponding vertex shader, and use the target execution result in the fragment shader. The target execution result is the execution result of the vertex shader executing the fragment. The vertex shader and fragment shader that render the same object correspond to each other.

[0133] Expression deletion, in the embodiment of the present application, expressions are also called functions. Functions in shaders are all used to produce special effects or complete calculations. Therefore, functions will have a certain impact on the final rendering result. These impacts may be large or small. The terminal device can obtain the impact value of the function and delete the function whose impact value is less than the preset impact value. Among them, the preset impact value can be set by code based on experience.

[0134] Syntax simplification is to simplify the code in the shader to reduce the amount of code. For example, the packaged package is used to replace the code that implements the same function.

[0135] In a possible implementation, the terminal device selects at least one variable or expression from the first shader as a moving target, extracts the statement related to calculating the moving target in the first shader as a target fragment, and moves the target fragment to the second shader, the first shader is a fragment shader, and the second shader is a vertex shader corresponding to the first shader. The terminal device deletes the target fragment in the first shader to obtain a simplified shader of the first shader. The simplified shader uses the execution result of the target fragment in the vertex shader.

[0136] In an example, Fig. 8A The terminal device shown moves the code of shader 1 to obtain simplified shader No. 1, and the identification of simplified shader No. 1 is 1-1. The terminal device simplifies the syntax and deletes the expression of shader 1 to obtain simplified shader No. 2, and the identification of simplified shader No. 2 is 1-2. The terminal device moves the code, deletes the expression and simplifies the syntax of shader 1 to obtain simplified shader No. 3, and the identification of simplified shader No. 3 is 1-3.

[0137] In another example, Figure 8B As shown, the terminal device extracts multiple applications (such as: Figure 8BThe terminal device extracts shaders in the application programs APP1 to APP3, etc., and identifies heavy-load shaders in the extracted shaders. The terminal device simplifies the heavy-load shaders according to the first preset rule to obtain a set of simplified shader models. The simplified model is used to implement shader extraction, shader identification, and shader simplification.

[0138] For example, a simplified model is extracted as Figure 8B The shaders in the application APP1 to the application APP3. For possible implementations, please refer to the description in S600 above and will not be repeated here. The simplified model performs shader identification on the extracted shaders to obtain the heavy-load shaders in the application APP1 including shader 1, shader 2 and shader 3. The heavy-load shaders in the application APP2 include shader 4, shader 5 and shader 6. The heavy-load shaders in the application APP3 include shader 7, shader 8 and shader 9. For possible implementations, please refer to the description in S601 above and will not be repeated here. The simplified model simplifies shader 1 according to different first preset rules to obtain three simplified shaders identified as 1-1 to 1-3. The simplification of shader 2 to shader 9 by the simplified model is similar to this and will not be repeated here. For possible implementations, please refer to the description in S602 above and will not be repeated here.

[0139] S603: The terminal device replaces the heavy-load shader in the application with different simplified shaders obtained by simplifying the heavy-load shader, thereby obtaining a plurality of simplified application programs, wherein the different simplified shaders are shaders obtained by simplifying the heavy-load shader according to different first preset rules.

[0140] based on Figure 8B In the example, the terminal device replaces shader 1 in application APP1 with a simplified shader identified as 1-1 to obtain a first simplified application, the terminal device replaces shader 1 in application APP1 with a simplified shader identified as 1-2 to obtain a second simplified application, and the terminal device replaces shader 1 in application APP1 with a simplified shader identified as 1-3 to obtain a third simplified application.

[0141] S604: The terminal device collects state information of the plurality of simplified applications when they are running on the target device and hardware information of the target device. The state information includes information such as the rendering time required for the simplified shader, the quality of the rendered image, the load of the target device, the power consumption of the target device, or the temperature of the target device. The hardware information includes at least one of the chip type and chip performance such as the central processing unit (CPU) frequency or the GPU frequency.

[0142] S605: The terminal device filters the simplified shader according to the collected status information and hardware information to obtain multiple target correspondences. The target correspondence includes the correspondence between the shader information (i.e., the heavy-load shader information), the target hardware information, the target load level, and the simplified shader information. Among them, the target hardware information is the hardware information included in the target device. The target load level is the load level corresponding to the load of the target device when the simplified application is run on the target device.

[0143] In a possible implementation, the terminal device groups the rendering time in the collected status information according to the collected status information and the hardware information of the terminal device, selects the target image quality corresponding to the group in the group, and determines multiple first simplified shaders. The terminal device obtains multiple target correspondences according to the heavy-load shader information, multiple first simplified shader information, the hardware information of the terminal device, and the load level of the terminal device. Among them, the target image quality is greater than the second preset threshold, and the first simplified shader is a simplified shader in a simplified application corresponding to the target image quality. The load level of the terminal device can be obtained according to the load of the terminal device in the status information.

[0144] Exemplarily, when the load of the terminal device is greater than the sixth preset threshold, the terminal device may quantify the load of the terminal device, and obtain a load level corresponding to the load of the terminal device as a fixed value. The fixed value is the quantized value corresponding to the load of the terminal device. For example, a fixed value of 1 is used to represent a high load. A fixed value of 0 is used to represent a low load.

[0145] In another possible implementation, the terminal device groups the image quality in the collected status information according to the collected status information and the hardware information of the terminal device, selects the target rendering time corresponding to the group in each group, and determines the second simplified shader. The terminal device obtains multiple target correspondences according to the heavy-load shader information, multiple second simplified shader information, the hardware information of the terminal device, and the load level of the terminal device. Among them, the target rendering time is less than the third preset threshold, and the target simplified shader is a simplified shader in the simplified application corresponding to the target rendering time.

[0146] Since different applications use very different shader files, and different application vendors write and generate shaders in different ways, when an application runs on a terminal device, it is not possible for every application to run smoothly and stably. Terminal devices use a wide variety of chips, and the GPU performance of chips varies greatly. When applications run on different mobile phone chips, the same shader file takes different rendering times. For different shaders in different applications, the above method can be used to obtain a simplified shader corresponding to the shader when the shader corresponds to different hardware information and load levels, so as to adapt to different applications and terminal devices with different hardware information.

[0147] It is understandable that the above two possible implementations can be combined to obtain two different types of target correspondences, and the two different types of target correspondences are classified and stored in the terminal device. The terminal device can respond to the configuration instruction to validate a specific type of target correspondence. The configuration instruction can be an operation instruction of the user received by the terminal device.

[0148] Fig. 9 FIG. 1 is a flow chart of a learning process for describing a plurality of target correspondence relationships from another perspective provided in an embodiment of the present application. Fig. 9 The heavy-load shaders in the application APP1 include shader 1, shader 2 and shader 3, wherein the shader variants of shader 1 include shaders numbered 1-1 to 1-3, the shader variants of shader 2 include shaders numbered 2-1 to 2-3, and the shader variants of shader 3 include shaders numbered 3-1 and 3-2.

[0149] The terminal device replaces each variant (i.e., shader variant) in the application, that is, replaces shader 1 in the application with its variant shader. A plurality of replaced applications are obtained. This step corresponds to the description in S603 in the above embodiment and will not be repeated here. The replaced application is run on a device including different hardware information (such as different chips), and the terminal device records the runtime state of the application, and records the image quality, rendering time, CPU & GPU load, device temperature and other information of the rendered image. This step corresponds to the description in S604 in the above embodiment and will not be repeated here. The terminal device divides the image quality of the rendered image, selects simplified models with different image quality qualities, and obtains a simplified list of shaders. This step corresponds to the description in S901 in the above embodiment and will not be repeated here.

[0150] Fig.10 A flowchart of a graphics rendering method using multiple target correspondences to simplify the shader process during the running of an application program is provided in an embodiment of the present application, such as Fig.10 As shown, the following steps may be included:

[0151] S1000: The terminal device obtains a call instruction of a source shader. The source shader is used to implement rendering of a target object. The target object is an object in a target scene, and the target scene is a scene in an application program running on the terminal device. The call instruction of the source shader includes source shader information. The source shader information may include at least one of an identifier of the source shader or a source shader.

[0152] In a possible implementation, the terminal device determines the calling instruction of the source shader according to a preset keyword and intercepts the calling instruction of the source shader. The preset keyword may be specified by the code.

[0153] Exemplarily, the preset keyword is ShaderSource, and when the terminal device determines that the instruction includes ShaderSource, the source shader call instruction is obtained. The source shader information included in the source shader call instruction obtained by the terminal device is SShader. The shader represented by SShader is used for Figure 7 The target object rendering color in the target scene shown is Figure 7 In the scene shown is soldier 701, where soldier 701 is drawn by a visual control of a viewing system in a terminal device.

[0154] S1001: The terminal device determines that a plurality of pre-stored target correspondences include source shader information.

[0155] In the embodiment of the present application, the multiple target correspondence relationships are the multiple target correspondence relationships obtained in the above embodiment and will not be repeated here.

[0156] In a possible implementation, the terminal device traverses the shader information in the pre-stored multiple target correspondences and determines that the multiple target correspondences include the source shader information.

[0157] Based on the example in S1000, the terminal device determines that the multiple target correspondences include SShader.

[0158] In another possible implementation, the terminal device traverses the shader information in a specific type of target correspondence and determines that the specific type of target correspondence includes source shader information. The specific type of target correspondence is a target correspondence that takes effect on the terminal device according to a configuration instruction.

[0159] S1002: The terminal device obtains its own hardware information and the scene complexity of the target scene.

[0160] In a possible implementation, the terminal device obtains its hardware information and calculates the number of rendering instructions in the target scene. The terminal device determines the scene complexity of the target scene based on the number of rendering instructions. The rendering instruction is a call command from the CPU to the underlying graphics drawing interface. The rendering instruction is used by the terminal device to perform tasks such as detecting rendering status, submitting rendering data, and submitting rendering status.

[0161] Exemplarily, the hardware information acquired by the terminal device includes that the CPU main frequency is 3.0 GHz and the GPU frequency is 850 MHz. If the terminal device calculates that the number of rendering instructions in the target scene is 600, the first preset threshold is 300, and the number of rendering instructions calculated by the terminal device is greater than the first preset threshold, then the terminal device determines the scene complexity of the target scene as a fixed value such as 1. 1 is used to represent high scene complexity.

[0162] It should be noted that the scene complexity corresponds to the load level in the above embodiment. If the hardware information of the terminal devices is the same, then the higher the scene complexity, the higher the load of the terminal device when the terminal device runs to the scene.

[0163] S1003: The terminal device obtains source shader information, hardware information of the terminal device, and target simplified shader information corresponding to the scene complexity of the target scene from a target correspondence including source shader information.

[0164] In the embodiment of the present application, the target correspondence relationship including the source shader information is the target correspondence relationship including the source shader information determined in S1001.

[0165] In a possible implementation, the terminal device obtains source shader information, hardware information of the terminal device, and target simplified shader information corresponding to the scene complexity of the target scene from a target correspondence including source shader information.

[0166] Exemplarily, the correspondence between multiple targets is shown in Table 1 below:

[0167] Table 1

[0168]

[0169] In Table 1, the simplified shader information corresponding to the shader information SShader, the hardware information GPU frequency is 850MHz, the CPU main frequency is 3.0GHz, and the scene complexity 1 is shader number 1. The simplified shader information corresponding to the shader information SShader, the hardware information GPU frequency is 2330MHZ, the CPU main frequency is 3.6GHz, and the scene complexity 1 is shader number 2. The shader information VShader, the hardware information GPU frequency is 1000MHz, the CPU main frequency is 3.8GHz, and the simplified shader information corresponding to the scene complexity 0 is shader number 3. Among them, the scene complexity represented by 0 is lower than the scene complexity represented by 1.

[0170] Based on the examples in S1001 and S1002, the terminal device obtains the shader information SShader, the CPU main frequency is 3.0GHz, the GPU frequency is 850MHz, and the scene complexity of the target scene is 1, and the corresponding target simplified shader information is shader number one.

[0171] It is understandable that the terminal device may also obtain the source shader information and the target simplified shader information corresponding to the hardware information of the terminal device from the target correspondence including the source shader information.

[0172] S1004: The terminal device replaces the source shader information in the calling instruction of the source shader with the target simplified shader information, and compiles the target simplified shader.

[0173] In the case where the source shader information is a source shader, the terminal device replaces the source shader with a target simplified shader, and converts the target simplified shader into a shader in a binary language that can be recognized by the terminal device. Wherein, both the source shader and the target simplified shader are shaders written in a shader language. Exemplarily, both the source shader and the target simplified shader are shaders written in an OpenGL shading language.

[0174] When the source shader information is an identifier of a source shader, the terminal device replaces the source shader indicated by the identifier of the source shader with a target simplified shader, and converts the target simplified shader into a shader in a binary language recognizable by the terminal device.

[0175] S1005: The terminal device transmits the compiled target simplified shader to the GPU to implement rendering of the target object.

[0176] In a possible implementation, the terminal device transmits the compiled target simplified shader to the GPU, and the GPU executes the target simplified shader to achieve rendering of the target object.

[0177] Based on the example in S1000, the target object is a group of soldiers, and the GPU executes the target simplified shader to render the same color for the armor worn by these soldiers.

[0178] In the embodiment of the present application, the terminal device learns the simplified shaders corresponding to the shaders in different applications in the terminal devices with different hardware information, and obtains multiple target correspondences. In this way, when the application is run in the terminal device, the terminal device can obtain the target simplified shader information in the target correspondence based on the scene complexity of the scene in the application, the shader used by the application, and its own hardware information, and use the simplified shader to run the application, because the simplified shader occupies less workload of the computing unit in the GPU when running, which can effectively improve the terminal device freeze caused by the shader occupying the GPU.

[0179] It is understandable that each step in the above embodiment can be performed by different devices, thereby obtaining a new embodiment. Fig.11 FIG. 1 is a flow chart of another graphics rendering method provided by an embodiment of the present application, which is applied to Figure 3 The graphics rendering system shown, Fig.11 include:

[0180] S1100: The server collects shaders used in the application, identifies heavy-load shaders among the collected shaders, and simplifies the heavy-load shaders according to a first preset rule to obtain a plurality of simplified shaders.

[0181] For possible implementation methods, please refer to the description in S600 to S602 above and will not be described in detail.

[0182] S1101: The server collects hardware information and state information of the simplified application when it is running on a terminal device with different hardware information, and analyzes the collected information to obtain multiple target correspondences. Among them, the simplified application is an application after the shader in the application is replaced with a simplified shader. The target correspondence is the target correspondence of shader information, hardware information, scene complexity, and simplified shader information.

[0183] For possible implementation methods, please refer to the description in S603 to S605 above and will not be described in detail.

[0184] S1102: The server sends multiple target correspondences to the terminal device.

[0185] S1103: The terminal device obtains a call instruction of a source shader. The source shader is used to implement rendering of a target object. The target object is an object in a target scene, and the target scene is a scene in an application program running on the terminal device. The call instruction of the source shader includes source shader information. The source shader information may include at least one of an identifier of the source shader or a source shader.

[0186] For possible implementation methods and examples, please refer to the description in S1000 above and will not be repeated here.

[0187] S1104: The terminal device determines that the pre-stored multiple target correspondences include information about the source shader.

[0188] For possible implementation methods and examples, please refer to the description in S1001 above and will not be repeated here.

[0189] S1105: The terminal device obtains its own hardware information and the scene complexity of the target scene.

[0190] For possible implementation methods and examples, please refer to the description in S1002 above and will not be repeated here.

[0191] S1106: The terminal device obtains source shader information, hardware information of the terminal device, and target simplified shader information corresponding to the scene complexity of the target scene according to multiple target correspondences.

[0192] For possible implementation methods and examples, please refer to the description in S1003 above and will not be repeated here.

[0193] S1107~S1108 refer to the description in the above S1004~S1005 and will not be repeated here.

[0194] In the embodiment of the present application, the server learns the simplified shaders corresponding to the shaders in different applications in the terminal devices with different hardware information, and obtains multiple target correspondences. The server sends the obtained multiple target correspondences to the terminal device, so that when the application is run in the terminal device, the terminal device can obtain the target simplified shader information in the multiple target correspondences based on the scene complexity of the scene in the application, the shader information used by the application, and its own hardware information, and use the simplified shader to run the application, because the simplified shader occupies less workload of the computing unit in the GPU when running, which can effectively improve the terminal device freeze phenomenon caused by the shader occupying the GPU.

[0195] Fig.12 FIG. 1 is a flow chart of another graphics rendering method provided in an embodiment of the present application. Fig.12 include:

[0196] S1200: The terminal device obtains a call instruction of a source shader. The source shader is used to implement rendering of a target object. The target object is an object in a target scene, and the target scene is a scene in an application program running on the terminal device. The call instruction of the source shader includes source shader information. The source shader information may include at least one of an identifier of the source shader or a source shader.

[0197] For possible implementation methods and examples, please refer to the description in S1000 above and will not be repeated here.

[0198] S1201: The terminal device determines that the source shader is a heavy-load shader according to at least one of the code complexity of the source shader and the number of times the source shader is called.

[0199] For possible implementation methods and examples, please refer to the description in S601 above and will not be repeated here.

[0200] S1202: The terminal device simplifies the source shader according to a first preset rule to obtain a target simplified shader, wherein the first preset rule includes at least one of code movement, expression deletion, or syntax simplification.

[0201] For possible implementation methods and examples, please refer to the description in S602 above and will not be repeated here.

[0202] S1203: The terminal device replaces the source shader information in the calling instruction of the source shader with the target simplified shader information, and compiles the target simplified shader.

[0203] For possible implementation methods and examples, please refer to the description in S1004 above and will not be repeated here.

[0204] S1204: The terminal device transmits the compiled target simplified shader to the GPU to implement rendering of the target object.

[0205] For possible implementation methods and examples, please refer to the description in S1005 above and will not be repeated here.

[0206] In an embodiment of the present application, when an application is running in a terminal device, the terminal device can simplify the heavy-load shaders in the application in real time, and use the simplified shaders to run the application. Because the simplified shaders occupy less workload of the computing units in the GPU when running, the terminal device can effectively improve the lag phenomenon caused by the shader occupying the GPU.

[0207] The above mainly introduces the solution provided by the embodiment of the present application from the perspective of the method. In order to achieve the above functions, it includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should easily appreciate that, in combination with the method steps of each example described in the embodiment disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present application.

[0208] The embodiment of the present application can divide the functional modules of the device for implementing the method for determining the ambient light intensity according to the above method example. For example, each functional module can be divided corresponding to each function, or two or more functions can be integrated into one processing module. Exemplarily, the functions of the target application, the drawing interface, and the display engine are integrated into the display control unit. The above integrated modules can be implemented in the form of hardware or in the form of software functional modules. It should be noted that the division of modules in the embodiment of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation.

[0209] like Fig.13 FIG. 1 is a schematic diagram of the structure of a terminal device provided in an embodiment of the present application. Fig.13 The terminal device 130 shown includes a processing module 1301, which is used to use a target shader to perform graphics rendering of an application; when it is determined that the application is running in a complex scene, a target simplified shader is obtained; a complex scene is a scene in which the number of instructions required for the application to run is greater than a first preset threshold, and the target simplified shader occupies less computing resources of the terminal device 130 than the target shader does; the target simplified shader is used to perform graphics rendering of the application in a complex scene. For example: combined with Figure 6 , the processing module 1301 can be used to execute S600 to S605, combined with Fig.10 , the processing module 1301 can be used to execute S1000 to S1005. Fig.11 , the processing module 1301 can be used to execute S1103 to S1108, combined with Fig.12 , the processing module can be used to execute S1200~S1204.

[0210] Optionally, processing module 1301 is specifically used to: obtain a target simplified shader according to multiple pre-stored correspondences; the multiple correspondences include shader information, device hardware information, and correspondences between simplified shaders; the shader information includes any one of a shader or a shader identifier.

[0211] Optionally, the processing module 1301 is specifically used to: obtain information of the target shader and the target simplified shader corresponding to the hardware information of the terminal device 130 in multiple corresponding relationships.

[0212] Optionally, the above correspondence also includes a load level; the load level is a quantized value corresponding to the load of the terminal device 130; the processing module 1301 is specifically used to: obtain the number of rendering instructions in a complex scene; obtain the target load level according to the number of rendering instructions; in multiple correspondences, obtain the target shader information, the target load level and the target simplified shader corresponding to the hardware information of the terminal device 130.

[0213] Optionally, the processing module 1301 is also used to: simplify the target shader according to a first preset rule to obtain multiple simplified shaders; the first preset rule includes at least one of code movement, expression deletion or syntax simplification; the multiple simplified shaders include a target simplified shader; collect status information of the terminal device 130 when the terminal device 130 uses multiple simplified shaders to perform graphics rendering of an application in a complex scene; the status information of the terminal device 130 includes the load of the terminal device 130, the power consumption of the terminal device 130, the temperature of the terminal device 130, the rendering time, and the image quality of the picture rendered by each simplified shader; the rendering time is the time required for each simplified shader to perform graphics rendering of the application in a complex scene; obtain information about the target shader, hardware information of the terminal device 130, the target load level, and the correspondence between the target simplified shader based on the status information.

[0214] Optionally, the processing module 1301 is specifically used to obtain multiple target rendering times from the status information, and the multiple target rendering times are all less than a second preset threshold; obtain target status information from the status information corresponding to the multiple target rendering times; the image quality of the target status information is greater than a third preset threshold; the target status information is the status information of the terminal device 130 when the target simplified shader performs graphics rendering of the application in a complex scene; obtain the target load level according to the load of the terminal device 130 in the target status information; establish a correspondence between the target shader information, the hardware information of the terminal device 130, the target load level, and the target simplified shader.

[0215] Optionally, the processing module 1301 is specifically used to: obtain multiple target image quality qualities from the status information, and the multiple target image quality qualities are all greater than a fourth preset threshold; obtain target state information from the status information corresponding to the multiple target image quality qualities; the rendering time of the target state information is less than a fifth preset threshold; the target state information is the state information of the terminal device 130 when the target simplified shader performs graphics rendering of the application in a complex scene; obtain the target load level according to the load of the terminal device 130 in the target state information; establish a correspondence between the target shader information, the hardware information of the terminal device 130, the target load level, and the target simplified shader.

[0216] Optionally, the terminal device 130 further includes a communication module 1302: configured to receive multiple corresponding relationships sent by other devices. Fig.11 , the communication module 1302 can be used to execute the receiving step in S1102.

[0217] Optionally, the processing module 1301 is specifically used to simplify the target shader according to a first preset rule to obtain a target simplified shader; the first preset rule includes at least one of code movement, expression deletion or syntax simplification.

[0218] Optionally, when the terminal device 130 determines that the application is running in a complex scene, the processing module 1301 is specifically used to: determine that the application is running in a complex scene, and the code complexity of the target shader is greater than a sixth preset threshold, or the number of calls of the target shader is greater than a seventh preset threshold, obtain the target simplified shader. Figure 4 The function of the processing module 1301 can be Figure 4 The processor 401 in the communication module 1302 calls the computer program in the memory 402 to implement the communication module 1302. Figure 4 The communication module 404 is implemented in the embodiment.

[0219] like Fig.14 FIG. 1 is a schematic diagram of the structure of a server provided in an embodiment of the present application. Fig.14The server 140 shown includes a processing module 1401. The processing module 1401 is used to: simplify the target shader according to a first preset rule to obtain multiple simplified shaders; the first preset rule includes at least one of code movement, expression deletion or syntax simplification; the multiple simplified shaders include a target simplified shader; collect the status information of the terminal device when the terminal device uses multiple simplified shaders to perform graphics rendering of the application in a complex scene; the status information of the terminal device includes the load of the terminal device, the power consumption of the terminal device, the temperature of the terminal device, the rendering time, and the image quality of the picture rendered by each simplified shader; the rendering time is the time required for each simplified shader to perform graphics rendering of the application in a complex scene; obtain the information of the target shader, the hardware information of the terminal device, the target load level and the correspondence between the target simplified shader according to the status information. For example, in combination with Fig.11 , the processing module 1401 can be used to execute S1100~S1101.

[0220] Optionally, processing module 1401 is specifically used to obtain multiple target rendering times from status information, and the multiple target rendering times are all less than a second preset threshold; obtain target status information from status information corresponding to the multiple target rendering times; the image quality of the target status information is greater than a third preset threshold; the target status information is status information of the terminal device when the target simplified shader performs graphics rendering of an application in a complex scene; obtain a target load level according to the load of the terminal device in the target status information; and establish a correspondence between the target shader information, the terminal device hardware information, the target load level, and the target simplified shader.

[0221] Optionally, the processing module 1401 is specifically used to: obtain multiple target image quality qualities from the status information, and the multiple target image quality qualities are all greater than a fourth preset threshold; obtain target state information from the status information corresponding to the multiple target image quality qualities; the rendering time of the target state information is less than a fifth preset threshold; the target state information is the state information of the terminal device when the target simplified shader performs graphics rendering of the application in a complex scene; obtain the target load level according to the load of the terminal device in the target state information; establish a correspondence between the target shader information, the hardware information of the terminal device, the target load level, and the target simplified shader.

[0222] Optionally, the server 140 further includes a communication module 1402: configured to send multiple corresponding relationships to the terminal device. Fig.11 , the communication module 1401 can be used to execute the sending step in S1102.

[0223] In one example, combining Figure 4 The function of the processing module 1401 can be Figure 4The processor 401 in the communication module 1402 calls the computer program in the memory 402 to implement the communication module 1402. Figure 4 The communication module 404 is implemented in the embodiment.

[0224] like Fig.15 FIG. 1 is a schematic diagram of a chip structure provided by an embodiment of the present application. The chip 150 includes one or more (including two) processors 1501 , a communication line 1502 , a communication interface 1503 and a memory 1504 .

[0225] In some implementations, the memory 1504 stores the following elements: executable modules or data structures, or a subset thereof, or an extended set thereof.

[0226] The method described in the above embodiment of the present application can be applied to the processor 1501, or implemented by the processor 1501. The processor 1501 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in the processor 1501 or an instruction in the form of software. The above-mentioned processor 1201 can be a general-purpose processor (for example, a microprocessor or a conventional processor), a digital signal processor (digital signal processing, DSP), an application specific integrated circuit (application specific integrated circuit, ASIC), a field-programmable gate array (field-programmable gate array, FPGA) or other programmable logic devices, discrete gates, transistor logic devices or discrete hardware components, and the processor 1501 can implement or execute the disclosed methods, steps and logic block diagrams in the embodiments of the present application.

[0227] The steps of the method disclosed in the embodiment of the present application can be directly embodied as being executed by a hardware decoding processor, or being executed by a combination of hardware and software modules in the decoding processor. Among them, the software module can be located in a mature storage medium in the field such as a random access memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable read only memory (EEPROM). The storage medium is located in the memory 1504, and the processor 1501 reads the information in the memory 1504 and completes the steps of the above method in combination with its hardware.

[0228] The processor 1501 , the memory 1504 , and the communication interface 1503 may communicate with each other via the communication line 1502 .

[0229] In the above embodiments, the instructions stored in the memory for execution by the processor may be implemented in the form of a computer program product, wherein the computer program product may be pre-written in the memory, or may be downloaded and installed in the memory in the form of software.

[0230] The present application embodiment also provides a computer program product including one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function according to the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from one website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a server or data center that includes one or more available media integrated. For example, the available medium may include a magnetic medium (e.g., a floppy disk, a hard disk or a tape), an optical medium (e.g., a digital versatile disc (DVD)), or a semiconductor medium (e.g., a solid state disk (SSD)), etc.

[0231] The present application also provides a computer-readable storage medium. The methods described in the above embodiments can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. Computer-readable media may include computer storage media and communication media, and may also include any medium that can transfer a computer program from one place to another. The storage medium may be any target medium that can be accessed by a computer.

[0232] As a possible design, the computer readable medium may include a compact disc read-only memory (CD-ROM), RAM, ROM, EEPROM or other optical disc storage; the computer readable medium may include a magnetic disk storage or other magnetic disk storage device. Moreover, any connection line may also be appropriately referred to as a computer readable medium. For example, if the software is transmitted from a website, server or other remote source using a coaxial cable, fiber optic cable, twisted pair, DSL or wireless technology (such as infrared, radio and microwave), the coaxial cable, fiber optic cable, twisted pair, DSL or wireless technology such as infrared, radio and microwave are included in the definition of medium. Disk and disc as used herein include compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk and Blu-ray disc, where disks usually reproduce data magnetically, while optical discs reproduce data optically using lasers.

[0233] The above combinations should also be included in the scope of computer-readable media. The above description is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A graphics rendering method, characterized in that: The method comprises: The simplified platform determines a shader calling instruction according to a preset keyword, intercepts the shader calling instruction, and obtains a shader used in the running process of the application; the shader calling instruction includes a shader identifier and at least one of the shaders, and the shader is used to render a target object in a target scene; The simplified platform obtains a heavy-load shader from the shaders according to at least one of code complexity of the shader and the number of times the shader is called; The simplification platform simplifies the heavy-load shader according to a first preset rule to obtain a plurality of simplified shaders; the first preset rule comprises at least one of expression deletion and syntax simplification; The simplified platform replaces the heavy-load shader in the application with different simplified shaders obtained by simplifying the heavy-load shader, thereby obtaining a plurality of simplified applications; wherein the different simplified shaders are shaders obtained by simplifying the heavy-load shader according to different first preset rules; The simplified platform collects status information of the multiple simplified applications when they are running on the terminal device and the hardware information of the terminal device; the status information includes the load of the terminal device, the power consumption of the terminal device, the temperature of the terminal device, the rendering time, and the image quality of the picture rendered by each simplified shader; the rendering time is the time required for each simplified shader to render the graphics of the application in a complex scene; The simplified platform filters the simplified shader according to the state information and the hardware information to obtain a plurality of target correspondences, wherein the target correspondences include shader information, hardware information of the terminal device, a target load level, and a correspondence between simplified shaders, wherein the target load level is a load level corresponding to a load when the terminal device runs a simplified application; The terminal device intercepts a call instruction of a source shader, wherein the call instruction of the source shader includes at least one of an identifier of the source shader and a source shader; The terminal device determines that the target correspondence includes source shader information, the terminal device obtains its own hardware information, and calculates the number of rendering instructions in the target scene, and determines the scene complexity of the target scene according to the number of rendering instructions, wherein the scene complexity corresponds to a load level, and the load level is a quantized value corresponding to the load of the terminal device; The terminal device determines, according to the target corresponding relationship, a target simplified shader corresponding to the source shader information, the hardware information of the terminal device, and the scene complexity of the target scene, wherein the target simplified shader occupies less computing resources of the terminal device than the target shader does; The terminal device replaces the source shader information in the calling instruction of the source shader with the target simplified shader information; The terminal device transmits the target simplified shader to a graphics processor, and the graphics processor executes the target simplified shader to render the target object.

2. The method according to claim 1, characterized in that The simplified platform filters simplified shaders according to the state information and the hardware information to obtain a plurality of target correspondences, including: The simplified platform obtains a plurality of target rendering durations from the state information, and the plurality of target rendering durations are all less than a second preset threshold; The simplified platform obtains target state information from the state information corresponding to the multiple target rendering durations; the image quality of the target state information is greater than a third preset threshold; the target state information is the state information of the terminal device when the simplified shader performs graphics rendering of the application in the complex scene; The simplified platform obtains the target load level according to the load of the terminal device in the target state information; The simplified platform establishes a correspondence between the shader information, the hardware information of the terminal device, the target load level, and the simplified shader.

3. The method according to claim 1, characterized in that The simplified platform filters the simplified shader according to the state information and the hardware information to obtain a plurality of target correspondence relationships, including: The simplified platform obtains a plurality of target image qualities from the state information, and the plurality of target image qualities are all greater than a fourth preset threshold; The simplified platform obtains target state information from the state information corresponding to the multiple target image qualities; the rendering time of the target state information is less than a fifth preset threshold; the target state information is the state information of the terminal device when the simplified shader performs graphics rendering of the application in the complex scene; The simplified platform obtains the target load level according to the load of the terminal device in the target state information; The simplified platform establishes a correspondence between the shader information, the hardware information of the terminal device, the target load level, and the simplified shader.

4. The method according to any one of claims 2 to 3, characterized in that: The method further comprises: The terminal device receives the multiple corresponding relationships sent by other devices.

5. An electronic device, characterized in that: include: A memory and a processor, wherein the memory is used to store a computer program, and the processor is used to execute the computer program to perform the graphics rendering method according to any one of claims 1 to 4.

6. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions, and when the instructions are executed, the computer executes the graphics rendering method according to any one of claims 1 to 4.

Images