Rendering method and device

Abstract

The invention discloses a rendering method and device, and belongs to the field of data processing. The method comprises the following steps: loading texture resource data of a first scene; processing the texture resource data based on a super-resolution algorithm to obtain super-resolution texture resource data; and rendering the first scene through the super-divided texture resource data.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of data processing, and particularly relates to a rendering method and device. BACKGROUND

[0002] In the field of mobile games, super-resolution technology (SR) is a mainstream real-time rendering quality enhancement scheme. The core process of SR is to separate the pictures of a 3D scene and a 2D user interface, improve the rendering resolution of the 3D scene through a specific algorithm, and then synthesize the two pictures to enhance the picture quality. Super-resolution technology has played an important role in low-resolution rendering scenes.

[0003] However, with the improvement of hardware performance, the basic internal rendering resolution of current mobile games has generally reached 1080P. On this high basis, the rendering quality enhancement effect brought by SR technology is extremely limited, and cannot meet the needs of users for high-quality pictures.

[0004] Therefore, the picture quality of the rendered scene still needs to be improved. SUMMARY

[0005] The purpose of the embodiments of the present application is to provide a rendering method and device, which can solve the problem that the picture quality still needs to be improved.

[0006] In a first aspect, the embodiments of the present application provide a rendering method, which comprises:

[0007] loading texture resource data of a first scene;

[0008] processing the texture resource data based on a super-resolution algorithm to obtain super-resolution texture resource data;

[0009] rendering the first scene through the super-resolution texture resource data.

[0010] In a second aspect, the embodiments of the present application provide a rendering device, which comprises:

[0011] a loading module configured to load texture resource data of a first scene;

[0012] a processing module configured to process the texture resource data based on a super-resolution algorithm to obtain super-resolution texture resource data;

[0013] a rendering module configured to render the first scene through the super-resolution texture resource data.

[0014] In a third aspect, the embodiments of the present application provide an electronic device, which comprises a processor and a memory. The memory stores programs or instructions that can be run on the processor. When the programs or instructions are executed by the processor, the steps of the method according to the first aspect are implemented.

[0015] In a fourth aspect, an embodiment of the present application provides a readable storage medium, the readable storage medium storing a program or instructions, the program or instructions being executed by a processor to implement the steps of the method according to the first aspect.

[0016] In a fifth aspect, an embodiment of the present application provides a chip, the chip comprising a processor and a communication interface, the communication interface being coupled to the processor, and the processor being configured to execute a program or instructions to implement the method according to the first aspect.

[0017] In a sixth aspect, an embodiment of the present application provides a computer program product stored in a storage medium, the computer program product being executed by at least one processor to implement the method according to the first aspect.

[0018] In the embodiments of the present application, by loading the texture resource data of the first scene, the texture resource data is processed based on a super-resolution algorithm, richer details are injected into the texture resource data, super-resolution texture resource data is obtained, and high-quality materials are reserved for quality improvement. By rendering the first scene based on the super-resolution texture resource data, the visual elements such as surface texture and texture clarity of objects in the scene are significantly improved, and the rendering quality is enhanced. BRIEF DESCRIPTION OF DRAWINGS

[0019] Figure 1 is a flowchart of a rendering method provided by an embodiment of the present application;

[0020] Figure 2 is a flowchart of another rendering method provided by an embodiment of the present application;

[0021] Figure 3 is an interaction flowchart of a rendering method provided by an embodiment of the present application;

[0022] Figure 4 is a structural diagram of a rendering device provided by an embodiment of the present application;

[0023] Figure 5 is one of hardware structure schematic diagrams of an electronic device according to an embodiment of the present application;

[0024] Figure 6 is another hardware structure schematic diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0025] The technical solutions of the embodiments of the present application will be described clearly below with reference to the drawings of the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by a person of ordinary skill in the art belong to the scope of protection of the present application.

[0026] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first", "second", etc. are generally of a kind and do not limit the number of objects, for example, the first object can be one or more. In addition, "and / or" in the specification and claims indicates at least one of the connected objects, and the character " / " generally indicates that the objects before and after are in an "or" relationship.

[0027] The rendering method provided by the embodiments of the present application can be applied to at least the following application scenarios, which are described below.

[0028] Runtime texture enhancement, as a key branch of real-time rendering quality enhancement technology, aims to dynamically improve the expressiveness of texture maps during application execution, thereby significantly improving the visual fidelity of the final picture.

[0029] Runtime texture enhancement technology, combined with virtual texture systems, can dynamically provide higher resolution normal, roughness, and other material property maps in areas that require high detail performance. This ensures that under harsh lighting and viewing angle conditions, materials can still exhibit accurate three-dimensionality, continuous highlight transitions, and realistic surface gloss changes, greatly enhancing visual realism and immersion. Therefore, runtime texture enhancement is an indispensable technology for balancing real-time rendering performance and top-notch visual quality, directly determining whether the final picture can reach an amazing level.

[0030] To solve the problems in the related art, the embodiments of the present application provide a rendering method and device, which can solve the problem that the picture quality cannot be improved by super-resolution technology.

[0031] The rendering method provided by the embodiments of the present application will be described in detail below in combination with the drawings, specific embodiments and application scenarios thereof.

[0032] Figure 1 A flowchart of a rendering method provided by the embodiments of the present application.

[0033] As shown in Figure 1 The rendering method can include steps 110-130, and the method is applied to a rendering device, as shown below:

[0034] Step 110, loading texture resource data of a first scene;

[0035] Texture resource data is digital information that describes the details of an image surface, including pixel color, transparency, etc., and is the basic data for rendering the texture of an object surface in a scene.

[0036] Loading the texture resource data of the first scene prepares the image materials used to represent the details of the object surface in the scene. The texture resource data can be an image stored in a file, such as an image representing the color and concave-convex of an object. These texture resource data are first read from storage, the format of the texture resource data is parsed, and information such as the arrangement of pixels and the color channels contained is determined. The purpose is to prepare the original texture materials, ensure that there is available original data for subsequent steps, and efficient loading can reduce waiting time and make the entire process smoother.

[0037] In step 120, the texture resource data is processed based on a super-resolution algorithm to obtain super-resolution texture resource data.

[0038] Super-resolution algorithm is an algorithm that restores or generates high-resolution details from low-resolution images through mathematical models and image processing techniques, which can improve image clarity without relying on original high-resolution data. Super-resolution texture resource data is texture data obtained after processing by a super-resolution algorithm, which has higher resolution and more detailed than the original data.

[0039] Processing the texture resource data based on a super-resolution algorithm is to make the texture clearer and more detailed. The role of the super-resolution algorithm is to restore higher resolution details from relatively blurred low-resolution textures through calculation. The algorithm based on deep learning can more intelligently restore details that may have been lost, such as the sharpness of object edges and the fine texture of surfaces, by learning the rules of a large number of images. The original texture data in the graphics card is processed to calculate higher resolution textures, resulting in better quality super-resolution texture resource data.

[0040] In step 130, the first scene is rendered using the super-resolution texture resource data.

[0041] The processed high-quality texture is used to generate the final scene picture. During the rendering process, the graphics card calculates the color of each pixel based on the shape of the objects in the scene, the lighting conditions, and the information of the super-resolution texture. For example, the patterns on the texture are mapped to the surface of the object, and the light shining produces shading effects, finally forming the picture seen by the user. Because the super-resolution texture is used, the surface of the objects in the picture will be more realistic and detailed, such as the texture of distant buildings and the texture of fabrics, which can be better presented, thereby improving the visual effect and immersion of the scene, making the entire scene look more delicate.

[0042] In one possible embodiment, step 120 may specifically include the following steps:

[0043] Step 210: Decompress the copy texture resource data created based on the texture resource data to generate intermediate texture data;

[0044] Step 220: The intermediate texture data is processed by a filtering algorithm based on image content to obtain diffuse texture resource data;

[0045] Step 230: Perform super-resolution algorithm processing on the diffuse texture resource data to generate high-resolution texture data;

[0046] Step 240: Compress the high-resolution texture data to obtain the super-resolution texture resource data.

[0047] Among them, the duplicate texture resource data is a copy created based on the original texture resource data, used to protect the original data from being modified or damaged during data processing; the decompression operation is the process of restoring the compressed texture data to the uncompressed format so that subsequent algorithms can directly process the pixel information of the texture; the intermediate texture data is the transitional texture data obtained after the decompression operation, which serves as the input source for subsequent processing steps; the image content filtering algorithm is an algorithm that performs targeted processing based on the content characteristics of different regions in the image, which can remove noise or redundant data while retaining key visual information;

[0048] Diffuse texture resource data describes the reflective properties of an object's surface when it receives non-directional light, directly affecting the basic color and texture of the object in rendering; high-resolution texture data is texture data that has been processed by super-resolution algorithms, significantly improving the number of pixels and the richness of details; compression is the process of converting the processed high-resolution texture data back into an efficient compression format to reduce data storage volume and transmission bandwidth.

[0049] In step 210, the copy texture resource data is decompressed to generate intermediate texture data. The decompression interface of the corresponding compression format is called to decode the encoded and compressed pixel information in the copy texture data into uncompressed raw pixel data. During this process, the color channel information of each pixel is restored to ensure that the texture details are completely preserved.

[0050] By manipulating the copy texture data, interference with the original texture resources is avoided, ensuring the security of the original data. At the same time, the decompressed intermediate texture data eliminates the limitations of the compression format on subsequent algorithms, providing directly operable basic data for steps such as image filtering and super-resolution processing, ensuring the accuracy of subsequent processing.

[0051] In step 220, the image content-based filtering algorithm processes the intermediate texture data to obtain diffuse texture resource data. The algorithm first analyzes the content features of the intermediate texture, such as identifying the color transition area, edge contour and noise points on the object surface, and then performs targeted smoothing or sharpening processing to finally extract the texture information that conforms to the diffuse reflection characteristics, that is, the basic color texture after removing interference from non-diffuse reflection factors such as highlights and shadows.

[0052] The filtered diffuse texture resource data better matches the physical reflection characteristics of the object's surface, reducing the interference of irrelevant information on subsequent super-resolution processing. This allows the super-resolution algorithm to focus more on improving diffuse details, laying the foundation for rendering realistic and natural object colors in the end.

[0053] In step 230, a super-resolution algorithm is performed on the diffuse texture resource data to generate high-resolution texture data. The super-resolution algorithm analyzes the low-resolution details in the diffuse texture, uses the trained feature mapping relationship or mathematical model to predict and fill in the pixel information that should be present under high resolution, increases the number of pixels in the low-resolution texture, and maintains the naturalness and consistency of the texture.

[0054] High-resolution texture data contains richer diffuse reflection details compared to the original data, which can present a more delicate surface texture of objects during rendering. It avoids problems such as blurring and pixelation that occur when low-resolution textures are enlarged, and significantly improves the visual detail of the scene.

[0055] In step 240, the high-resolution texture data is compressed to obtain super-resolution texture resource data. An efficient texture compression algorithm is used to reduce the amount of data storage while ensuring that the key details of the high-resolution texture are not lost. During the compression process, the pixel information is reorganized and encoded to balance the compression rate and image quality loss.

[0056] The compressed super-resolution texture resource data size is significantly reduced, which reduces storage usage and transmission bandwidth requirements, making it easier for the rendering thread to load and use it quickly. At the same time, it retains the detail improvement brought by super-resolution processing, ultimately achieving a balance between high image quality and high performance, ensuring the efficiency and stability of the rendering process.

[0057] The copy texture resource data is in the adaptive scalable texture compression ASTC compression format.

[0058] Step 210 includes:

[0059] Perform decompression on the ASTC compressed copy texture resource data to generate intermediate texture data in RGBA format with red, green, and blue transparency.

[0060] Step 240 includes:

[0061] The high-resolution texture data in RGBA format is compressed into ASTC format to obtain the super-resolution texture resource data.

[0062] The copy texture resource data uses the Adaptive Scalable Texture Compression (ASTC) format. This format is characterized by its flexible block size settings and adaptive encoding methods, which achieve efficient data compression while ensuring high image quality. It can dynamically adjust the compression strategy according to different texture features and is suitable for various resolution requirements.

[0063] The Red Green Blue Alpha (RGBA) format is an uncompressed pixel format where R (Red), G (Green), and B (Blue) represent the three primary color channels, and A (Alpha) represents the transparency channel. Each channel typically occupies 8 bits of data, which can completely preserve the color and transparency information of the pixel, making it easy to perform pixel-level algorithm processing directly.

[0064] In step 210, the copy texture resource data in ASTC compressed format is decompressed to generate intermediate texture data in RGBA format. The ASTC dedicated decoding logic is called to first parse the block division information, encoding parameters and texture feature data in the compressed data, and then perform decoding operations on each compressed block based on this information to restore the compressed encoded value to the corresponding RGBA four-channel pixel value, and finally obtain uncompressed texture data in which each pixel contains complete color and transparency information.

[0065] The ASTC format copy data not only ensures the safety of the original texture information, but also eliminates the limitations of compression encoding on subsequent image processing by decompressing and converting it to RGBA format. This is because RGBA format pixel data can be directly read and calculated by image filtering algorithms and super-resolution algorithms, ensuring the accuracy and integrity of texture details in subsequent processing and laying the data foundation for high-quality texture enhancement.

[0066] In step 240, the high-resolution texture data in RGBA format is compressed into ASTC format to obtain super-resolution texture resource data. An appropriate ASTC block size is selected according to the content characteristics of the high-resolution texture. Then, the RGBA pixel data is divided into blocks, analyzed for features, and compressed by encoding. During the compression process, adaptive quantization and predictive coding are used to significantly reduce the data volume while preserving as much detail as possible from the super-resolution processing.

[0067] By recompressing high-resolution RGBA data into the ASTC format, the rich texture details after super-resolution are preserved, while the efficient compression characteristics of ASTC significantly reduce the data storage and transmission bandwidth requirements. This avoids the problem of excessive video memory usage caused by high-resolution data, and facilitates the rapid loading and use of super-resolution texture resources. Ultimately, while ensuring high-quality rendering effects, the efficiency and stability of the rendering process are maintained.

[0068] In one possible embodiment, step 220 may specifically include the following steps:

[0069] The image content-based filtering algorithm analyzes the pixel value distribution of the intermediate texture data to identify the texture type;

[0070] Resource data whose texture type is diffuse texture is identified as diffuse texture resource data.

[0071] Among them, pixel value distribution refers to the distribution characteristics of the RGBA channel values ​​of all pixels in the intermediate texture data throughout the entire texture range, including the range of color values, frequency of occurrence, gradient change amplitude, etc.; texture type is a category divided according to the optical properties and visual characteristics of the object surface represented by the texture, such as diffuse texture, specular texture, normal texture, etc. Different types correspond to different reflection behaviors of the object surface; diffuse texture type is a texture category specifically used to describe the basic color and texture of the object surface when it receives uniform ambient light. Its pixel value distribution usually has a relatively smooth color transition and fewer areas with extreme brightness or strong contrast.

[0072] In step 220, the image content-based filtering algorithm analyzes the pixel value distribution of the intermediate texture data and identifies the texture type. Then, the resource data of the diffuse texture type is determined as diffuse texture resource data. The pixel value distribution features of the intermediate texture are extracted by the filtering algorithm, such as calculating the numerical range and variance of each color channel, statistically analyzing the distribution frequency of brightness values, and detecting gradient changes in pixel values.

[0073] The algorithm matches these features with a preset diffuse texture type feature model. Typical features of diffuse textures include: small color channel variance, brightness values ​​concentrated in a medium range, and smooth pixel gradient changes. When the pixel value distribution characteristics of the intermediate texture data match the diffuse texture type to a set threshold, the algorithm determines that the texture is a diffuse texture type and identifies it as diffuse texture resource data required for subsequent processing.

[0074] By accurately analyzing pixel value distribution, the diffuse texture that truly represents the basic color and texture of an object can be effectively filtered out from the intermediate texture data, eliminating the interference of other types of textures such as specular highlights and normals. This allows subsequent super-resolution processing to focus on the diffuse details that play a key role in the rendering effect, improving the processing efficiency and targeting of the super-resolution algorithm.

[0075] Specifically, if the pixel values ​​of the texture data are continuously and smoothly distributed in the RGB color space, it is identified as a diffuse texture; if the pixel values ​​of the texture data are centrally symmetrically distributed in the normal space, it is identified as a normal map; if the pixel values ​​of the texture data are concentrated in a single channel and conform to the characteristics of metallic reflection, it is identified as a metallic map; if the pixel values ​​of the texture data conform to the distribution of roughness characteristics, it is identified as a roughness map.

[0076] When analyzing intermediate texture data, image content-based filtering algorithms distinguish different texture types by identifying the distribution characteristics of different pixel values, and utilize the pixel distribution patterns presented by the differences in optical properties of different textures.

[0077] For diffuse textures, the essence is to present the basic color of the object's surface under uniform lighting. This color is usually determined by the object's own material and does not have strong highlights or directional reflections. Therefore, in the RGB color space, the red, green, and blue channel values ​​of its pixels will show a continuous and smooth distribution. That is, the color values ​​of adjacent pixels change smoothly without abrupt jumps, the color transition is natural, and the overall color distribution range conforms to the inherent color characteristics of real objects. By detecting this continuous and smooth color distribution feature, the algorithm can identify it as a diffuse texture.

[0078] Normal maps record the direction of normals on an object's surface to simulate its subtle unevenness. These normal directions are typically isotropically distributed in three-dimensional space. Metallic maps characterize the metallic properties of an object's surface. Their values ​​are usually single-channel, and the pixel values ​​have clear meanings: 0 represents non-metallic, and 1 represents metallic. Algorithms can identify metallic maps by detecting this binary or transitional distribution feature within a single channel.

[0079] Roughness maps are used to describe the roughness of an object's surface. Their values ​​are mostly single-channel; smaller values ​​indicate a smoother surface, while larger values ​​indicate a rougher surface. The distribution of pixel values ​​follows specific patterns based on the object's surface characteristics. For example, the roughness values ​​of a smooth metal surface are concentrated in the low range, while the roughness values ​​of a frosted surface are concentrated in the medium-to-high range. Algorithms can identify roughness maps by matching these distribution features corresponding to surface roughness.

[0080] Through the aforementioned recognition logic based on pixel value distribution features, the worker thread can accurately distinguish different types of textures, thereby confirming resource data identified as diffuse textures as diffuse texture resource data. Targeted feature detection enables automatic classification of texture types, avoiding confusion between textures with different functions. This ensures that subsequent super-resolution algorithms only apply to diffuse textures that play a crucial role in basic color and texture, improving processing efficiency while guaranteeing the accuracy of the super-resolution effect. This provides a precise data foundation for ultimately rendering a layered and realistic scene effect.

[0081] In one possible embodiment, step 110 may specifically include the following steps:

[0082] Control the rendering thread to load the texture resource data of the first scene;

[0083] Step 120 may specifically include the following steps:

[0084] The control worker thread processes the texture resource data based on the super-resolution algorithm to obtain super-resolution texture resource data;

[0085] Step 130 may specifically include the following steps:

[0086] The rendering thread is controlled to render the first scene using the super-resolution texture resource data.

[0087] The rendering thread is specifically responsible for drawing graphics data onto the display device. It interacts directly with the graphics processor and undertakes core rendering tasks such as scene drawing and pixel filling. The rendering thread controls the loading of texture resource data for the first scene. The rendering thread reads the stored texture resources through the file system, and after format parsing and data verification, uploads the texture data to the GPU's video memory, completing the mapping from storage to video memory and preparing the basic data for subsequent rendering operations.

[0088] By having the rendering thread directly perform the loading operation, it can make full use of its efficient communication mechanism with the GPU, ensuring that texture data quickly enters the rendering-ready state, avoiding rendering stutters caused by data transmission delays, and laying the data foundation for the entire rendering process.

[0089] Worker threads are background threads used to handle computationally intensive tasks. They run independently of the rendering thread and can process data in parallel without interfering with the rendering process. After obtaining the original texture data from memory or video memory, they call the super-resolution algorithm to predict and fill in the pixel details that should be present in the high resolution by analyzing the edge features and texture patterns in the texture, thus upscaling the low-resolution texture data to a higher resolution.

[0090] By using worker threads to handle computationally intensive super-resolution tasks independently, the resources of the rendering thread can be avoided, ensuring the smoothness of the rendering process. At the same time, super-resolution processing can significantly improve the detail richness of textures and make up for the blurring problem of the original low-resolution textures when they are enlarged.

[0091] The rendering thread controls the rendering of the first scene using super-resolution texture resource data. The rendering thread calls the GPU's rendering pipeline to combine the super-resolution texture resource data with the 3D models and lighting information in the scene. Through steps such as vertex shading and fragment shading, the texture details are mapped onto the object surface, and finally, an image containing high-resolution textures is generated and output to the display device.

[0092] The super-resolution texture data provides a more refined foundation for scene rendering, making the rendered scene visually clearer and more realistic. At the same time, the rendering thread focuses on the rendering task, working in conjunction with the data processing of the previous working threads to maintain a stable rendering frame rate while ensuring high image quality, thus improving the user's visual experience and interaction smoothness.

[0093] In one possible embodiment, step 130 may specifically include the following steps:

[0094] In the rendering thread, it is checked whether the super-resolution texture resource data corresponding to the currently rendered object has been generated;

[0095] If the corresponding super-resolution texture resource data has been generated, the current rendering object is rendered using the super-resolution texture resource data.

[0096] If the corresponding super-resolution texture resource data is not generated, the current rendering object is rendered using the texture resource data corresponding to the current rendering object.

[0097] Among them, the rendering object refers to the specific renderable elements in the first scene, such as 3D models and interface icons, which are the specific targets of the rendering operation; the super-resolution texture resource data is the high-detail texture data obtained after super-resolution processing by the worker thread; and the texture resource data is the original texture data initially loaded by the rendering thread, which has not undergone super-resolution processing.

[0098] The rendering thread first checks whether the super-resolution texture resource data corresponding to the current rendering object has been generated. The rendering thread obtains the processing progress of the super-resolution texture in real time through the status flag shared with the worker thread. After the worker thread completes the super-resolution processing, it updates the status flag of the corresponding rendering object. Before rendering the object, the rendering thread reads this flag to determine whether the super-resolution texture is available.

[0099] When the super-resolution texture resource data is found to have been generated, the rendering thread uses the super-resolution texture to render the current object. Utilizing the richer detail information in the super-resolution texture, and through the GPU's texture sampling and pixel shading process, these details are mapped onto the surface of the rendered object, resulting in a clearer texture on the screen. This fully leverages the advantages of super-resolution processing, making the rendered objects visually more refined and improving the overall image quality of the scene.

[0100] When the super-resolution texture resource data is not generated, the rendering thread switches to using the original texture resource data corresponding to the current object for rendering. Since the original texture is pre-loaded into video memory and immediately usable, the rendering pipeline is directly invoked to complete the drawing, ensuring the rendering process is uninterrupted. By using the original texture as a fallback, frame freezes or stuttering caused by waiting for super-resolution processing are avoided, ensuring the stability of the rendering frame rate and balancing the needs for improved image quality and smoothness.

[0101] By dynamically querying and selectively using texture data, efficient collaboration between the rendering thread and the worker thread is achieved. This mechanism enhances image quality when the super-resolution texture is ready and maintains rendering continuity using the original texture when it is not ready. Ultimately, while ensuring smooth scene rendering, the enhancement effect of super-resolution processing on visual effects is maximized, thus improving the overall user experience.

[0102] The step of rendering the current rendering object using native texture resource data when the corresponding super-resolution texture resource data has not been generated may further include the following steps:

[0103] The rendering thread is controlled to monitor the generation status of the super-resolution texture resource data;

[0104] If it is detected that the super-resolution texture resource data required by the current rendering object has been generated, the rendering thread is controlled to replace the texture resource data used by the current rendering object with the super-resolution texture resource data.

[0105] Among them, the generation status refers to the processing progress status of the super-resolution texture resource data, which is usually recorded in the form of flag bits, status codes, etc., to indicate whether the super-resolution texture has been processed and can be used for rendering; texture replacement is the operation of the rendering thread to switch the texture resource bound to the current rendering object from the original texture to the super-resolution texture, which needs to be completed in a specific stage of the rendering pipeline to ensure image consistency.

[0106] After rendering the current rendering object using native texture resource data, the rendering thread monitors the generation status of the super-resolution texture resource data. The rendering thread continuously tracks the processing progress of the super-resolution texture through a preset synchronization mechanism. The worker thread updates the status in real time during the super-resolution processing. The rendering thread reads this status during each frame rendering interval or in a dedicated monitoring cycle to ensure that it knows in a timely manner whether the super-resolution texture is available.

[0107] When it is detected that the super-resolution texture resource data required by the current rendering object has been generated, the rendering thread controls the replacement of the texture resource data used by the object with the super-resolution texture resource data. After confirming that the super-resolution texture is available, the rendering thread removes the association with the original texture during the texture binding stage of the rendering pipeline, and rebinds the memory address, format information, etc. of the super-resolution texture to the material properties of the rendering object to ensure that subsequent drawing operations use the newly generated super-resolution texture.

[0108] By continuously monitoring the generation status of super-resolution textures, the latency from the completion of super-resolution processing to actual application in rendering is minimized, allowing users to experience the improvement in image quality more quickly. The texture replacement mechanism ensures a seamless transition from the original texture to the super-resolution texture, avoiding problems such as screen flickering, tearing, or local blurring that may occur during the replacement process. While ensuring smooth rendering, the scene image quality is gradually improved to the super-resolution level, taking into account both the user's immediate experience and the final visual effect.

[0109] In the embodiments of this application, by loading the texture resource data of the first scene and processing the texture resource data based on a super-resolution algorithm, richer details are injected into the texture resource data to obtain super-resolution texture resource data, thus reserving high-quality materials for image quality improvement. Rendering the first scene using super-resolution texture resource data significantly improves visual elements such as the surface texture and texture clarity of objects in the scene, thereby enhancing the rendered image quality.

[0110] The following is combined with Figure 3 The embodiments of this application are described below:

[0111] Step 310: Load the texture resource data for the first scene.

[0112] The rendering thread loads the texture data required by the scene from the storage medium, including diffuse textures, normal maps, metallic maps, etc., to provide raw materials for subsequent rendering and processing.

[0113] Step 320: The rendering thread sends the texture resource data of the first scene to the worker thread.

[0114] By using an inter-thread communication mechanism, the original texture data is passed to the worker thread, achieving asynchronous decoupling between the rendering process and super-resolution calculation, thus avoiding rendering blockage.

[0115] Step 330: Process the texture resource data based on the super-resolution algorithm to generate super-resolution texture resource data.

[0116] The worker thread uses a super-resolution algorithm to reconstruct and enhance the pixel details of the original texture, outputting higher quality texture data.

[0117] Step 340: The worker thread sends the super-resolution texture resource data back to the rendering thread.

[0118] After completing the super-resolution processing, the rendering thread is notified through a synchronization mechanism, and the enhanced texture data is fed back to provide materials for the final rendering.

[0119] Step 350: Render the first scene using super-resolution texture resource data.

[0120] The rendering thread binds the super-resolution texture to the rendering pipeline, driving the GPU to complete scene rendering. It then uses the enhanced texture details to improve image sharpness and achieve image quality enhancement. Thus, the above steps, through the division of labor and cooperation between the rendering thread and worker threads, ensure smooth rendering while simultaneously enhancing texture details through super-resolution algorithms, ultimately improving the quality of scene rendering.

[0121] The rendering method provided in this application can be executed by a rendering device. This application uses the example of a rendering device executing the rendering method to illustrate the rendering device provided in this application.

[0122] Figure 4 This is a block diagram of a rendering apparatus 400 provided in an embodiment of this application. The apparatus 400 includes:

[0123] Loading module 410 is used to load texture resource data of the first scene;

[0124] Processing module 420 is used to process the texture resource data based on the super-resolution algorithm to obtain super-resolution texture resource data;

[0125] The rendering module 430 is used to render the first scene using the super-resolution texture resource data.

[0126] In one possible embodiment, the processing module 420 includes:

[0127] The decompression submodule is used to perform a decompression operation on the copy texture resource data created based on the texture resource data to generate intermediate texture data;

[0128] The filtering processing submodule is used to process the intermediate texture data based on the image content filtering algorithm to obtain diffuse texture resource data;

[0129] The super-resolution processing submodule is used to perform super-resolution algorithm processing on the diffuse texture resource data to generate high-resolution texture data;

[0130] The compression submodule is used to compress the high-resolution texture data to obtain the super-resolution texture resource data.

[0131] In one possible embodiment, the copied texture resource data is in the adaptive scalable texture compression ASTC compression format; the decompression submodule is specifically used for:

[0132] Perform decompression on the ASTC compressed copy texture resource data to generate intermediate texture data in RGBA format with red, green, and blue transparency.

[0133] The compression submodule is specifically used for:

[0134] The high-resolution texture data in RGBA format is compressed into ASTC format to obtain the super-resolution texture resource data.

[0135] In one possible embodiment, the filtering submodule is specifically used for:

[0136] The image content-based filtering algorithm analyzes the pixel value distribution of the intermediate texture data to identify the texture type;

[0137] Resource data whose texture type is diffuse texture is identified as diffuse texture resource data.

[0138] In one possible embodiment, the loading module is specifically used for:

[0139] Control the rendering thread to load the texture resource data of the first scene;

[0140] The processing module is specifically used for:

[0141] The control worker thread processes the texture resource data based on a super-resolution algorithm to obtain the super-resolution texture resource data; the worker thread is a background thread used to handle computationally intensive tasks.

[0142] The rendering module is specifically used for:

[0143] The rendering thread is controlled to render the first scene using the super-resolution texture resource data.

[0144] In the embodiments of this application, by loading the texture resource data of the first scene and processing the texture resource data based on a super-resolution algorithm, richer details are injected into the texture resource data to obtain super-resolution texture resource data, thus reserving high-quality materials for image quality improvement. Rendering the first scene using super-resolution texture resource data significantly improves visual elements such as the surface texture and texture clarity of objects in the scene, thereby enhancing the rendered image quality.

[0145] The rendering device in this application embodiment can be an electronic device or a component within an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other devices besides a terminal. For example, the electronic device can be a mobile phone, tablet computer, laptop computer, PDA, in-vehicle electronic device, mobile internet device (MID), augmented reality (AR) / virtual reality (VR) device, robot, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc. It can also be a server, network attached storage (NAS), personal computer (PC), television set (TV), ATM, or self-service machine, etc. This application embodiment does not specifically limit the scope of the device.

[0146] The rendering apparatus in this application embodiment can be a device with an action system. The action system can be an Android action system, an iOS action system, or other possible action systems; this application embodiment does not specifically limit it.

[0147] The rendering apparatus provided in this application embodiment can implement all the processes implemented in the above method embodiments, and will not be described again here to avoid repetition.

[0148] Optionally, such as Figure 5 As shown, this application embodiment also provides an electronic device 510, including a processor 511, a memory 512, and a program or instructions stored in the memory 512 and executable on the processor 511. When the program or instructions are executed by the processor 511, they implement the various steps of any of the above-described rendering method embodiments and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0149] It should be noted that the electronic devices in the embodiments of this application include the mobile electronic devices and non-mobile electronic devices described above.

[0150] Figure 6 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of this application.

[0151] The electronic device 600 includes, but is not limited to, components such as: radio frequency unit 601, network module 602, audio output unit 603, input unit 604, sensor 605, display unit 606, user input unit 607, interface unit 608, memory 609, and processor 610.

[0152] Those skilled in the art will understand that the electronic device 600 may also include a power supply (such as a battery) for supplying power to various components. The power supply may be logically connected to the processor 610 through a power management system, thereby enabling functions such as managing charging, discharging, and power consumption through the power management system. Figure 6 The electronic device structure shown does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements, which will not be elaborated here.

[0153] Among them, processor 610 is used to load texture resource data of the first scene;

[0154] The processor 610 is also used to process the texture resource data based on a super-resolution algorithm to obtain super-resolution texture resource data;

[0155] The processor 610 is also used to render the first scene using the super-resolution texture resource data.

[0156] Optionally, the processor 610 is further configured to perform a decompression operation on the copy texture resource data created based on the texture resource data to generate intermediate texture data;

[0157] The processor 610 is also used to process the intermediate texture data using a filtering algorithm based on image content to obtain diffuse texture resource data;

[0158] Processor 610 is also used to perform super-resolution algorithm processing on the diffuse texture resource data to generate high-resolution texture data;

[0159] The processor 610 is also used to compress the high-resolution texture data to obtain the super-resolution texture resource data.

[0160] Optionally, the copy texture resource data is in the adaptive scalable texture compression ASTC compression format;

[0161] The processor 610 is also used to perform decompression operations on the copy texture resource data in ASTC compressed format to generate intermediate texture data in RGBA format with red, green and blue transparency.

[0162] The processor 610 is also used to compress high-resolution texture data in RGBA format to ASTC format to obtain the super-resolution texture resource data.

[0163] Optionally, the processor 610 is also configured to analyze the pixel value distribution of the intermediate texture data using a filtering algorithm based on image content, and identify the texture type;

[0164] The processor 610 is further configured to determine the resource data whose texture type is diffuse texture as diffuse texture resource data.

[0165] Optionally, the processor 610 is also configured to control the rendering thread to load the texture resource data of the first scene;

[0166] The processor 610 is also used to control a worker thread to process the texture resource data based on a super-resolution algorithm to obtain the super-resolution texture resource data; the worker thread is a background thread used to process computationally intensive tasks.

[0167] The processor 610 is also used to control the rendering thread to render the first scene using the super-resolution texture resource data.

[0168] In the embodiments of this application, by loading the texture resource data of the first scene and processing the texture resource data based on a super-resolution algorithm, richer details are injected into the texture resource data to obtain super-resolution texture resource data, thus reserving high-quality materials for image quality improvement. Rendering the first scene using super-resolution texture resource data significantly improves visual elements such as the surface texture and texture clarity of objects in the scene, thereby enhancing the rendered image quality.

[0169] It should be understood that, in this embodiment, the input unit 604 may include a graphics processing unit (GPU) 6041 and a microphone 6042. The GPU 6041 processes image data of still images or video images obtained by an image capture device (such as a camera) in video image capture mode or image capture mode. The display unit 606 may include a display panel 6061, which may be configured in the form of a liquid crystal display, an organic light-emitting diode, etc. The user input unit 607 includes at least one of a touch panel 6071 and other input devices 6072. The touch panel 6071 is also called a touch screen. The touch panel 6071 may include a touch detection device and a touch controller. Other input devices 6072 may include, but are not limited to, physical keyboards, function keys (such as volume control buttons, power buttons, etc.), trackballs, mice, and joysticks, which will not be described in detail here. The memory 609 can be used to store software programs and various data, including but not limited to applications and motion systems. Processor 610 can integrate an application processor and a modem processor. The application processor mainly handles the action system, user page, and applications, while the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into processor 610.

[0170] The memory 609 can be used to store software programs and various data. The memory 609 may primarily include a first storage area for storing programs or instructions and a second storage area for storing data. The first storage area may store the operating system, application programs or instructions required for at least one function (such as sound playback, image playback, etc.). Furthermore, the memory 609 may include volatile memory or non-volatile memory, or it may include both volatile and non-volatile memory. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus RAM (DRRAM). The memory 609 in this embodiment includes, but is not limited to, these and any other suitable types of memory.

[0171] Processor 610 may include one or more processing units; optionally, processor 610 integrates an application processor and a modem processor, wherein the application processor mainly handles operations involving the operating system, user interface, and applications, and the modem processor mainly handles wireless communication signals, such as a baseband processor. It is understood that the aforementioned modem processor may also not be integrated into processor 610.

[0172] This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described rendering method embodiments and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0173] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0174] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement the various processes of the above-described rendering method embodiments and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0175] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0176] This application provides a computer program product that is stored in a storage medium and executed by at least one processor to implement the various processes of the rendering method embodiments described above, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0177] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0178] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0179] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. A rendering method, characterized in that, The method includes: Load the texture resource data for the first scene; The texture resource data is processed based on the super-resolution algorithm to obtain super-resolution texture resource data; The first scene is rendered using the super-resolution texture resource data.

2. The method according to claim 1, characterized in that, The process of processing the texture resource data based on the super-resolution algorithm to obtain super-resolution texture resource data includes: Perform a decompression operation on the copy texture resource data created based on the texture resource data to generate intermediate texture data; The intermediate texture data is processed by an image content-based filtering algorithm to obtain diffuse texture resource data; Perform super-resolution algorithm processing on the diffuse texture resource data to generate high-resolution texture data; The high-resolution texture data is compressed to obtain the super-resolution texture resource data.

3. The method according to claim 2, characterized in that, The copy texture resource data is in the adaptive scalable texture compression ASTC compression format; The step of decompressing the copy texture resource data created based on the texture resource data to generate intermediate texture data includes: Perform decompression on the ASTC compressed copy texture resource data to generate intermediate texture data in RGBA format with red, green, and blue transparency. The compression of the high-resolution texture data to obtain the super-resolution texture resource data includes: The high-resolution texture data in RGBA format is compressed into ASTC format to obtain the super-resolution texture resource data.

4. The method according to claim 2 or 3, characterized in that, The image content-based filtering algorithm processes the intermediate texture data to obtain diffuse texture resource data, including: The image content-based filtering algorithm analyzes the pixel value distribution of the intermediate texture data to identify the texture type; Resource data whose texture type is diffuse texture is identified as diffuse texture resource data.

5. The method according to any one of claims 1-4, characterized in that, The loading of texture resource data for the first scene includes: Control the rendering thread to load the texture resource data of the first scene; The process of processing the texture resource data based on the super-resolution algorithm to obtain super-resolution texture resource data includes: The control thread processes the texture resource data based on the super-resolution algorithm to obtain the super-resolution texture resource data; The rendering of the first scene using the super-resolution texture resource data includes: The rendering thread is controlled to render the first scene using the super-resolution texture resource data.

6. A rendering apparatus, characterized in that, The device includes: The loading module is used to load the texture resource data of the first scene; The processing module is used to process the texture resource data based on the super-resolution algorithm to obtain super-resolution texture resource data; The rendering module is used to render the first scene using the super-resolution texture resource data.

7. The apparatus according to claim 6, characterized in that, The processing module includes: The decompression submodule is used to perform a decompression operation on the copy texture resource data created based on the texture resource data to generate intermediate texture data; The filtering processing submodule is used to process the intermediate texture data based on the image content filtering algorithm to obtain diffuse texture resource data; The super-resolution processing submodule is used to perform super-resolution algorithm processing on the diffuse texture resource data to generate high-resolution texture data; The compression submodule is used to compress the high-resolution texture data to obtain the super-resolution texture resource data.

8. The apparatus according to claim 7, characterized in that, The copy texture resource data is in the adaptive scalable texture compression ASTC compression format; The decompression submodule is specifically used for: Perform decompression on the ASTC compressed copy texture resource data to generate intermediate texture data in RGBA format with red, green, and blue transparency. The compression submodule is specifically used for: The high-resolution texture data in RGBA format is compressed into ASTC format to obtain the super-resolution texture resource data.

9. The apparatus according to claim 7 or 8, characterized in that, The filtering submodule is specifically used for: The image content-based filtering algorithm analyzes the pixel value distribution of the intermediate texture data to identify the texture type; Resource data whose texture type is diffuse texture is identified as diffuse texture resource data.

10. The apparatus according to any one of claims 6-9, characterized in that, The loading module is specifically used for: Control the rendering thread to load the texture resource data of the first scene; The processing module is specifically used for: The control thread processes the texture resource data based on the super-resolution algorithm to obtain the super-resolution texture resource data; The rendering module is specifically used for: The rendering thread is controlled to render the first scene using the super-resolution texture resource data.

11. An electronic device, characterized in that, It includes a processor and a memory, the memory storing a program or instructions that can run on the processor, the program or instructions being executed by the processor to implement the steps of the method as described in any one of claims 1 to 5.

Images