Texture supersampling method, apparatus, device, medium, and program product

Abstract

The present disclosure provides a texture supersampling method, a texture supersampling device, a computer device, a computer readable storage medium and a computer program product; the method comprises: in response to a texture supersampling task triggered by an application program, determining a target neural network model matched with a target super-resolution parameter of the texture supersampling task from a plurality of loaded neural network models based on the target super-resolution parameter; processing the texture supersampling task based on the target neural network model to convert a source texture corresponding to the texture supersampling task into a high-resolution texture.

Description

Technical Field

[0001] This disclosure relates to, but is not limited to, the field of image processing technology, and in particular to a texture supersampling method, a texture supersampling device, a computer device, a computer-readable storage medium, and a computer program product. Background Technology

[0002] Super-resolution (SR) technology aims to reconstruct high-resolution images from low-resolution images. Traditional methods include bilinear interpolation and Lanczos resampling. Although current mainstream super-resolution technologies are widely deployed in games and graphics applications, significant technical bottlenecks still exist in terms of cross-platform compatibility, resource efficiency, and integration complexity. Summary of the Invention

[0003] In view of the above, the present disclosure provides at least one texture supersampling method, texture supersampling device, computer device, computer-readable storage medium, and computer program product.

[0004] The technical solution of this disclosure embodiment is implemented as follows: On one hand, embodiments of this disclosure provide a texture supersampling method applied to a graphics driver in a computer device, including: In response to an application-triggered texture supersampling task, a target neural network model matching the target super-resolution parameters is determined from multiple loaded neural network models based on the target super-resolution parameters of the texture supersampling task. The target neural network model is used to process texture supersampling tasks, so as to convert the source texture corresponding to the texture supersampling task into a high-resolution texture.

[0005] On the other hand, embodiments of this disclosure provide a texture oversampling device, including: The matching module is configured to respond to a texture supersampling task triggered by the application, and to determine the executor of the target neural network model that matches the target super-resolution parameters from a plurality of loaded neural network models based on the target super-resolution parameters of the texture supersampling task. The inference module is configured to process texture supersampling tasks based on the target neural network model, in order to convert the source texture corresponding to the texture supersampling task into a high-resolution texture.

[0006] In another aspect, embodiments of this disclosure provide a computer device including a graphics driver, which stores a computer program that can run on a processor, and the graphics driver executes the computer program to implement some or all of the steps in the above-described method.

[0007] In another aspect, embodiments of this disclosure provide a computer-readable storage medium having a graphics driver stored thereon, the graphics driver being used to implement some or all of the steps in the above method.

[0008] In another aspect, embodiments of this disclosure provide a graphics driver that, when running in a computer device, allows a processor in the computer device to perform some or all of the steps in the above-described method.

[0009] In another aspect, embodiments of this disclosure provide a computer program product, which includes a non-transitory computer-readable storage medium storing a graphics driver. When the graphics driver is read and executed by a computer device, it implements some or all of the steps in the above method.

[0010] In this embodiment of the disclosure, in response to a texture supersampling task triggered by an application, a target neural network model matching the target super-resolution parameters can be automatically indexed based on the target super-resolution parameters of the texture supersampling task, thereby achieving efficient model scheduling; the texture supersampling task is processed based on the target neural network model to convert the source texture corresponding to the texture supersampling task into a high-resolution texture, thereby achieving automated texture supersampling processing; thus, by embedding the texture supersampling function into the graphics driver, the texture supersampling function can be automatically enabled when the application triggers the texture supersampling task, without the need for software development kit (SDK) integration and AI hardware support, achieving zero-intrusion integration and greatly reducing development and maintenance costs.

[0011] It should be understood that the above general description and the following detailed description are merely exemplary and explanatory, and are not intended to limit the technical solutions of this disclosure. Attached Figure Description

[0012] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the specification, serve to illustrate the technical solutions of this disclosure.

[0013] Figure 1 A schematic diagram illustrating the implementation process of a texture supersampling method provided in this embodiment of the disclosure; Figure 2 This is a schematic diagram of the composition structure of a texture supersampling system provided in an embodiment of the present disclosure; Figure 3 This is a schematic diagram of the composition structure of a texture supersampling device provided in an embodiment of the present disclosure; Figure 4 This is a schematic diagram of the hardware entity of a computer device provided in an embodiment of this disclosure. Detailed Implementation

[0014] To make the objectives, technical solutions, and advantages of this disclosure clearer, the disclosure will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this disclosure. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0015] To make the objectives, technical solutions, and advantages of this disclosure clearer, the technical solutions of this disclosure will be further described in detail below with reference to the accompanying drawings and embodiments. The described embodiments should not be regarded as limitations on this disclosure. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0016] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0017] If similar descriptions such as "first / second" appear in the application documents, the following explanation shall be added: the terms "first / second / third" are used only to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first / second / third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this disclosure described herein can be implemented in an order other than that illustrated or described herein.

[0018] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs. The terminology used herein is for descriptive purposes only and is not intended to limit this disclosure.

[0019] To better understand the texture supersampling method provided in the embodiments of this disclosure, the solutions in related technologies will be described below.

[0020] Among related technologies, Deep Learning Super Sampling (DLSS) relies on the Tensor Cores in the GeForce Ray Tracing eXtension Graphics Processing Unit (GeForce RTX GPU) for acceleration. However, Deep Learning Super Sampling has at least the following drawbacks: it only supports NVIDIA RTX series graphics cards, resulting in poor cross-platform compatibility; on older graphics cards such as the RTX 30 series, DLSS 4.5 experiences a 24% drop in frame rate and an 87-103% increase in memory usage; and it requires dedicated hardware such as Tensor Cores and optical flow accelerators. Therefore, the existing technical challenges are: how to achieve high-performance, low-resource-consumption super-sampling on general-purpose GPUs; and how to improve cross-platform compatibility.

[0021] Among related technologies, FidelityFX Super Resolution (FSR) technology reconstructs frames using information such as depth buffers and motion vectors. FidelityFX Super Resolution technology has at least the following drawbacks: it does not rely on artificial intelligence models and uses spatial or temporal algorithms, leading to blurred image details and edge distortion, especially noticeable in performance mode; it is also prone to ghosting and jitter in dynamic scenes. Therefore, the existing technical challenges are: how to introduce lightweight artificial intelligence (AI) models to improve image quality; and how to reduce motion blur and artifacts.

[0022] Among related technologies, Xe Super Sampling (XeSS) technology utilizes the Xe Matrix eXtension (XMX AI) engine to accelerate the execution of neural network inference using the core or the Dot Product 4 Accumulate (DP4a) instruction set. Xe Super Sampling technology has at least the following drawbacks: on non-Intel Graphics Processing Units (Intel GPUs) (such as NVIDIA / AMD graphics cards), it can only use the DP4a instruction set for simulation and cannot call the XMX AI core, thus limiting its performance advantage; XeSS-FG (frame generation) and XeLL (low latency) functions only support DirectX 12 and do not support DX11 and Vulkan. Based on this, the existing technical problems are: how to achieve truly unified access across APIs; and how to optimize the execution efficiency of non-native platforms.

[0023] In related technologies, HarmonyOS NEXT provides an extended interface for the OpenGL for Embedded Systems (GLES) graphics library through the XEngine Kit to achieve intelligent super-resolution. However, HarmonyOS NEXT suffers from at least the following drawbacks: it only supports specific devices equipped with specific GPU chips; and its application programming interface (API) lacks an open standard, resulting in a closed ecosystem. Therefore, the technical challenges are: how to build an open and standardized super-resolution interface; and how to achieve hardware independence and an open ecosystem.

[0024] Based on the analysis of the shortcomings of related technologies, this disclosure aims to solve the following technical problems: 1. How to provide native super-resolution functionality in the standard OpenGL API, avoiding reliance on proprietary SDKs from specific vendors.

[0025] 2. How to embed lightweight AI super-resolution models into GPU drivers to achieve unified management and efficient scheduling of models, thereby reducing application layer resource consumption.

[0026] 3. How to design a standardized extension interface that enables developers to enable super-resolution in the simplest way (single function call) and achieve "zero-intrusion" integration.

[0027] 4. How to ensure that the solution has good cross-platform compatibility and supports the unified use of GPUs from different manufacturers in the OpenGL environment.

[0028] This disclosure provides a texture supersampling method that embeds a neural network model within a graphics driver and provides standard API calls. Its core lies in deeply integrating AI super-resolution capabilities into the underlying graphics driver, forming an enhanced service transparent to upper-layer applications. This method can be applied to graphics drivers in computer devices. Here, computer devices refer to devices with data processing capabilities, such as servers, laptops, tablets, desktop computers, smart TVs, set-top boxes, and mobile devices (e.g., mobile phones, portable video players, personal digital assistants, dedicated messaging devices, portable gaming devices). Figure 1 As shown, the method includes the following steps 101 to 102: Step 101: In response to the texture supersampling task triggered by the application, determine the target neural network model that matches the target supersampling parameters from the multiple loaded neural network models based on the target super-resolution parameters of the texture supersampling task.

[0029] A graphics driver is a software layer located between the operating system and the graphics hardware (GPU). It translates instructions triggered by the application into hardware instructions that the GPU can recognize and execute, enabling graphics rendering and hardware interaction. In some implementations, the graphics driver can be an OpenGL driver. An OpenGL driver is a software program that implements the OpenGL specification and sits between the application and the GPU, translating OpenGL API instructions into hardware instructions that the GPU can recognize and execute. Of course, the graphics driver can also be other types of drivers such as DirectX drivers and Vulkan drivers; this disclosure does not limit this type.

[0030] An application (or app) is a computer program that runs on hardware devices such as computers and mobile phones to meet users' actual needs and to complete specific tasks or perform specific functions. For example, applications can be classified as sharing applications, game applications, video applications, and e-commerce applications.

[0031] Texture supersampling is used to sample multiple points of the source texture and then filter them to obtain a high-resolution texture, avoiding jagged edges, flickering and aliasing when the texture is enlarged or reduced.

[0032] Target super-resolution parameters refer to a set of super-resolution parameters used to control and adjust the super-resolution effect. Super-resolution parameters include, but are not limited to: scaling ratio, quality level, source texture, high-resolution texture, number of sampling points / sampling mode, etc.

[0033] The target neural network model is the one among multiple loaded neural network models that matches the target super-resolution parameters. Matching means that the super-resolution parameters of the neural network model are the same as the target super-resolution parameters. Neural network models include, but are not limited to: attention-based texture super-resolution models (such as EHNet) and generative texture super-resolution models (such as MSADN).

[0034] In some implementations, the user can trigger a texture supersampling task through an application; the graphics driver responds to the texture supersampling task by automatically indexing a target neural network model that matches the target super-resolution parameters based on the target super-resolution parameters of the texture supersampling task.

[0035] In some implementations, the application can trigger a texture supersampling task automatically during graphics rendering. Alternatively, the application can receive texture supersampling instructions from the user and passively trigger a texture supersampling task based on those instructions.

[0036] In some implementations, the application can trigger a texture supersampling task by using a custom supersampling function.

[0037] In some implementations, the graphics driver incorporates multiple neural network models, which can be associated with corresponding parameter combinations during storage. Thus, based on the target super-resolution parameters, a target neural network model matching the target super-resolution parameters can be determined from among the multiple neural network models.

[0038] Step 102: Process the texture supersampling task based on the target neural network model to convert the source texture corresponding to the texture supersampling task into a high-resolution texture.

[0039] In some implementations, the execution program of the target neural network model can be used to schedule the computational shaders corresponding to different model layers to process the texture supersampling task and convert the source texture corresponding to the texture supersampling task into a high-resolution texture.

[0040] In this embodiment of the disclosure, in response to a texture supersampling task triggered by an application, a target neural network model matching the target super-resolution parameters can be automatically indexed based on the target super-resolution parameters of the texture supersampling task, thereby achieving efficient model scheduling; the texture supersampling task is processed based on the target neural network model to convert the source texture corresponding to the texture supersampling task into a high-resolution texture, thereby achieving automated texture supersampling processing; thus, by embedding the texture supersampling function into the graphics driver, the texture supersampling function can be automatically enabled when the application triggers the texture supersampling task, without the need for software development kit (SDK) integration and AI hardware support, achieving zero-intrusion integration and greatly reducing development and maintenance costs.

[0041] In some embodiments, step 101 above can be implemented by the following steps 1011 to 1012: Step 1011: When the application triggers a texture supersampling task through a custom supersampling function, obtain the target super-sampling parameters passed in when the application calls the custom supersampling function; the custom supersampling function is defined based on the graphics programming interface specification of the computer device.

[0042] Custom supersampling functions are used to convert low-resolution textures (source textures) to high-resolution textures. Custom supersampling functions are defined based on the graphics programming interface specification of computer devices to achieve a calling experience consistent with the native application programming interface (API).

[0043] Custom supersampling functions are defined based on the graphics programming interface specification of computer devices. They are designed to embed texture supersampling functionality into the graphics driver to fill the gap in texture supersampling functionality in the graphics driver.

[0044] In some implementations, multiple super-sampling parameters are defined in the custom super-sampling function. In this way, when the texture super-sampling task is triggered by the custom super-sampling function, the currently passed target super-sampling parameters can be obtained.

[0045] In some implementations, the custom supersampling function can be parsed to obtain the target super-sampling parameters.

[0046] In some implementations, objects in a custom supersampling function are parsed to obtain parameter objects; target super-resolution parameters are determined based on the values ​​of the parameter objects. The parameter objects include scaling parameters (GL_MT_TEXTURE_SUPERSAMPLING) and quality level parameters (GL_MT_TEXTURE_SUPERSAMPLING_LEVEL).

[0047] For example, if the value of GL_MT_TEXTURE_SUPERSAMPLING is _2X and the value of GL_MT_TEXTURE_SUPERSAMPLING_LEVEL is _2, then the scaling factor in the target super-resolution parameters is 2x and the quality level in the target super-resolution parameters is the third quality level.

[0048] Step 1012: Based on the target super-resolution parameters, determine the target neural network model that matches the target super-resolution parameters from multiple neural network models.

[0049] In some implementations, the graphics driver incorporates multiple neural network models, which can be associated with corresponding parameter combinations during storage. Thus, based on the target super-resolution parameters, a target neural network with the same super-resolution parameters as the target can be determined from among the multiple neural networks.

[0050] In some implementations, the super-resolution parameters include multiple scaling ratios and multiple quality levels; all parameter combinations can be pre-trained with their corresponding neural network models, and each parameter combination can be associated with its corresponding neural network model and stored in the driver's internal resources; thus, the target super-resolution parameters can be used as a matching term to determine the parameter combinations that match the target super-resolution parameters, and the neural network model corresponding to the matched parameter combinations can be used as the target neural network model.

[0051] In this embodiment, by passing the target super-resolution parameters through a custom super-sampling function, texture conversion that best suits the current texture super-sampling task can be achieved. Relying on the standard graphics programming interface specification, the entire super-sampling logic is seamlessly embedded into the graphics driver and standard rendering pipeline. The neural network model can be dynamically adjusted according to the super-resolution parameters, which can balance image quality and GPU computing overhead. By generating high-definition textures in real time, the memory usage is greatly reduced. At the same time, the custom super-sampling function facilitates rapid iteration and replacement of the super-sampling logic, so as to balance visual presentation quality, program running smoothness and project development and maintenance efficiency.

[0052] In some embodiments, the custom oversampling function involved in step 1011 above can be implemented through the following steps A1 to A2: Step A1: Define a custom supersampling function based on the graphics programming interface specification; the custom supersampling function should include at least: source texture, high-resolution texture, scaling ratio, and quality level.

[0053] The graphics programming interface specification refers to the interface specification of the graphics driver. For example, if the graphics driver is an OpenGL driver, the graphics programming interface specification can be an OpenGL extension specification, such as the Khronos extension specification.

[0054] The source texture refers to the original input texture image, which is the initial data for texture super-resolution processing. The high-resolution texture refers to a texture image with a higher resolution than the source texture, which is the result of supersampling.

[0055] Scaling ratio refers to the ratio of the target texture size to the source texture size. The scaling ratio determines whether the texture is enlarged or reduced, affecting the number of sampling points and computational cost. For example, scaling ratios include, but are not limited to, 2× (times) scaling ratio and 4× scaling ratio.

[0056] Quality levels are parameters used to control the quality and performance trade-offs of texture sampling, filtering, and scaling. Quality levels are used to switch between various filtering algorithms, covering a complete quality spectrum from low computational cost / low image quality to high-precision reconstruction filtering. Filtering algorithms can include, but are not limited to: nearest-nearest, bilinear, trilinear, anisotropic filtering, etc. For example, quality levels include, but are not limited to: first quality level, second quality level, third quality level, and fourth quality level. The first quality level represents the worst quality, the second quality level represents relatively poor quality, the third quality level represents relatively good quality, and the fourth quality level represents the best quality.

[0057] In some implementations, custom oversampling functions can be manually defined based on the graphics programming interface specification. Alternatively, the graphics driver can automatically define custom oversampling functions based on the graphics programming interface specification.

[0058] In some implementations, if the graphics driver is an OpenGL driver, a custom supersampling function can be defined based on the Khronos extension specification. If the graphics driver is a Vulkan driver, a custom supersampling function can be defined based on the Vulkan extension specification. It should be noted that the graphics programming interface specification used to define the custom supersampling function differs depending on the graphics driver.

[0059] For example, a brand-new OpenGL extension, MT_texture_supersampling, is defined, which includes supersampling functions and enumeration values ​​to achieve a calling experience consistent with the native API.

[0060] Specifically, a texture oversampling operation is defined: data is read from the source texture, and the filtered high-resolution result is written to the target texture (i.e., the high-resolution texture). It supports two scaling ratios (2x and 4x) and multiple quality levels.

[0061] The following is an example of defining a custom texture supersampling function: void glTextureSuperSamplingMT( GLuint src_tex, / / Source texture name; GLuint dst_tex, / / Target texture name; GLenum scale_factor, / / Scaling ratio; GLenum quality / / Quality grade; ); Supported enumeration values ​​include: Scaling ratios: GL_MT_TEXTURE_SUPERSAMPLING_2X, L_MT_TEXTURE_SUPERSAMPLING_4X; Quality levels: GL_MT_TEXTURE_SUPERSAMPLING_LEVEL_0 (worst quality), GL_MT_TEXTURE_SUPERSAMPLING_LEVEL_1 (poor quality), GL_MT_TEXTURE_SUPERSAMPLING_LEVEL_2 (good quality), GL_MT_TEXTURE_SUPERSAMPLING_LEVEL_3 (best quality).

[0062] Function Description: glTextureSuperSamplingMT reads a source texture from a texture object named src_tex and writes the supersampling result to a texture object named dst_tex. The size of the target texture must be equal to the size of the source texture multiplied by the selected scaling factor.

[0063] The scaling factor (scale_factor) must be one of the following enumeration values: GL_MT_TEXTURE_SUPERSAMPLING_2X, GL_MT_TEXTURE_SUPERSAMPLING_4X.

[0064] The quality level must be one of the following enumeration values: GL_MT_TEXTURE_SUPERSAMPLING_LEVEL_0 to GL_MT_TEXTURE_SUPERSAMPLING_LEVEL_3.

[0065] In some implementations, if the source texture (src_tex) or target texture (dst_tex) is an invalid texture name, a warning message indicating an invalid operation (GL_INVALID_OPERATION) is generated. If the scaling factor (scale_factor) or quality level (quality) is an undefined enumeration value, a warning message indicating an invalid enumeration value (GL_INVALID_ENUM) is generated. If the size of the target texture (dst_tex) is not equal to the source texture (src_tex) multiplied by the selected scaling factor, a warning message indicating that the operation does not match the current OpenGL state (GL_INVALID_OPERATION) is generated.

[0066] Step A2: Mount the function pointer of the custom oversampling function to the function table of the graphical programming interface specification so that the application can call the custom oversampling function based on the function table.

[0067] The function table of the graphics programming interface specification is a collection of interfaces that enable graphics rendering interactions between the graphics driver, application programs, and graphics hardware. Internally, the graphics driver maintains a function table that stores the application programming interface (API) implementations. Mounting function pointers for custom supersampling functions to the function table of the graphics programming interface specification allows the rendering pipeline to call these custom supersampling functions through the function table, enabling high-quality sampling, anti-aliasing, and texture filtering.

[0068] In some implementations, the application calls the graphics programming interface (such as graphics_api->supersample(x,y)). The graphics driver uses the graphics_api pointer to determine the location of the function table in memory; in the function table, it locates the supersample member and reads its stored address value (the address of the custom supersampling function); based on the read address value, it jumps to the code segment of the custom supersampling function and executes the supersampling logic.

[0069] The following example demonstrates how to perform texture supersampling using a custom supersampling function, taking 2x resolution and the third quality level (GL_MT_TEXTURE_SUPERSAMPLING_LEVEL_2) as an example: / / Create and assign source and destination textures; GLuint srcTex, dstTex; glCreateTextures(GL_TEXTURE_2D, 1,&srcTex); glCreateTextures(GL_TEXTURE_2D, 1,&dstTex); / / Assuming the source texture size is already set (e.g., 512×512); int srcWidth = 512, srcHeight = 512; / / Allocate storage space for the source texture; glTextureStorage2D(srcTex, 1, GL_RGBA8, srcWidth, srcHeight); / / (Upload image data to srcTex as needed here...); / / Allocate storage space for the target texture: the size must be twice that of the source texture; glTextureStorage2D(dstTex, 1, GL_RGBA8, srcWidth 2, srcHeight 2); / / Call the oversampling function: 2x scaling, quality level 2 (medium quality); glTextureSuperSamplingMT( srcTex, dstTex, GL_MT_TEXTURE_SUPERSAMPLING_2X, GL_MT_TEXTURE_SUPERSAMPLING_LEVEL_2 ); / / At this point, dstTex already contains the high-resolution results after oversampling; / / Bind dstTex for subsequent rendering; glBindTextureUnit(0, dstTex); / / Draw primitives sampled using dstTex (the texture is sampled in the shader); glDrawArrays(GL_TRIANGLES, 0, 6). In this embodiment, defining a custom supersampling function based on the graphics programming interface specification is to ensure that the interface design conforms to the extension specification, guarantees compatibility with the graphics driver ecosystem, and achieves a calling experience consistent with the native API. By attaching the function pointer of the custom supersampling function to the function table of the graphics programming interface specification, applications can directly call the custom supersampling function based on the function table. This deeply integrates texture supersampling capabilities into the underlying graphics driver, forming an enhanced service transparent to upper-layer applications. It fills the gap in graphics drivers lacking native texture super-resolution processing capabilities, supports integration of graphics processors from different manufacturers in different environments, and exhibits strong cross-platform compatibility.

[0070] In some embodiments, step 102 above can be implemented by the following steps 1021 to 1022: Step 1021: Determine the execution program of the target neural network model based on the storage address of the target neural network model.

[0071] The executable program for the target neural network model refers to the executable program that runs the target neural network to implement the texture supersampling task. The executable code for the target neural network model can refer to the corresponding model information structure (modelInfo) of the target neural network model.

[0072] In some implementations, the storage address of the target neural network model stores the execution code of the target neural network model. In this way, the execution code of the target neural network model can be located directly based on the storage address of the target neural network model.

[0073] In some implementations, the storage address of the target neural network model contains a code address, which is the storage address of the executable code of the target neural network model; thus, the code address can be obtained first based on the storage address of the target neural network model, and then the executable code of the target neural network model can be located.

[0074] Step 1022: Based on the execution program of the target neural network model, schedule the computation shaders corresponding to different model layers to process the texture supersampling task, so as to convert the source texture corresponding to the texture supersampling task into a high-resolution texture.

[0075] In some implementations, kernel functions can be executed based on thread groups to trigger the execution program of the target neural network model, so as to schedule the computation shaders corresponding to different model layers to process the texture supersampling task and convert the source texture corresponding to the texture supersampling task into a high-resolution texture.

[0076] In some implementations, multiple thread groups can be scheduled, based on the execution program of the target neural network model, to schedule the computation shaders corresponding to different model layers to process the texture supersampling task and convert the source texture corresponding to the texture supersampling task into a high-resolution texture.

[0077] In this embodiment, the execution program of the target neural network model is determined based on its storage address, ensuring that the correct model code can be quickly located and loaded at runtime, avoiding the rigidity of hard coding and providing flexibility for algorithm iteration. Based on the execution program of the target neural network model, the computational shaders corresponding to different model layers are scheduled to process texture supersampling tasks, making full use of the parallel computing characteristics of the GPU. This allows for precise selection of the optimal shader variant or computational core for acceleration based on the computational density and memory access mode of different layers (such as convolutional layers, activation layers, and upsampling layers). This reduces unnecessary global memory accesses, improves on-chip cache utilization, and provides high-quality, low-latency, and scalable supersampling capabilities for upper-layer graphics applications. This decoupling of the model structure and hardware execution through dynamic binding (such as mounting function pointers to specific interfaces) enables the supersampling algorithm to flexibly adapt to the instruction sets and computing units of different hardware architectures, achieving a better dynamic balance between image quality and performance.

[0078] In some embodiments, step 1022 described above can be implemented by the following steps 1022a to 1022c: Step 1022a: Based on the mapping relationship between the number of thread groups and the texture resolution, determine the number of thread groups that match the resolution of the source texture.

[0079] The resolution of the source texture can also be referred to as the input texture resolution or low resolution.

[0080] In some implementations, the optimal number of thread groups for different texture resolutions can be determined through multiple trials. The performance of the oversampling operation is optimal with the optimal number of thread groups. The mapping relationship between different texture resolutions and their corresponding optimal number of thread groups is stored. This mapping can be done using key-value pairs. Thus, from the mapping relationship between the number of thread groups and texture resolution, the texture resolution with the same resolution as the source texture can be determined, and the optimal number of thread groups corresponding to this same texture resolution is used as the matching number of thread groups.

[0081] Step 1022b: Determine the value in the thread group execution kernel function based on the number of thread groups; the thread group execution kernel function is used to submit parallel computing tasks to the graphics hardware, and the value in the thread group execution kernel function is used to represent the number of thread groups executing in parallel.

[0082] In some implementations, the thread group execution kernel function can be Dispatch(m_threadGroupCountX, m_threadGroupCountY). Here, m_threadGroupCountX represents the number of thread groups in dimension X, and m_threadGroupCountY represents the number of thread groups in dimension Y.

[0083] Step 1022c: Based on the thread group execution kernel function, schedule the corresponding number of thread groups to execute in parallel, trigger the execution program of the target neural network model, so as to schedule the computation shaders corresponding to different model layers to process the texture supersampling task and convert the source texture corresponding to the texture supersampling task into a high-resolution texture.

[0084] In this embodiment of the disclosure, the number of thread groups matching the resolution of the source texture is determined based on the mapping relationship between the number of thread groups and the texture resolution, thus determining the optimal number of thread groups at the current time. The value in the kernel function of the thread group is determined based on the number of thread groups, and the kernel function of the thread group is executed to trigger the execution program of the target neural network model to schedule the computation shaders corresponding to different model layers to process the texture supersampling task. This is to process the supersampling task with optimal parallelism and improve the processing efficiency of the supersampling operation.

[0085] In some embodiments, the step 1022c above, "scheduling the computation shaders corresponding to different model layers to process the texture supersampling task and converting the source texture corresponding to the texture supersampling task into a high-resolution texture," can be achieved through the following steps B1 to B5: Step B1: In the feature extraction layer of the target neural network model, schedule the first computation shader to extract features from the source texture and obtain a low-level feature map.

[0086] The feature extraction layer is used to extract low-level feature maps from the source texture. The first computational shader refers to the computational shader corresponding to the feature extraction layer. The low-level feature map refers to the features extracted from the source texture.

[0087] Step B2: In the mapping layer of the target neural network model, schedule the second computation shader to perform multiple nonlinear mappings on the low-level feature maps to obtain multiple high-dimensional feature maps.

[0088] The mapping layer is used to reconstruct high-dimensional features. The second computational shader refers to the computational shader corresponding to the mapping layer. Multiple high-dimensional feature maps refer to high-dimensional features reconstructed through multiple nonlinear mappings.

[0089] In some implementations, in the mapping layer of the target neural network model, a second computation shader is scheduled to perform multiple nonlinear mappings on the low-level feature maps through multiple nonlinear layers to obtain multiple high-dimensional feature maps.

[0090] Step B3: In the aggregation layer of the target neural network model, schedule the third computation shader to aggregate multiple high-dimensional feature maps to obtain an aggregated feature map.

[0091] The aggregation layer is used to aggregate the results of multiple mapping layers, preparing for upsampling input. The third computation shader refers to the computation shader corresponding to the aggregation layer. The aggregated feature map refers to the features obtained after aggregating multiple high-dimensional feature layers.

[0092] Step B4: In the deconvolution layer of the target neural network model, schedule the fourth computation shader to deconvolve the aggregated feature map to obtain the deconvolution feature map.

[0093] The deconvolutional layer performs the deconvolution operation, providing initial upsampling. The fourth computation shader refers to the computation shader corresponding to the deconvolutional layer. The deconvolutional feature map refers to the features resulting from the deconvolution operation.

[0094] Step B5: In the pixel rearrangement layer of the target neural network model, schedule the fifth computation shader to rearrange the pixels of the deconvolution feature map to obtain a high-resolution texture.

[0095] The pixel rearrangement layer is used to generate the final high-resolution image through pixel re-detection. In this embodiment, multiple operations such as feature extraction, nonlinear mapping, aggregation, deconvolution, and pixel resampling are used to form a high-efficiency pipeline within the graphics processor, thereby converting the source texture into a high-resolution texture. The entire process is highly efficient and intelligent, greatly improving the processing speed of the oversampling operation.

[0096] In some embodiments, step B2 above can be implemented by the following steps B21 to B22: Step B21: In the mapping layer of the target neural network model, schedule the first sub-computation shader to perform dimensionality reduction on the low-level feature map to obtain the dimensionality-reduced feature map.

[0097] The first sub-computation shader can be the computation shader corresponding to the contraction layer. Dimensionality reduction is performed to reduce subsequent computation. Dimensionality-reduced feature maps refer to the features after dimensionality reduction.

[0098] In some implementations, a shrinking layer can be used to schedule the first sub-computation shader to perform dimensionality reduction on the low-level feature map, resulting in a dimensionality-reduced feature map. The shrinking layer can be part of the mapping layer or a network layer outside the mapping layer, but it can be scheduled by the mapping layer.

[0099] Step B22: Schedule the second sub-computation shader to perform multiple nonlinear mappings on the dimensionality-reduced feature map to obtain multiple high-dimensional feature maps; the second computation shader includes at least the first sub-computation shader and the second sub-computation shader.

[0100] The second sub-computation shader refers to the computation shader of the mapping layer itself. A mapping layer may include multiple non-linear mapping layers.

[0101] In some implementations, multiple nonlinear mappings can be performed on the dimensionality-reduced feature map through multiple nonlinear mapping layers to obtain multiple high-dimensional feature maps.

[0102] In this embodiment of the disclosure, the low-dimensional feature map is first reduced in dimensionality, and then multiple nonlinear mappings are performed on the reduced-dimensional feature map, which can improve the computational load of nonlinear mapping and subsequent operations and provide processing efficiency.

[0103] In some embodiments, step B4 above can be implemented by the following steps B41 to B42: Step B41: In the deconvolution layer of the target neural network model, schedule the third sub-computation shader to expand the aggregated feature map and obtain the expanded feature map.

[0104] The third sub-computation shader refers to the computation shader corresponding to the extended layer. The extended feature map refers to the extended feature map.

[0105] In some implementations, an extended layer can be used to schedule a third sub-computation shader to expand the dimensions of the aggregated feature map, resulting in an extended feature map. This mapping layer can be part of a deconvolutional layer or a network layer outside of a deconvolutional layer, but it can be scheduled by the extended layer.

[0106] Step B42: Schedule the fourth sub-computation shader to perform deconvolution on the extended feature map to obtain a convolutional feature map; the fourth computation shader includes at least the third sub-computation shader and the fourth sub-computation shader.

[0107] The fourth sub-computation shader refers to the computation shader of the deconvolution layer itself.

[0108] In this embodiment of the disclosure, the feature map dimension is expanded first, and then the deconvolution operation is performed in order to expand the feature map to a higher dimension so that upsampling can be performed in a higher-dimensional space, so that the output pixel can fuse more abstract features.

[0109] In some embodiments, the texture oversampling method provided in this disclosure further includes the following step C1: Step C1: After the computation shader of the i-th model layer is executed, based on the association between the input resources, output resources and storage buffers stored in the resource mapping structure, the shader storage buffer object output by the i-th model layer is associated with the input of the (i+1)-th model layer, so that the output resources of the i-th model layer in the shader storage buffer object are used as the input resources of the (i+1)-th model layer; the storage buffer includes shader storage buffer objects of multiple model layers; i is a positive integer.

[0110] The resource mapping structure is used to dynamically manage the correspondence between input / output resources and storage buffers (also known as mapping or binding relationships).

[0111] A model consists of multiple layers of neural networks. In implementation, each layer of the neural network corresponds to a computation shader, and the inputs and outputs of the single-layer neural network are bound to the inputs and outputs of the computation shader. For example, FSCNN has 20 layers of neural networks, so there are 20 computation shaders. Each computation shader performs a computation task for one layer, requiring the mapping of the output of the previous layer to the input of the next layer. After the computation shader of the first layer completes its execution, the output is written to the first output shader storage buffer object (SSBO). There may be multiple SSBOs to represent different types of features, depending on the complexity of the model. If the model is good and has many features, there will be more types of SSBOs. The mapping relationship between input / output resources and storage buffers (resourceMapping) is stored in the model's execution code (modelinfo). After the computation shader of the first layer completes its execution, it binds the first output SSBO to the SSBO used as the input of the next layer according to this mapping relationship. The resource mapping structure refers to the mapping relationship between input / output resources and storage buffers; the SSBO is the actual location where features are stored.

[0112] All neural network models have a mapping between input / output resources and storage buffers. This mapping is stored in the model's executable code (modelinfo). Different models have different mappings, and the input / output matching between different layers also differs. Information such as how layers should be mapped is stored in the resource mapping structure. For all models, the mapping can be set according to the resource mapping structure, and the code only needs to set it once. Although the resource mapping structure is stored in the executable code of different models, the same code can support resource mapping for all models.

[0113] In this embodiment of the disclosure, the correspondence between input / output resources and storage buffers is dynamically managed through a resource mapping structure. By storing the resource mapping structure in the execution code of different models, the resource mapping of all models can be supported with the same code, thereby improving the efficiency of resource management.

[0114] In some embodiments, the texture super-resolution method provided in this disclosure further includes the following steps D1 to D2: Step D1: Initialization phase, load multiple neural network models; different neural network models correspond to different parameter combinations; parameter combinations include quality level and scaling ratio.

[0115] Multiple neural network models are all neural network models configured for multiple parameter combinations supported by the computer device. For example, the enumerated values ​​for scaling factor include 2x and 4x, and the quality levels include four quality levels: first instruction level, second quality level, third quality level, and fourth quality level. Therefore, it is necessary to cover the combination of 2x × 4 quality levels and 4x × 4 quality levels, for a total of 8 neural network models.

[0116] In some implementations, the multiple neural network models can be pre-trained neural network models or pre-trained neural network models. For example, the neural network model can be a Fast Super-Resolution Convolutional Neural Network (FSRCNN) model or an Efficient Sub-Pixel Convolutional Neural Network (ESPCN) model. Of course, the neural network model can also be other lightweight models, which are not limited in this disclosure.

[0117] In some implementations, during the initialization phase, multiple neural network models are loaded from the internal resources of the graphics driver.

[0118] In some implementations, if the graphics driver is an OpenGL driver, the initialization phase can occur during OpenGL context creation. During OpenGL context creation, the OpenGL driver automatically loads multiple neural network models from internal resources. Internal resources refer to predefined resources that come with the driver and are stored in the driver package / graphics card firmware, rather than external resources loaded in the code.

[0119] Step D2: Reside the model parameters of multiple neural network models in the memory of the graphics processor in the computer device as weights.

[0120] In some implementations, the model parameters of each neural network model can be converted into a compute shader, compiled into static code, and then stored in the memory of the processor in the computer device, thereby enabling the model parameters of each neural network model to reside in the memory of the graphics processor in the computer device in the form of weights.

[0121] In this embodiment, during initialization, the graphics driver loads multiple neural network models simultaneously. This allows for direct model selection upon subsequent use, eliminating the need for time-consuming loading and initialization operations, thus improving the performance of the oversampling operation and enhancing the user experience. The model parameters of the neural network models reside in the graphics processor's memory as weights, managed uniformly by the driver. This avoids redundant loading by multiple application instances, significantly reducing video memory usage.

[0122] In some embodiments, the texture oversampling method provided in this disclosure further includes the following step E1: Step E1: Register a custom supersampling function based on the extended descriptor, so that the application can query the support for the custom supersampling function based on the query function corresponding to the extended descriptor, and call the custom supersampling function if it is supported.

[0123] Extended descriptors are used to register custom oversampling functions.

[0124] In some implementations, if the graphics driver is an OpenGL driver, the extension descriptor can be an OpenGL extension string, and the query function can be glGetString(GL_EXTENSIONS). In this way, a custom supersampling function (defined as the extension of the custom supersampling function, GL_MT_texture_supersampling) can be registered via the OpenGL extension string. Applications can then query support using glGetString(GL_EXTENSIONS) (the function name of the custom supersampling function) and directly call glTextureSuperSamplingMT, conforming to the Khronos standard extension specification.

[0125] In this embodiment of the disclosure, a custom supersampling function is registered using the extended descriptor mechanism, so that the application can call the supersampling function without linking additional libraries.

[0126] The following describes the application of the texture supersampling method provided in the embodiments of this disclosure in real-world scenarios.

[0127] This disclosure proposes a texture super-resolution processing method (corresponding to the texture supersampling method described above) that embeds a neural network model within an OpenGL driver and provides standard API calls. Its core lies in deeply integrating AI super-resolution capabilities into the underlying graphics driver (hereinafter referred to as the driver), forming an enhanced service transparent to upper-layer applications. This solution consists of the following key components: (a) Standardized extended API definition.

[0128] This disclosure defines a novel OpenGL extension, MT_texture_supersampling (corresponding to the aforementioned custom supersampling function), which defines the supersampling function and enumeration values ​​to achieve a calling experience consistent with the native API. Overview: This extension defines a texture supersampling operation: reading data from a source texture and writing the filtered, high-resolution result to a target texture. It supports two scaling ratios (2x and 4x) and multiple quality levels.

[0129] Add new procedures and functions: void glTextureSuperSamplingMT( GLuint src_tex, / / Source texture name; GLuint dst_tex, / / Target texture name; GLenum scale_factor, / / Scaling ratio; GLenum quality / / Quality grade; ); Supported enumeration values ​​include: Scaling ratios: GL_MT_TEXTURE_SUPERSAMPLING_2X, L_MT_TEXTURE_SUPERSAMPLING_4X; Quality levels: GL_MT_TEXTURE_SUPERSAMPLING_LEVEL_0 (worst quality), GL_MT_TEXTURE_SUPERSAMPLING_LEVEL_1 (poor quality), GL_MT_TEXTURE_SUPERSAMPLING_LEVEL_2 (good quality), GL_MT_TEXTURE_SUPERSAMPLING_LEVEL_3 (best quality).

[0130] Function Description: glTextureSuperSamplingMT reads a source texture from a texture object named src_tex and writes the supersampling result to a texture object named dst_tex. The size of the target texture must be equal to the size of the source texture multiplied by the selected scaling factor.

[0131] The scaling factor (scale_factor) must be one of the following enumeration values: GL_MT_TEXTURE_SUPERSAMPLING_2X, GL_MT_TEXTURE_SUPERSAMPLING_4X.

[0132] The quality level must be one of the following enumeration values: GL_MT_TEXTURE_SUPERSAMPLING_LEVEL_0 to GL_MT_TEXTURE_SUPERSAMPLING_LEVEL_3.

[0133] In some implementations, if the source texture (src_tex) or target texture (dst_tex) is an invalid texture name, a warning message indicating an invalid operation (GL_INVALID_OPERATION) is generated. If the scaling factor (scale_factor) or quality level (quality) is an undefined enumeration value, a warning message indicating an invalid enumeration value (GL_INVALID_ENUM) is generated. If the size of the target texture (dst_tex) is not equal to the source texture (src_tex) multiplied by the selected scaling factor, a warning message indicating that the operation does not match the current OpenGL state (GL_INVALID_OPERATION) is generated.

[0134] The following example demonstrates how to perform texture supersampling using a custom supersampling function, taking 2x resolution and the third quality level (GL_MT_TEXTURE_SUPERSAMPLING_LEVEL_2) as an example: / / Create and assign source and destination textures; GLuint srcTex, dstTex; glCreateTextures(GL_TEXTURE_2D, 1,&srcTex); glCreateTextures(GL_TEXTURE_2D, 1,&dstTex); / / Assuming the source texture size is already set (e.g., 512×512); int srcWidth = 512, srcHeight = 512; / / Allocate storage space for the source texture; glTextureStorage2D(srcTex, 1, GL_RGBA8, srcWidth, srcHeight); / / (Upload image data to srcTex as needed here...); / / Allocate storage space for the target texture: the size must be twice that of the source texture; glTextureStorage2D(dstTex, 1, GL_RGBA8, srcWidth 2, srcHeight 2); / / Call the oversampling function: 2x scaling, quality level 2 (medium quality); glTextureSuperSamplingMT( srcTex, dstTex, GL_MT_TEXTURE_SUPERSAMPLING_2X, GL_MT_TEXTURE_SUPERSAMPLING_LEVEL_2 ); / / At this point, dstTex already contains the high-resolution results after oversampling; / / Bind dstTex for subsequent rendering; glBindTextureUnit(0, dstTex); / / Draw primitives sampled using dstTex (the texture is sampled in the shader); glDrawArrays(GL_TRIANGLES, 0, 6); The interface design follows the Khronos extension specification, ensuring compatibility with the existing OpenGL ecosystem.

[0135] (ii) Driving the embedded model management mechanism.

[0136] When the OpenGL context is created, the driver automatically loads eight pre-trained FSRCNN models from internal resources, covering a combination of 2x × 4 quality levels and 4x × 4 quality levels.

[0137] Model parameters reside in GPU memory as weights and are managed uniformly by the driver, avoiding repeated loading by multiple application instances and significantly reducing GPU memory usage.

[0138] The model selection logic is jointly determined by the scaling ratio and quality level parameters, driving the execution program (modelInfo) to quickly index the corresponding model based on the input parameters (including scaling ratio and quality level).

[0139] (iii) Pipeline execution architecture based on Compute Shader.

[0140] After calling the supersampling function glTextureSuperSamplingMT (corresponding to the custom supersampling function mentioned above), the driver schedules a series of compute shaders to complete model inference, forming an efficient GPU pipeline. The functions and data transfer methods of each stage are shown in Table 1 below: Table 1

[0141] Each stage is implemented by an independent compute shader program, which is triggered to execute in parallel by Dispatch(m_threadGroupCountX, m_threadGroupCountY). The number of thread groups is dynamically calculated based on the resolution of the source texture to ensure optimal parallelism.

[0142] (iv) Resource mapping and dynamic scheduling mechanism.

[0143] The resource mapping structure is used to dynamically manage the binding relationship between input / output resources and storage buffers.

[0144] Each model layer corresponds to a compute shader program in modelInfo.pProgram[i], and resource binding is completed through UpdateResource(resourceMapping, ...).

[0145] During execution, intermediate results are efficiently exchanged between SSBOs, avoiding frequent data copying between the CPU and GPU, and ensuring that the entire process is completed within the GPU.

[0146] Specific implementation of this embodiment: Performing 2x super-resolution, quality level 2; 1. Initialization phase; When creating the GL context, the driver loads the weight data of the fsrcnn_x2_q2 model into GPU memory.

[0147] Register the function pointer of the supersampling function glTextureSuperSamplingMT to the OpenGL function table so that the application can call it directly.

[0148] 2. Invocation phase; The application executes the following code: glTextureSuperSamplingMT(src_tex_id, dst_tex_id, GL_MT_TEXTURE_SUPERSAMPLING_2X, GL_MT_TEXTURE_SUPERSAMPLING_LEVEL_2); 3. Drive the execution process; The driver parses the parameters and locates the model information structure (modelInfo) corresponding to fsrcnn_x2_q2.

[0149] Execute the Feature phase: Read the src_tex data into SSBO and run the feature compute shader.

[0150] Perform the Shrink phase (if the model configuration is enabled): Reduce the dimensionality of the feature map.

[0151] Multi-layer mapping is performed: UpdateResource and Dispatch are called in a loop to complete feature mapping layer by layer.

[0152] Perform Deconv and Pixel Shuffle: generate a high-resolution image and write it to dst_tex.

[0153] 4. Data flow summary; The data flow of the entire processing procedure is as follows; src_tex (Image) → Feature (SSBO) → Shrink (SSBO) → Mapping×N(SSBO) → Deconvolution (SSBO) → Pixel Shuffle → dst_tex (Image). All intermediate calculations are performed within the GPU without CPU intervention, ensuring low latency and high efficiency.

[0154] like Figure 2 As shown, this disclosure provides a texture supersampling system, which includes: an OpenGL context initialization stage: the driver's internal resources include 8 pre-trained model instances, including a 2x × 4 quality level FSRCNN model and a 4x × 4 quality level FSRCNN model. Model loading and registration: automatically loaded when the GL context is created; weights reside in GPU memory; registered extension function: glTextureSuperSamplingMT. Super-resolution execution flow: the application calls glTextureSuperSamplingMT(src_tex, dst_tex, scale_factor, quality). Driver scheduling: the model is selected according to the parameters; the models include: Feature Extraction, Shrink (optional), Mapping (multi-layer), MappingEnd, Expand (optional), Deconvolution, Pixel Shuffle → dst_tex; each stage is executed by an independent ComputeShader. Output results: high-resolution texture is written to dst_tex; the entire process is executed within the GPU without CPU intervention; intermediate data is transferred via SSBO. The system's data flow is from left to right and from top to bottom; the extension functions conform to the Khronos standard; and the model weights are uniformly managed by GPU memory.

[0155] The technical features, technical effects, and supporting basis of the technical solutions of this disclosure are shown in Table 2 below: Table 2

[0156] The alternatives, feasibility analysis, and conclusions of the embodiments disclosed herein are shown in Table 3 below: Table 3

[0157] This disclosure includes at least the following key points and points to be protected: The core innovation of this disclosure lies in the deep integration of lightweight AI super-resolution capabilities into the OpenGL driver layer, and the provision of transparent calling capabilities to the application layer through standardized extension interfaces. This solution overcomes the limitations of hardware binding, API fragmentation, and high integration complexity in existing technologies, forming a unified super-resolution architecture with high compatibility and low invasiveness. The following are the key technical differences of this disclosure: 1. Embed the neural network model within the OpenGL driver and load it automatically during GL context initialization. The neural network model can be a pre-trained FSRCNN model, containing 4 quality levels each at 2x and 4x magnification, for a total of 8 model instances; the model weights are uniformly loaded into GPU memory by the driver when the GL context is created, achieving cross-application sharing and zero redundant loading overhead.

[0158] 2. By defining a new OpenGL extension function glTextureSuperSamplingMT, super-resolution functionality is exposed using a standard API. This function accepts parameters such as input / output texture ID, scaling factor, and quality level, and its calling method is consistent with native OpenGL functions; it does not require linking to vendor-specific SDKs, achieving "zero-intrusion" integration and significantly reducing the barrier to entry for developers.

[0159] 3. Supports a unified management and scheduling mechanism for eight pre-trained models across two scaling ratios (2x / 4x) and four quality levels. The driver dynamically selects the computation shader program chain for the corresponding model based on a joint index of parameters such as scaling ratio and quality level; all models share the same execution framework, enabling configurable scheduling and resource reuse.

[0160] 4. A compute shader pipeline architecture is adopted, in which intermediate data is passed between model layers via SSBO and finally written to the target texture. Model inference is decomposed into stages such as Feature Extraction, Mapping, and Deconvolution, each stage is executed by an independent compute shader, and intermediate features are efficiently passed through SSBO to avoid data copying between the CPU and GPU, ensuring that the entire process is completed within the GPU.

[0161] 5. Register custom supersampling functions using the extension descriptor mechanism, allowing applications to call super-sampling functions without linking additional libraries. Register GL_MT_texture_supersampling through the OpenGL extension descriptor (extension string). Applications can query support using glGetString(GL_EXTENSIONS) and directly call glTextureSuperSamplingMT, conforming to the Khronos standard extension specification.

[0162] Based on the technical features of the embodiments disclosed herein, Intel XeSS technology is the closest prior art. The reasons are as follows: both are AI-driven super-resolution technologies that use neural network models for image reconstruction; both support cross-vendor GPU operation (Intel XeSS can run on NVIDIA / AMD graphics cards); both involve graphics API integration, aiming to improve rendering efficiency; both attempt to implement AI inference on non-native hardware (XeSS uses DP4a to simulate XMX); and both focus on driver-level optimization to reduce application invasiveness.

[0163] Nevertheless, the embodiments of this disclosure differ significantly in terms of technical implementation path, system architecture, and ecosystem openness, thereby bringing a series of outstanding technical advantages. The core technical innovations of the embodiments of this disclosure are compared with related technologies in Table 4 below: Table 4

[0164] (II) Technical advantages of the embodiments disclosed herein: Based on the aforementioned distinguishing features, the embodiments disclosed herein have the following significant technical advantages compared to the closest prior art (Intel XeSS): 1. Achieve true cross-API compatibility; XeSS is primarily bound to the DirectX 12 environment and does not support OpenGL, limiting its use in many traditional OpenGL-based graphics applications, embedded systems, and cross-platform engines (such as some configurations of Unity and Unreal Engine). In contrast, the embodiments disclosed in this disclosure directly implement super-resolution functionality within the OpenGL framework, filling a technological gap in this ecosystem and significantly improving the applicability of the technical solution.

[0165] 2. Completely eliminate reliance on dedicated AI hardware; On non-Intel GPUs, XeSS can only be emulated via the DP4a instruction and cannot call the XMX core, resulting in a significant performance drop. The embodiments disclosed herein adopt a general-purpose parallel computing architecture based on Compute Shader, which only relies on standard GPU computing units and can run on various GPUs such as AMD, NVIDIA, and Intel with similar performance, truly achieving "develop once, run anywhere".

[0166] 3. Significantly reduces application layer resource consumption and initialization overhead; XeSS requires each application to load models independently or rely on runtime library management, which can easily lead to duplicate memory usage. This disclosed embodiment loads eight pre-trained FSRCNN models (covering 2x / 4x scaling and 4 quality levels) uniformly by the driver when the GLcontext is created, achieving model sharing and zero duplicate loading, effectively reducing memory usage and startup latency.

[0167] 4. Provide standardized, low-intrusion API interfaces; Compared to XeSS's highly intrusive integration method, which requires integrating a specific SDK and modifying the rendering process, this disclosure defines the standard extension function glTextureSuperSamplingMT, which is called in the same way as the native OpenGL function. Developers only need one line of code to enable the super-resolution function, achieving "zero-intrusion" integration and greatly reducing development and maintenance costs.

[0168] 5. Build an open and scalable technological ecosystem; While XeSS is partially open source, it is still dominated by Intel, posing a risk of a closed ecosystem. The design philosophy of this disclosed embodiment aligns with the open standard spirit promoted by the Khronos Group, and its extension mechanism can be widely adopted by the industry, potentially becoming a universal AI enhancement interface standard in the future OpenGL ecosystem.

[0169] 6. Optimize execution efficiency and data flow management; This disclosure employs an SSBO-based intermediate data transfer mechanism to achieve efficient intra-GPU data exchange between Feature Extraction, Mapping, and Deconvolution stages, avoiding frequent CPU-GPU synchronization and memory copying, and ensuring low-latency execution throughout the entire process. In contrast, XeSS's execution flow is constrained by runtime scheduling strategies, resulting in lower flexibility.

[0170] (III) The overall technological advantages are summarized in Table 5 below: Table 5

[0171] Based on the foregoing embodiments, this disclosure provides a texture supersampling device, which includes the included units and the modules included in each unit, and can be implemented by a processor (such as a GPU) in a computer device; of course, it can also be implemented by specific logic circuits.

[0172] Figure 3 This is a schematic diagram of the composition structure of a texture supersampling device provided in an embodiment of this disclosure, as shown below. Figure 3 As shown, the texture oversampling device 300 includes: a matching module 310 and an inference module 320, wherein: Matching module 310 is configured to respond to a texture supersampling task triggered by an application, and to determine a target neural network model that matches the target supersampling parameters from a plurality of loaded neural network models based on the target supersampling parameters of the texture supersampling task; the custom supersampling function is defined based on the graphical programming interface specification of the computer device. The inference module 320 is configured to process the texture supersampling task based on the target neural network model, so as to convert the source texture corresponding to the texture supersampling task into a high-resolution texture.

[0173] In some embodiments, the matching module 310 is further configured to: obtain the target super-sampling parameters passed in when the application calls the custom super-sampling function, when the application triggers the texture super-sampling task through the custom super-sampling function; the custom super-sampling function is defined based on the graphics programming interface specification of the computer device; and determine the target neural network model that matches the target super-sampling parameters from multiple neural network models based on the target super-sampling parameters.

[0174] In some embodiments, the matching module 310 is further configured to: define a custom supersampling function based on the graphics programming interface specification; the parameters of the custom supersampling function include at least: source texture identifier, high-resolution texture identifier, scaling ratio, and quality level; and mount the function pointer of the custom supersampling function to the function table of the graphics programming interface specification so that the application can call the custom supersampling function based on the function table; the function table of the graphics programming interface specification is a set of interfaces that realize the graphics rendering interaction between the graphics driver, the application, and the graphics hardware.

[0175] In some embodiments, the inference module 320 is further configured to: determine the execution program of the target neural network model based on the storage address of the target neural network model; and, based on the execution program of the target neural network model, schedule the computation shaders corresponding to different model layers to process the texture supersampling task, so as to convert the source texture corresponding to the texture supersampling task into a high-resolution texture.

[0176] In some embodiments, the inference module 320 is further configured to: determine the number of thread groups matching the resolution of the source texture based on the mapping relationship between the number of thread groups and the texture resolution; determine the value in the thread group execution kernel function based on the number of thread groups; the thread group execution kernel function is used to submit parallel computing tasks to the graphics hardware, and the value in the thread group execution kernel function is used to characterize the number of thread groups executing in parallel; schedule the corresponding number of thread groups to execute in parallel based on the thread group execution kernel function, trigger the execution program of the target neural network model to run, so as to schedule the computation shaders corresponding to different model layers to process the texture supersampling task and convert the source texture corresponding to the texture supersampling task into a high-resolution texture.

[0177] In some embodiments, the inference module 320 is further configured to: after the computation shader of the i-th model layer has been executed, based on the association between the input resources, output resources and storage buffers stored in the resource mapping structure, associate the shader storage buffer object output by the i-th model layer with the input of the (i+1)-th model layer, so as to use the output resources of the i-th model layer in the shader storage buffer object as the input resources of the (i+1)-th model layer; the storage buffer includes shader storage buffer objects of multiple model layers; i is a positive integer.

[0178] In some embodiments, the texture supersampling device further includes: a loading module configured to: in an initialization phase, load multiple neural network models; different neural network models correspond to different parameter combinations, the parameter combinations including quality level and scaling ratio; and store the model parameters of the multiple neural network models in the memory of the graphics processor in the computer device in the form of weights.

[0179] In some embodiments, the texture supersampling apparatus further includes a registration module configured to register a custom supersampling function based on an extended descriptor, so that an application queries the support for the custom supersampling function based on a query function corresponding to the extended descriptor, and calls the custom supersampling function if it is supported.

[0180] The descriptions of the apparatus embodiments above are similar to those of the method embodiments above, and have similar beneficial effects. In some embodiments, the functions or modules included in the apparatus provided in this disclosure can be used to perform the methods described in the method embodiments above. For technical details not disclosed in the apparatus embodiments of this disclosure, please refer to the descriptions of the method embodiments of this disclosure for understanding.

[0181] It should be noted that, in the embodiments of this disclosure, if the above-described texture supersampling method is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiments of this disclosure, or the part that contributes to related technologies, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods of the various embodiments of this disclosure. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, external hard drive, read-only memory (ROM), magnetic disk, or optical disk. Thus, the embodiments of this disclosure are not limited to any specific hardware, software, or firmware, or any combination of hardware, software, and firmware.

[0182] This disclosure provides a computer device including a memory and a processor. The memory stores a graphics driver that can run on the processor. The graphics driver is used to implement some or all of the steps in the above method.

[0183] This disclosure provides a computer-readable storage medium storing a graphics driver, which is used to implement some or all of the steps in the above-described method. The computer-readable storage medium can be transient or non-transient.

[0184] This disclosure provides a graphics driver in which, when the graphics driver is running in a computer device, the processor in the computer device performs some or all of the steps for implementing the above-described method.

[0185] This disclosure provides a computer program product including a non-transitory computer-readable storage medium storing a graphics driver. When the graphics driver is read and executed by a computer, it implements some or all of the steps in the above-described method. This computer program product can be implemented specifically through hardware, software, or a combination thereof. In some embodiments, the computer program product is specifically embodied as a computer storage medium; in other embodiments, the computer program product is specifically embodied as a software product, such as a software development kit (SDK), etc.

[0186] It should be noted that the descriptions of the various embodiments above tend to emphasize the differences between them, while their similarities or commonalities can be referenced interchangeably. The descriptions of the above embodiments of the device, storage medium, computer program, and computer program product are similar to the descriptions of the above method embodiments and have similar beneficial effects. For technical details not disclosed in the embodiments of the device, storage medium, computer program, and computer program product of this disclosure, please refer to the descriptions of the method embodiments of this disclosure for understanding.

[0187] It should be noted that, Figure 4 This is a schematic diagram of a hardware entity of a computer device in an embodiment of this disclosure, such as... Figure 4 As shown, the hardware entity of the computer device 400 includes: a processor 401, a communication interface 402, and a memory 403, wherein: The processor 401 is used to control the overall operation of the computer device 400.

[0188] Communication interface 402 enables computer devices to communicate with other terminals or servers via a network.

[0189] The memory 403 is configured to store instructions and applications executable by the processor 401, and can also cache data to be processed or already processed (e.g., image data, audio data, voice communication data, and video communication data) in the processor 401 and various modules in the computer device 400. It can be implemented using flash memory or random access memory (RAM). Data transfer between the processor 401, the communication interface 402, and the memory 403 can be performed via bus 404.

[0190] It should be understood that the phrase "an embodiment" or "one embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of this disclosure. Therefore, "in one embodiment" or "one embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. It should be understood that in the various embodiments of this disclosure, the sequence numbers of the above steps / processes do not imply a sequential order of execution; the execution order of each step / process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this disclosure. The sequence numbers of the above embodiments of this disclosure are merely descriptive and do not represent the superiority or inferiority of the embodiments.

[0191] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0192] In the several embodiments provided in this disclosure, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components may be combined, or integrated into another system, or some features may be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed may be through some interfaces, and the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0193] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units. They may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.

[0194] In addition, each functional unit in the various embodiments of this disclosure can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0195] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media that can store program code, such as mobile storage devices, read-only memory (ROM), magnetic disks, or optical disks.

[0196] Alternatively, if the integrated units described above are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this disclosure, or the part that contributes to related technologies, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods of the various embodiments of this disclosure. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROMs, magnetic disks, or optical disks.

[0197] The above are merely embodiments of this disclosure, but the scope of protection of this disclosure is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A texture supersampling method, characterized in that, The method, applied to a graphics driver in a computer device, includes: In response to a texture supersampling task triggered by an application through a custom supersampling function, the target super-sampling parameters passed in by the application when calling the custom supersampling function are obtained; the custom supersampling function is defined based on the graphics programming interface specification of the computer device. Based on the target super-resolution parameters, a target neural network model matching the target super-resolution parameters is determined from multiple loaded neural network models; The texture supersampling task is processed based on the target neural network model to convert the source texture corresponding to the texture supersampling task into a high-resolution texture.

2. The method according to claim 1, characterized in that, The method further includes: Based on the aforementioned graphics programming interface specification, a custom supersampling function is defined; the parameters of the custom supersampling function include at least: source texture identifier, high-resolution texture identifier, scaling ratio, and quality level; The function pointer of the custom oversampling function is attached to the function table of the graphics programming interface specification, so that the application can call the custom oversampling function based on the function table; the function table of the graphics programming interface specification is a set of interfaces that realize the graphics rendering interaction between the graphics driver, the application, and the graphics hardware.

3. The method according to claim 1 or 2, characterized in that, The process of processing the texture supersampling task based on the target neural network model to convert the source texture corresponding to the texture supersampling task into a high-resolution texture includes: Based on the storage address of the target neural network model, determine the executable program of the target neural network model; The execution program based on the target neural network model schedules the computation shaders corresponding to different model layers to process the texture supersampling task, so as to convert the source texture corresponding to the texture supersampling task into a high-resolution texture.

4. The method according to claim 3, characterized in that, The execution program based on the target neural network model schedules the computation shaders corresponding to different model layers to process the texture supersampling task, so as to convert the source texture corresponding to the texture supersampling task into a high-resolution texture, including: Based on the mapping relationship between the number of thread groups and the texture resolution, determine the number of thread groups that match the resolution of the source texture; Based on the number of thread groups, the value in the thread group execution kernel function is determined; the thread group execution kernel function is used to submit parallel computing tasks to the graphics hardware, and the value in the thread group execution kernel function is used to characterize the number of thread groups executing in parallel; Based on the kernel function of the thread group execution, a corresponding number of thread groups are scheduled to execute in parallel, triggering the execution program of the target neural network model to run, so as to schedule the computation shaders corresponding to different model layers to process the texture supersampling task and convert the source texture corresponding to the texture supersampling task into a high-resolution texture.

5. The method according to claim 4, characterized in that, The method further includes: After the computation shader of the i-th model layer has been executed, based on the association between the input resources, output resources and storage buffers stored in the resource mapping structure, the shader storage buffer object output by the i-th model layer is associated with the input of the (i+1)-th model layer, so that the output resources of the i-th model layer in the shader storage buffer object are used as the input resources of the (i+1)-th model layer; the storage buffer includes shader storage buffer objects of multiple model layers; i is a positive integer.

6. The method according to claim 1 or 2, characterized in that, The method further includes: During the initialization phase, multiple neural network models are loaded; different neural network models correspond to different parameter combinations, which include quality level and scaling ratio. The model parameters of the multiple neural network models reside in the memory of the graphics processor in the computer device in the form of weights.

7. The method according to claim 1 or 2, characterized in that, The method further includes: Based on the extended descriptor, the custom supersampling function is registered so that the application queries the support for the custom supersampling function based on the query function corresponding to the extended descriptor, and calls the custom supersampling function if it is supported.

8. A texture oversampling device, characterized in that, The device includes: The matching module is configured to, in response to a texture supersampling task triggered by an application through a custom supersampling function, obtain the target super-sampling parameters passed in by the application when calling the custom supersampling function; the custom supersampling function is defined based on the graphics programming interface specification of the computer device. Based on the target super-resolution parameters, a target neural network model matching the target super-resolution parameters is determined from multiple loaded neural network models; The supersampling module is configured to process the texture supersampling task based on the target neural network model to convert the source texture corresponding to the texture supersampling task into a high-resolution texture.

9. A computer device, characterized in that, The computer device includes a memory and a processor, the memory storing a graphics driver executable on the processor, the graphics driver being used to implement the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing a graphics driver thereon, characterized in that, The graphics driver is used to implement the steps of the method according to any one of claims 1 to 7.

11. A computer program product, characterized in that, The computer program product includes a non-transitory computer-readable storage medium storing a graphics driver, which, when read and executed by a computer device, implements the steps of the method according to any one of claims 1 to 7.

Images