Arbitrary-scale super-resolution methods, devices, and equipment based on compact Gaussian splashing

Abstract

This application specifically relates to an arbitrary-scale super-resolution method, apparatus, and device based on compact Gaussian splashing. The method includes: acquiring a first-resolution image and extracting features from the first-resolution image to obtain a latent code; compressing the latent code using a preset cascaded content consistency function to obtain compact Gaussian primitive parameters, and sampling the compact Gaussian primitive parameters based on a spatially aware sampling strategy to obtain sampled Gaussian primitive parameters; splashing the sampled Gaussian primitive parameters in screen space to generate screen-space latent features; and parsing the screen-space latent features to obtain a second-resolution image, wherein the resolution of the second-resolution image is higher than that of the first-resolution image. This solves the problems of low efficiency, loss of detail, and aliasing caused by redundant primitives, insufficient features, and sampling mismatch in related technologies for arbitrary-scale super-resolution tasks, thereby improving the efficiency of arbitrary-scale super-resolution.

Description

Technical Field

[0001] This application relates to the field of computer vision and image processing technology, and in particular to an arbitrary-scale super-resolution method, apparatus and device based on compact Gaussian splashing. Background Technology

[0002] Super-resolution algorithms, as a fundamental application of low-level vision tasks, reconstruct high-resolution images by mining the relationships between pixels in a low-resolution input image. In practical applications, when users zoom in to view local details, traditional fixed-scale super-resolution methods show a significant decline in detail recovery when the target is scaled down beyond a specific magnification. To address this challenge, the research goal of efficient arbitrary-scale super-resolution methods is to achieve fidelity reconstruction of local image details while flexibly scaling at any scale.

[0003] Among related technologies, implicit neural representations (INRs) achieve arbitrary-scale super-resolution through implicit continuous modeling, demonstrating significant advantages due to their ability to implicitly model and represent continuous functions; Gaussian splatting (GS) achieves fast rendering through explicit Gaussian primitives, maintaining strong and stable performance even when dealing with large scale factors.

[0004] However, methods based on implicit neural representations (such as LIIF and LTE) rely on complex decoders, have slow inference speeds, and struggle to balance detail preservation and anti-aliasing effects when magnified at large scales. Methods based on Gaussian splashing (such as GSASR and GaussianSR) require a large number of redundant Gaussian primitives to ensure texture accuracy, resulting in high computational complexity and storage costs. When the discrete sampling frequency does not match the target resolution, high-frequency signal redundancy and aliasing are prone to occur, which urgently need to be addressed. Summary of the Invention

[0005] This application provides an arbitrary-scale super-resolution method, apparatus, and device based on compact Gaussian splashing to solve problems such as slow inference speed, difficulty in balancing detail preservation and anti-aliasing effect and computational complexity in related technologies when magnified at large scales.

[0006] The first aspect of this application provides an arbitrary-scale super-resolution method based on compact Gaussian splashing, comprising the following steps:

[0007] A first-resolution image is acquired, and features of the first-resolution image are extracted to obtain the latent code;

[0008] The latent code is compressed using a preset cascaded content consistency function to obtain compact Gaussian meta-parameters, and the compact Gaussian meta-parameters are sampled based on a spatially aware sampling strategy to obtain sampled Gaussian meta-parameters.

[0009] The sampled Gaussian meta-parameters are screen-space splashed to generate screen-space latent features. The screen-space latent features are then parsed to obtain a second-resolution image, wherein the resolution of the second-resolution image is higher than that of the first-resolution image.

[0010] Optionally, the compact Gaussian parameters include: position, scale, rotation angle, and eigenvector.

[0011] Optionally, the sampling of the compact Gaussian meta-parameters based on the spatially aware sampling strategy includes:

[0012] Based on the Nyquist sampling theorem, the preset Gaussian covariance matrix is ​​controlled to match the output signal frequency with the target resolution.

[0013] A scale-aware dynamic filtering mechanism is adopted to dynamically adjust the frequency domain response threshold of Gaussian splashing based on the amplification factor.

[0014] Optionally, before controlling the preset Gaussian covariance matrix based on the Nyquist sampling theorem, the method further includes:

[0015] Based on the compact Gaussian element parameters, the preset Gaussian covariance matrix is ​​obtained.

[0016] Optionally, the cascaded content consistency function consists of multiple content consultation modules, wherein the compression of the latent code using a preset cascaded content consistency function to obtain compact Gaussian meta-parameters includes:

[0017] In the multiple content consultation modules, an expansion map and a correction map are generated based on a multilayer perceptron;

[0018] The compact Gaussian parameters are subjected to noise filtering and detail enhancement using the expansion map and the correction map.

[0019] A second aspect of this application provides an arbitrary-scale super-resolution device based on compact Gaussian splashing, comprising:

[0020] An acquisition module is used to acquire a first-resolution image and extract features from the first-resolution image to obtain a latent code;

[0021] The processing module is used to compress the latent code using a preset cascaded content consistency function to obtain compact Gaussian meta-parameters, and to sample the compact Gaussian meta-parameters based on a spatially aware sampling strategy to obtain sampled Gaussian meta-parameters.

[0022] The parsing module is used to perform screen space splashing on the sampled Gaussian parameters to generate screen space latent features, and to parse the screen space latent features to obtain a second resolution image, wherein the resolution of the second resolution image is higher than the resolution of the first resolution image.

[0023] Optionally, the compact Gaussian parameters include: position, scale, rotation angle, and eigenvector.

[0024] Optionally, the processing module is specifically used for:

[0025] Based on the Nyquist sampling theorem, the preset Gaussian covariance matrix is ​​controlled to match the output signal frequency with the target resolution.

[0026] A scale-aware dynamic filtering mechanism is adopted to dynamically adjust the frequency domain response threshold of Gaussian splashing based on the amplification factor.

[0027] Optionally, before controlling the preset Gaussian covariance matrix based on the Nyquist sampling theorem, the processing module is further configured to:

[0028] Based on the compact Gaussian element parameters, the preset Gaussian covariance matrix is ​​obtained.

[0029] Optionally, the cascaded content consistency function consists of multiple content consultation modules, wherein the compression of the latent code using a preset cascaded content consistency function to obtain compact Gaussian meta-parameters includes:

[0030] In the multiple content consultation modules, an expansion map and a correction map are generated based on a multilayer perceptron;

[0031] The compact Gaussian parameters are subjected to noise filtering and detail enhancement using the expansion map and the correction map.

[0032] A third aspect of this application provides an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being configured to perform an arbitrary-scale super-resolution method based on compact Gaussian splashing as described in the above embodiments.

[0033] A fourth aspect of this application provides a computer-readable storage medium having a computer program stored thereon, which is executed by a processor to implement the arbitrary-scale super-resolution method based on compact Gaussian splashing as described in the above embodiments.

[0034] A fifth aspect of this application provides a computer program product storing a computer program that, when executed by a processor, implements the arbitrary-scale super-resolution method based on compact Gaussian splashing as described in the above embodiments.

[0035] Therefore, this embodiment of the application acquires a first-resolution image and extracts features from the first-resolution image to obtain a latent code; it compresses the latent code using a preset cascaded content consistency function to obtain compact Gaussian meta-parameters, and samples the compact Gaussian meta-parameters based on a spatially aware sampling strategy to obtain sampled Gaussian meta-parameters; it then performs screen-space splashing on the sampled Gaussian meta-parameters to generate screen-space latent features, and analyzes the screen-space latent features to obtain a second-resolution image, wherein the resolution of the second-resolution image is higher than that of the first-resolution image. This solves the problems of low efficiency, loss of detail, and aliasing caused by redundant primitives, insufficient features, and sampling mismatch in related technologies for arbitrary-scale super-resolution tasks, thereby improving the efficiency of arbitrary-scale super-resolution.

[0036] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0037] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:

[0038] Figure 1 This is a flowchart of an arbitrary-scale super-resolution method based on compact Gaussian splashing, according to an embodiment of this application.

[0039] Figure 2 This is a schematic diagram of a content consultation module for an arbitrary-scale super-resolution method based on compact Gaussian splashing, according to an embodiment of this application.

[0040] Figure 3 This is a schematic diagram of the overall architecture of an arbitrary-scale super-resolution method based on compact Gaussian splashing according to an embodiment of this application;

[0041] Figure 4 This is a schematic diagram of an arbitrary-scale super-resolution device based on compact Gaussian splashing provided according to an embodiment of this application;

[0042] Figure 5This is a schematic diagram of the structure of an electronic device provided according to an embodiment of this application. Detailed Implementation

[0043] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.

[0044] Before introducing the arbitrary-scale super-resolution method based on compact Gaussian splashing in the embodiments of this application, let's briefly introduce the arbitrary-scale super-resolution method based on compact Gaussian splashing in related technologies.

[0045] Specifically, implicit neural representations (INRs) exhibit significant advantages due to their ability to implicitly model continuous functions. INR-based super-resolution methods aim to provide additional prior knowledge for more accurately recovering details of low-resolution pixels by modeling upsampling functions based on geometric or physical laws in the continuous world. These methods achieve controllable image reconstruction capabilities at arbitrary magnification by incorporating scale information into pixel-level coordinates. However, while maintaining strong and stable performance when handling large scale factors, the complex implicit continuous representation decoders they employ require processing a large number of independent sampling points and complex implicit representation features, which leads to reduced rendering speed when generating high-resolution images.

[0046] Furthermore, in recent years, Gaussian Splatting (GS) has attracted attention due to its fast rendering speed and high fidelity in 3D tasks. Unlike implicit neural representations, many studies explicitly learn the shape and appearance parameters of a single scene, sputter them onto a 2D plane, and efficiently utilize differentiable rasterization techniques to complete rendering with extremely short inference times. In addition, several studies have explored the application of GS in 2D image data. For example, GaussianImage uses 2D Gaussian sputtering to improve the efficiency of image representation in the color gamut. However, this method requires dense Gaussian points and redundant primitives to achieve fine texture rendering at high resolutions, which leads to a significant increase in computational complexity and storage costs. Lightweight Gaussian sputtering algorithms are also rapidly developing. These algorithms significantly reduce computational complexity and storage requirements by optimizing the distribution of Gaussian points, simplifying primitive representations, or introducing more efficient computational methods, while maintaining the advantages of Gaussian sputtering in detail recovery and rendering speed. To reduce the excessive number of Gaussian points in the generation task, LRF (Local Radiance Field) improves 3D scene reconstruction by learning spatial consistency in the latent radiation field. It effectively solves the artifact problem caused by finite discrete Gaussian points in explicit Gaussian position interpolation, which is due to the inability of Gaussian sputtering to fully model continuous spatial textures.

[0047] Furthermore, to compress redundant Gaussian features in scene reconstruction, HAC (Hierarchical Attention with Context) employs structured mutual information for scene context modeling, revealing the spatial relationships of anchor points. Then, entropy encoding is used for efficient feature quantization, thereby compressing redundant Gaussian primitives. Therefore, lightweight Gaussian sputtering shows good application potential in super-resolution tasks of arbitrary scales because Gaussian anchor points can represent spatial regions rather than single points. This makes it possible to achieve continuous sampling in local regions around the original sampling points, thus achieving compact and efficient feature representations.

[0048] However, arbitrary-scale super-resolution methods in related technologies include those based on implicit neural representations and those based on Gaussian splashing. Methods based on implicit neural representations (such as LIIF and LTE) achieve arbitrary-scale super-resolution through implicit continuous modeling, but rely on complex decoders, resulting in slow inference speeds and difficulty balancing detail preservation and anti-aliasing effects at large scales. Methods based on Gaussian splashing (such as GSASR and GaussianSR) achieve fast rendering through explicit Gaussian primitives, but require a large number of redundant Gaussian primitives to ensure texture accuracy, leading to high computational complexity and storage costs. Furthermore, when the discrete sampling frequency does not match the target resolution, high-frequency signal redundancy and aliasing are prone to occur.

[0049] This application addresses the aforementioned problems by proposing an arbitrary-scale super-resolution method based on compact Gaussian splashing. In this method, embodiments acquire a first-resolution image and extract its features to obtain a latent code. The latent code is then compressed using a pre-defined cascaded content consistency function to obtain compact Gaussian meta-parameters. These compact Gaussian meta-parameters are then sampled using a spatially aware sampling strategy to obtain sampled Gaussian meta-parameters. The sampled Gaussian meta-parameters are then splashed in screen space to generate screen-space latent features. These screen-space latent features are then analyzed to obtain a second-resolution image, where the resolution of the second-resolution image is higher than that of the first-resolution image. This solves the problems of low efficiency, detail loss, and aliasing caused by redundant primitives, insufficient features, and sampling mismatches in related technologies for arbitrary-scale super-resolution tasks, achieving a balance between image detail restoration and computational efficiency.

[0050] Specifically, Figure 1 This is a flowchart illustrating an arbitrary-scale super-resolution method based on compact Gaussian splashing, provided in an embodiment of this application.

[0051] like Figure 1 As shown, this arbitrary-scale super-resolution method based on compact Gaussian splashing includes the following steps:

[0052] In step S101, a first resolution image is acquired, and features of the first resolution image are extracted to obtain a latent code.

[0053] Among them, latent encoding is a mathematical representation of high-dimensional input data compressed into a low-dimensional latent space through a deep learning model.

[0054] Specifically, a low-quality first-resolution image (such as a blurry 640×480 image) is used as the system input. A compact Gaussian embedding is designed to optimize the Gaussian point distribution through spatial correlation. A Feature Decomposition Module (FDM) is used to extract key features of the first-resolution image to obtain a latent code, thereby reducing redundancy and improving computational efficiency.

[0055] In step S102, the latent code is compressed using a preset cascaded content consistency function to obtain compact Gaussian meta-parameters, and the compact Gaussian meta-parameters are sampled based on a spatially aware sampling strategy to obtain sampled Gaussian meta-parameters.

[0056] Optionally, in some embodiments, the compact Gaussian meta-parameters include: position, scale, rotation angle, and eigenvector.

[0057] Specifically, the Gaussian latent space dimension is compressed by using pre-defined cascaded content-consistent functions to extract compact Gaussian primitive parameters and reduce redundancy. Secondly, the spatially aware sampling strategy uses a scale-aware filter to dynamically adjust the frequency threshold of Gaussian splashing according to the target amplification factor, suppressing high-frequency redundant signals and aliasing, achieving adaptive sampling under different amplification ratios, and improving reconstruction accuracy.

[0058] Optionally, in some embodiments, the compact Gaussian meta-parameters are sampled based on a spatially aware sampling strategy, including: controlling a preset Gaussian covariance matrix based on the Nyquist sampling theorem to match the output signal frequency with the target resolution; and using a scale-aware dynamic filtering mechanism to dynamically adjust the frequency domain response threshold of the Gaussian splash based on the amplification factor.

[0059] Optionally, in some embodiments, before controlling the preset Gaussian covariance matrix based on the Nyquist sampling theorem, the method further includes: obtaining the preset Gaussian covariance matrix based on compact Gaussian element parameters.

[0060] Understandably, based on the Nyquist sampling theorem, a low-pass filter controls the Gaussian covariance matrix to ensure that the output signal frequency matches the target resolution. Simultaneously, a scale-aware dynamic filtering mechanism is employed to dynamically adjust the frequency response threshold of the Gaussian splashing operation based on the real-time input amplification factor; for example, when the amplification factor increases, the high-frequency cutoff threshold is correspondingly lowered to suppress aliasing artifacts; conversely, the frequency limit is appropriately relaxed to retain more detail.

[0061] Optionally, in some embodiments, the cascaded content consistency function consists of multiple content consultation modules, wherein the latent code is compressed using a preset cascaded content consistency function to obtain compact Gaussian meta-parameters, including: generating an expansion map and a correction map based on a multilayer perceptron in the multiple content consultation modules; and performing noise filtering and detail enhancement on the compact Gaussian meta-parameters using the expansion map and the correction map.

[0062] It is understandable that, such as Figure 2 As shown, Figure 2 This is a schematic diagram of a content consultation module for an arbitrary-scale super-resolution method based on compact Gaussian splashing, according to an embodiment of this application. The cascaded content consistency function consists of multiple content consultation modules. The nth layer CCM (content consultation module) generates an expansion map (d) through MLP. n ) and correction diagram (c n This involves noise filtering and detail enhancement of Gaussian features.

[0063] In step S103, the sampled Gaussian parameters are splashed in screen space to generate screen space latent features. The screen space latent features are then parsed to obtain a second resolution image, wherein the resolution of the second resolution image is higher than that of the first resolution image.

[0064] Among them, screen space latent features are intermediate image representations formed by projecting three-dimensional or high-dimensional features onto a two-dimensional screen coordinate system using Gaussian splashing technology.

[0065] Specifically, this invention abandons the traditional implicit decoder and directly generates a high-resolution image (second resolution image) through the latent features of the screen space after Gaussian splashing, thereby reducing computational complexity.

[0066] It should be noted that on datasets such as Set5 and Set14, CGSSR (Compact Gaussian Splatting for Super-Resolution) outperforms the comparison methods in both PSNR (Peak Signal-to-Noise Ratio) and SSIM (Structural Similarity Index Measure) (e.g., PSNR of 28.51dB at ×4 vs. 28.46dB for Gaussian SR). Peak GPU memory usage is reduced by 65%, supporting single-card (e.g., NVIDIA 3090) training and inference.

[0067] Therefore, compared with related technologies, this invention reduces computational complexity and storage requirements, improves scale generalization ability, and enhances image quality; it optimizes feature representation, enabling the model to reduce the number of parameters while maintaining performance, thereby improving computational efficiency and making the model more suitable for low-computing devices; the number of parameters is reduced by 13% to 15%, FLOPs (Floating Point Operations) are reduced by 88% to 93%, and memory usage is reduced by more than 50%; it can adapt to super-resolution tasks at different magnifications, improving the model's performance in cross-scale tasks. The reconstruction quality of this invention at scales from ×2 to ×30 shows a PSNR (Peak Signal-to-Noise Ratio) superior to the Gaussian splashing method of related technologies (e.g., an improvement of 0.27 to 0.46 dB at ×30).

[0068] To facilitate those skilled in the art to further understand the arbitrary-scale super-resolution method based on compact Gaussian splashing in the embodiments of this application, the following is combined with... Figure 3 The embodiments shown will be described in detail.

[0069] Specifically, such as Figure 3 As shown, Figure 3This is a schematic diagram of the overall architecture of an arbitrary-scale super-resolution method based on compact Gaussian splashing according to an embodiment of this application. The method involves inputting a low-resolution image, extracting features through an encoder, generating Gaussian parameters through a compact Gaussian embedding module, splashing the image onto the screen space after scale-aware filtering, and finally outputting a high-resolution image by a lightweight decoder.

[0070] Therefore, this application's embodiments employ a lightweight Gaussian modeling approach based on spatial correlation optimization for super-resolution tasks, and obtain compact Gaussian embeddings using a content consistency function cascade compression technique, reducing computational complexity and storage overhead. Secondly, a spatially aware sampling strategy utilizes a scale-aware dynamic filtering mechanism to adjust the frequency response of the Gaussian covariance matrix according to the magnification factor, achieving adaptive adjustment at different magnifications and improving super-resolution quality. Finally, images are directly generated using the latent features obtained after Gaussian splashing, and the lightweight decoding architecture avoids complex implicit decoding.

[0071] According to the arbitrary-scale super-resolution method based on compact Gaussian splashing proposed in this application, the method acquires a first-resolution image and extracts its features to obtain a latent code. The latent code is compressed using a preset cascaded content consistency function to obtain compact Gaussian meta-parameters. These compact Gaussian meta-parameters are then sampled using a spatially aware sampling strategy to obtain sampled Gaussian meta-parameters. The sampled Gaussian meta-parameters are then splashed in screen space to generate screen-space latent features. These screen-space latent features are then analyzed to obtain a second-resolution image, where the resolution of the second-resolution image is higher than that of the first-resolution image. This solves the problems of low efficiency, detail loss, and aliasing caused by redundant primitives, insufficient features, and sampling mismatch in related technologies for arbitrary-scale super-resolution tasks, achieving a balance between image detail restoration and computational efficiency.

[0072] Next, referring to the accompanying drawings, an arbitrary-scale super-resolution device based on compact Gaussian splashing proposed according to an embodiment of this application is described.

[0073] Figure 4 This is a block diagram of an arbitrary-scale super-resolution device based on compact Gaussian splashing, according to an embodiment of this application.

[0074] like Figure 4 As shown, the arbitrary-scale super-resolution device 10 based on compact Gaussian splashing includes: an acquisition module 100, a processing module 200, and a parsing module 300.

[0075] The acquisition module 100 is used to acquire a first resolution image and extract features from the first resolution image to obtain a latent code.

[0076] The processing module 200 is used to compress the latent code using a preset cascaded content consistency function to obtain compact Gaussian meta-parameters, and to sample the compact Gaussian meta-parameters based on a spatially aware sampling strategy to obtain sampled Gaussian meta-parameters.

[0077] The parsing module 300 is used to perform screen space splashing on the sampled Gaussian parameters to generate screen space latent features, and to parse the screen space latent features to obtain a second resolution image, wherein the resolution of the second resolution image is higher than that of the first resolution image.

[0078] Optionally, compact Gaussian parameters include: position, scale, rotation angle, and eigenvector.

[0079] Optionally, the processing module 200 is specifically used to: control the preset Gaussian covariance matrix based on the Nyquist sampling theorem to match the output signal frequency with the target resolution; and use a scale-aware dynamic filtering mechanism to dynamically adjust the frequency domain response threshold of the Gaussian splash based on the amplification factor.

[0080] Optionally, before controlling the preset Gaussian covariance matrix based on the Nyquist sampling theorem, the processing module 200 is further configured to: obtain the preset Gaussian covariance matrix based on the compact Gaussian element parameters.

[0081] Optionally, the cascaded content consistency function consists of multiple content consultation modules, wherein the latent code is compressed using a preset cascaded content consistency function to obtain compact Gaussian meta-parameters, including: generating an expansion map and a correction map based on a multilayer perceptron in multiple content consultation modules; and performing noise filtering and detail enhancement on the compact Gaussian meta-parameters using the expansion map and the correction map.

[0082] It should be noted that the foregoing explanation of the embodiment of the arbitrary-scale super-resolution method based on compact Gaussian splashing also applies to the arbitrary-scale super-resolution device based on compact Gaussian splashing in this embodiment, and will not be repeated here.

[0083] According to the arbitrary-scale super-resolution device based on compact Gaussian splashing proposed in this application, the embodiments of this application acquire a first-resolution image and extract its features to obtain a latent code; the latent code is compressed using a preset cascaded content consistency function to obtain compact Gaussian meta-parameters, and the compact Gaussian meta-parameters are sampled based on a spatially aware sampling strategy to obtain sampled Gaussian meta-parameters; the sampled Gaussian meta-parameters are then splashed in screen space to generate screen-space latent features, and the screen-space latent features are analyzed to obtain a second-resolution image, wherein the resolution of the second-resolution image is higher than that of the first-resolution image. This solves the problems of low efficiency, detail loss, and aliasing caused by redundant primitives, insufficient features, and sampling mismatch in related technologies for arbitrary-scale super-resolution tasks, achieving a balance between image detail restoration and computational efficiency.

[0084] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device may include:

[0085] The memory 501, the processor 502, and the computer program stored on the memory 501 and capable of running on the processor 502.

[0086] When the processor 502 executes the program, it implements the arbitrary-scale super-resolution method based on compact Gaussian splashing provided in the above embodiments.

[0087] Furthermore, electronic devices also include:

[0088] Communication interface 503 is used for communication between memory 501 and processor 502.

[0089] The memory 501 is used to store computer programs that can run on the processor 502.

[0090] The memory 501 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0091] If the memory 501, processor 502, and communication interface 503 are implemented independently, then the communication interface 503, memory 501, and processor 502 can be interconnected via a bus to complete communication between them. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of representation, Figure 5 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0092] Optionally, in a specific implementation, if the memory 501, processor 502, and communication interface 503 are integrated on a single chip, then the memory 501, processor 502, and communication interface 503 can communicate with each other through an internal interface.

[0093] Processor 502 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application.

[0094] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described arbitrary-scale super-resolution method based on compact Gaussian splashing.

[0095] This application also provides a computer program product that stores a computer program that, when executed by a processor, implements the above-described arbitrary-scale super-resolution method based on compact Gaussian splashing.

[0096] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0097] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0098] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more N executable instructions for implementing custom logic functions or processes, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of this application pertain.

[0099] It should be understood that the various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, the N steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0100] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

Claims

1. A compact Gaussian splash based arbitrary scale super-resolution method, characterized in that, Includes the following steps: A first-resolution image is acquired, and features of the first-resolution image are extracted to obtain the latent code; The latent code is compressed using a preset cascaded content consistency function to obtain compact Gaussian meta-parameters, and the compact Gaussian meta-parameters are sampled based on a spatially aware sampling strategy to obtain sampled Gaussian meta-parameters. The sampled Gaussian meta-parameters are screen-space splashed to generate screen-space latent features. The screen-space latent features are then parsed to obtain a second-resolution image, wherein the resolution of the second-resolution image is higher than that of the first-resolution image. The sampling of the compact Gaussian meta-parameters based on the spatially aware sampling strategy includes: controlling the preset Gaussian covariance matrix based on the Nyquist sampling theorem to match the output signal frequency with the target resolution; and using a scale-aware dynamic filtering mechanism to dynamically adjust the frequency domain response threshold of the Gaussian splash based on the amplification factor. Among them, the spatial sensing sampling strategy uses a scale-sensing filter to dynamically adjust the frequency threshold of Gaussian splashing according to the target amplification factor, thereby suppressing high-frequency redundant signals and aliasing. The cascaded content consistency function is composed of multiple content consultation modules. The step of compressing the latent code using a preset cascaded content consistency function to obtain compact Gaussian meta-parameters includes: generating an expansion map and a correction map based on a multilayer perceptron in the multiple content consultation modules; and performing noise filtering and detail enhancement on the compact Gaussian meta-parameters using the expansion map and the correction map.

2. The method of claim 1, wherein, The compact Gaussian meta-parameters include: position, scale, rotation angle, and eigenvector.

3. The method of claim 1, wherein, Before controlling the preset Gaussian covariance matrix based on the Nyquist sampling theorem, the following steps are also included: Based on the compact Gaussian element parameters, the preset Gaussian covariance matrix is ​​obtained.

4. An arbitrary scale super-resolution device based on compact Gaussian splash, characterized in that, include: An acquisition module is used to acquire a first-resolution image and extract features from the first-resolution image to obtain a latent code; The processing module is used to compress the latent code using a preset cascaded content consistency function to obtain compact Gaussian meta-parameters, and to sample the compact Gaussian meta-parameters based on a spatially aware sampling strategy to obtain sampled Gaussian meta-parameters. The parsing module is used to perform screen space splashing on the sampled Gaussian meta-parameters to generate screen space latent features, and to parse the screen space latent features to obtain a second resolution image, wherein the resolution of the second resolution image is higher than the resolution of the first resolution image; The sampling of the compact Gaussian meta-parameters based on the spatially aware sampling strategy includes: controlling the preset Gaussian covariance matrix based on the Nyquist sampling theorem to match the output signal frequency with the target resolution; and using a scale-aware dynamic filtering mechanism to dynamically adjust the frequency domain response threshold of the Gaussian splash based on the amplification factor. Among them, the spatial sensing sampling strategy uses a scale-sensing filter to dynamically adjust the frequency threshold of Gaussian splashing according to the target amplification factor, thereby suppressing high-frequency redundant signals and aliasing. The cascaded content consistency function is composed of multiple content consultation modules. The step of compressing the latent code using a preset cascaded content consistency function to obtain compact Gaussian meta-parameters includes: generating an expansion map and a correction map based on a multilayer perceptron in the multiple content consultation modules; and performing noise filtering and detail enhancement on the compact Gaussian meta-parameters using the expansion map and the correction map.

5. The apparatus of claim 4, wherein, The compact Gaussian meta-parameters include: position, scale, rotation angle, and eigenvector.

6. An electronic device, comprising: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, the processor executing the program to implement the arbitrary-scale super-resolution method based on compact Gaussian splashing as described in any one of claims 1-3.

7. A computer readable storage medium having stored thereon a computer program, characterized in that, The program is executed by the processor to implement the arbitrary-scale super-resolution method based on compact Gaussian splashing as described in any one of claims 1-3.

8. A computer program product storing a computer program, characterized in that, When the program is executed by the processor, it implements the arbitrary-scale super-resolution method based on compact Gaussian splashing as described in any one of claims 1-3.

Images