Image adaptive upsampling method based on semantic guidance
By employing a feature point description method based on local depth convolution features, combined with channel attention and semantic-guided branching, the problem of insufficient semantic awareness and geometric adaptability in infrared image upsampling is solved, achieving efficient and robust feature point matching and detail preservation.
Patent Information
- Application Number
- CN202511128610.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-08-13
- Publication Date
- 2025-11-14
AI Technical Summary
Existing image upsampling methods suffer from insufficient semantic perception, limited geometric adaptability, and difficulty in balancing computational efficiency and quality, resulting in problems such as blurred textures, jagged edges, and loss of details, which are particularly evident in infrared image processing.
We employ a feature point description method based on local depth convolution features. Through channel attention enhancement, semantic guidance branch to generate dynamic semantic kernel weights, and adaptive sampling branch to generate position offsets, combined with a depth feature refinement flow, we achieve efficient image upsampling.
It improves the robustness and accuracy of infrared image feature matching, achieving high-precision feature point matching in low-contrast, weak-texture, and complex lighting environments, reducing computational complexity, and maintaining image details and structural integrity.
Smart Images
Figure FT_1 
Figure FT_2 
Figure FT_3
Abstract
Description
Technical Field
[0001] This invention belongs to the field of computer vision and image processing technology, specifically relating to a semantically guided adaptive image upsampling method. This method can be widely applied to upsampling operations in convolutional neural networks, image super-resolution, video frame interpolation, medical image reconstruction, remote sensing image enhancement, and other fields, and is particularly suitable for image processing tasks requiring high detail preservation and robustness to geometric deformation. Background Technology
[0002] Image upsampling is a core fundamental operation in computer vision, widely used in super-resolution reconstruction, video frame interpolation, and medical image analysis. While traditional interpolation methods (such as bilinear and bicubic interpolation) are computationally efficient, they can lead to severe texture blurring and jagged edges. Deep learning-based upsampling methods (such as transposed convolution and subpixel convolution) can generate clearer textures, but they still face three key bottlenecks: insufficient semantic awareness, making it impossible to dynamically adjust feature aggregation strategies based on image content; limited geometric adaptability, lacking robustness to non-rigid deformations; and difficulty in balancing computational efficiency and quality, resulting in excessive memory overhead during high-resolution processing.
[0003] In recent years, methods combining semantic guidance and geometric adaptation have become a research hotspot, but existing solutions still have significant limitations. While deformable convolution (DCNv2) can learn sampling offsets, the lack of constraints on the offset range leads to sampling point overflow, and it does not consider the offset differences between different semantic regions. Attention-based methods (RCAN, SAN) incur significant computational overhead, with global attention on high-resolution feature maps resulting in quadratic memory increases. The feature reconstruction scheme (CARAFE) is computationally burdensome when handling large-kernel neighborhood aggregation. More critically, current methods generally separate semantic understanding from geometric sampling, leading to the accumulation of feature alignment errors and resulting in loss of detail and structural distortion in the reconstruction of complex structures such as vascular networks in medical images and road grids in remote sensing images. Therefore, there is an urgent need for an image upsampling method that can collaboratively handle semantic perception and geometric adaptation while optimizing computational efficiency. Summary of the Invention
[0004] The purpose of this invention is to provide a feature point description method that is suitable for infrared images, has good feature matching effect, and is highly robust, so as to solve the problems of poor adaptability and insufficient robustness of existing technologies for infrared image feature point description. It can still achieve high-precision feature point matching in low contrast, weak texture and complex lighting environments.
[0005] To achieve the above objectives, this invention provides a method for describing infrared image feature points based on local depth convolution features, comprising the following steps:
[0006] S1: Input the image feature map to be upsampled, the feature map has dimensions of B×C×H×W, where B is the batch size, C is the number of channels, H is the height, and W is the width;
[0007] S2: Channel attention enhancement is applied to the input features to increase the response weights of important feature channels;
[0008] S3: Generate dynamic semantic kernel weights through semantic guidance branches for subsequent neighborhood feature aggregation;
[0009] S4: Generate position offset and range factor through adaptive sampling branch to construct sampling grid;
[0010] S5: Use a sampling grid to perform bilinear sampling on the input features to obtain presampled features;
[0011] S6: Efficient neighborhood aggregation of presampled features based on dynamic semantic kernel weights;
[0012] S7: Enhance aggregated features through deep feature refinement stream and output upsampled feature map;
[0013] A further technical solution is that, in S2, the channel attention enhancement includes the following steps:
[0014] Perform global average pooling on the input feature map to obtain channel-level statistical features;
[0015] The channel dependencies are learned by two 1×1 convolutional layers. The first convolutional layer compresses the number of channels by a reduction factor, and the second layer restores the original number of channels.
[0016] Use the Sigmoid activation function to generate a channel weight map;
[0017] The formula for enhancing channel attention is as follows:
[0018]
[0019] in, For the input feature map, This indicates channel-by-channel multiplication. For the Sigmoid function, and It is a 1×1 convolutional layer, and reduction is the channel compression ratio parameter.
[0020] A further technical solution is that, in S3, the semantic guidance branch includes the following steps:
[0021] The features enhanced by channel attention are compressed using 1×1 convolution with a compression ratio of 1 / 4.
[0022] The compressed features are concatenated with the original input features along the channel dimension;
[0023] The concatenated features are processed by two consecutive 3×3 convolutional layers. The number of output channels in the first layer is C / 2, and the number of output channels in the second layer is groups×kernel_size².
[0024] The output features are subjected to Softmax normalization to generate dynamic semantic kernel weights;
[0025] The formula for normalizing semantic kernel weights is as follows:
[0026]
[0027] in, For batch indexing, For grouped indexes, For kernel element index, For spatial location, It is kernel_size.
[0028] A further technical solution is that, in S4, the adaptive sampling branch includes the following steps:
[0029] Spatial features are extracted using a 3×3 convolutional layer, with a total output channel count of C / 2.
[0030] Use the ReLU activation function to enhance nonlinear expressiveness;
[0031] The joint offset parameters are generated through a 1×1 convolutional layer, and the number of output channels is 3×groups×scale².
[0032] The Tanh activation function is applied to constrain the position offset within the range of [-1, 1].
[0033] The joint parameters are split into two parts: position offset and range factor.
[0034] Multiply the position offset by the range factor after Sigmoid activation;
[0035] The final sampling points are constructed by overlaying a pre-defined base grid.
[0036] The coordinates of the sampling points are normalized to the range of [-1, 1].
[0037] A further technical solution is that the method for generating the basic mesh includes the following steps:
[0038] Generate a uniformly distributed array of sampling points in both the horizontal and vertical directions;
[0039] The formula for calculating the basic grid coordinates is as follows:
[0040]
[0041]
[0042] in, Function generation from arrive The arithmetic sequence with a step size of 2 eventually forms a basic grid tensor of shape [1,2,scale,scale,1,1].
[0043] A further technical solution is characterized in that the coordinate normalization process includes the following steps:
[0044] Transform the sampling point coordinates from the image pixel coordinate system to the normalized coordinate system;
[0045] The coordinate normalization formula is as follows:
[0046]
[0047] in, These are the original coordinate values. For the corresponding dimension (width or height).
[0048] A further technical solution is that, in S5, the bilinear sampling includes the following steps: copying the input feature map into groups of copies according to the channel dimension;
[0049] Implement differentiable bilinear interpolation using PyTorch's grid_sample function;
[0050] The sampling formula is as follows:
[0051]
[0052] The sampling mode is bilinear interpolation, and the edge processing uses border padding.
[0053] A further technical solution is that, in S6, the efficient neighborhood aggregation includes:
[0054] The presampled features are divided into groups according to the channel dimension;
[0055] The semantic kernel weights are reorganized into convolutional kernels.
[0056] Implement variable kernel weighted aggregation using grouped depthwise convolution;
[0057] The aggregation operation formula is as follows:
[0058]
[0059] in, For kernel_size, The operation extracts local neighborhood feature blocks.
[0060] A further technical solution is that, in S7, the deep feature refining flow includes the following steps:
[0061] The first depthwise separable convolutional layer is used for spatial feature extraction;
[0062] Batch normalization layer standardized feature distribution;
[0063] The GELU activation function enhances nonlinearity;
[0064] The channel attention module recalibrates the channel weights;
[0065] The second deep separable convolutional layer deepens feature extraction;
[0066] 1×1 convolutional layers integrate channel information;
[0067] Residual connections preserve the original feature information;
[0068] The refining formula is as follows:
[0069]
[0070] in, This represents depthwise separable convolution. Indicates channel attention. This indicates batch normalization.
[0071] A further technical solution is that the upsampling factor scale, the number of feature groups groups, and the neighborhood aggregation kernel size kernel_size of the method are configurable parameters, wherein the value of scale is an integer from 2 to 8, the value of groups is a positive integer from 1 to the number of channels C, kernel_size is an odd number greater than or equal to 3, and the channel compression ratio reduction is an integer from 4 to 16. Attached Figure Description
[0072] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0073] Figure 1 This is a schematic diagram illustrating the steps of a semantically guided adaptive upsampling method for images provided by the present invention.
[0074] Figure 2 This is a schematic diagram of the processing flow of a semantically guided adaptive upsampling method for images provided by the present invention.
[0075] Figure 3 This is a schematic diagram of the neighborhood aggregation module provided by the present invention.
[0076] Figure 4 This is a schematic diagram of the feature refining module provided by the present invention. Detailed Implementation
[0077] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.
[0078] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Unless otherwise specified, the following embodiments and features can be combined with each other.
[0079] The accompanying drawings are for illustrative purposes only and are schematic diagrams, not actual pictures. They should not be construed as limiting the invention. To better illustrate the embodiments of the invention, some parts in the drawings may be omitted, enlarged, or reduced, and do not represent the actual product dimensions. It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.
[0080] In the accompanying drawings of the embodiments of the present invention, the same or similar reference numerals correspond to the same or similar components. In the description of the present invention, it should be understood that if terms such as "upper," "lower," "left," "right," "front," and "rear" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, they are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, the terms describing positional relationships in the drawings are for illustrative purposes only and should not be construed as limiting the present invention. Those skilled in the art can understand the specific meaning of the above terms according to the specific circumstances. In the description of the present invention, "a plurality of" means two or more, unless otherwise explicitly specified.
[0081] Please see Figures 1-4 This invention provides a semantically guided image adaptive upsampling method, comprising the following steps:
[0082] (1) Input the image feature map to be upsampled. The feature map has dimensions of B×C×H×W, where B is the batch size, C is the number of channels, H is the height, and W is the width.
[0083] (2) Enhance channel attention on input features to increase the response weights of important feature channels;
[0084] (3) Generate dynamic semantic kernel weights through semantic guidance branches for subsequent neighborhood feature aggregation;
[0085] (4) The sampling grid is constructed by generating position offset and range factor through adaptive sampling branch;
[0086] (5) Use the sampling grid to perform bilinear sampling on the input features to obtain presampled features;
[0087] (6) Efficient neighborhood aggregation of presampled features based on dynamic semantic kernel weights;
[0088] (7) Enhance the aggregated features through deep feature refinement stream and output upsampled feature map;
[0089] In a preferred embodiment of the present invention, step (1) of the input feature map processing includes the following steps:
[0090] Step (1-1): Obtain the input feature map from the deep feature extraction layer of the convolutional neural network. This feature map contains high-level semantic information of the image. During loading, it is necessary to verify that the tensor dimension meets the processing requirements to ensure that subsequent operations can be executed normally.
[0091]
[0092] in, Batch size (at least 1). The number of channels (it is recommended to have at least 16 to ensure feature representation capability). The height of the feature map (in pixels). The width of the feature map is in pixels, and the data storage format uses 32-bit floating-point numbers to ensure calculation accuracy.
[0093] Steps (1-2): Configure the upsampling parameter system according to the target application scenario. These parameters control the detailed reconstruction quality and computational complexity of the upsampling process and need to be set reasonably to balance performance and efficiency.
[0094]
[0095] in, This is the upsampling factor (an integer ranging from 2 to 8, typically 2 or 4). The number of feature groups (an integer between 1 and one-quarter of the number of channels). The size of the neighborhood aggregation kernel (it is recommended to choose an odd number such as 3, 5 or 7). Set the channel compression ratio (4, 8, or 16 are recommended).
[0096] In a preferred embodiment of the present invention, in step (2), the channel attention enhancement includes the following steps:
[0097] Step (2-1): Perform a global average pooling operation, which compresses the feature map along the spatial dimension and extracts the global statistical features of each channel, providing a basis for subsequent channel importance evaluation.
[0098]
[0099] in, Indicates the first Global eigenvalues of each channel ( From 1 to After calculation, the resulting dimension is The channel statistical vector.
[0100] Step (2-2): A bottleneck structure is constructed using two 1×1 convolutional layers to learn the dependencies between channels. This structure can effectively capture channel correlations while controlling computational cost. The first layer compresses the number of channels, and the second layer restores the original dimensions.
[0101]
[0102]
[0103] in, The first layer convolution weights (dimensions) ), It's the compression ratio. The second layer convolution weights (dimensions) ), It is a sigmoid activation function, and the output channel weight vector has a dimension of . .
[0104] Step (2-3): Apply the learned channel weights to the original feature map, enhance the response strength of important feature channels through channel-by-channel multiplication, suppress the influence of minor channels, and improve the quality of feature representation.
[0105]
[0106] in, This represents a channel-wise multiplication operation (i.e., multiplying the channel value at each spatial location by the corresponding channel weight), preserving the dimensionality of the output feature map. constant.
[0107] In a preferred embodiment of the present invention, step (3) includes the following steps:
[0108] Step (3-1): Perform 1×1 convolutional channel compression on the channel attention-enhanced features. This operation reduces the feature dimension to lower computational complexity while preserving key semantic information.
[0109]
[0110] in, Convolution weights (dimensions) The number of channels in the output feature map is compressed to one-quarter of the original, and the dimension becomes... .
[0111] Step (3-2): Concatenate the compressed features with the original input features along the channel dimension. This fusion strategy combines the efficient representation of compressed features with the rich information of the original features, providing comprehensive input for subsequent semantic kernel generation.
[0112]
[0113] in, The operation concatenates two feature maps along the channel dimension, resulting in an output dimension of... This increases the ability to express features.
[0114] Step (3-3): The concatenated features are processed through two consecutive 3×3 convolutional layers. This design can effectively capture local spatial context information, gradually transform the feature representation, and finally output the original semantic kernel parameters.
[0115]
[0116] in, The first layer convolution weights (dimensions) ), The second layer convolution weights (dimensions) ), Number of groups For kernel size, the output dimension is .
[0117] Step (3-4): Apply Softmax normalization along the kernel element dimension. This operation ensures that the semantic kernel weights at each position sum to 1, forming a probability distribution and realizing dynamic weight allocation for semantic awareness.
[0118]
[0119] in, It is a batch index (from 1 to ...) ), It is a grouped index (from 1 to ...) ), It is the kernel element index (from 1 to 1). ), It is the spatial location coordinate, and the output dynamic semantic kernel dimension is... .
[0120] In a preferred embodiment of the present invention, step (4) of the adaptive sampling branch includes the following steps:
[0121] Step (4-1): Spatial features are extracted using a 3×3 convolutional layer, and then joint offset parameters are generated using a 1×1 convolutional layer. This cascaded design effectively captures the spatial context and generates accurate sampling position adjustment parameters.
[0122]
[0123] in, It is a 3×3 convolution weight (dimension) ), It is a 1×1 convolution weight (dimension) ), As an upsampling factor, the Tanh activation function ensures the offset range is between [-1, 1], and the output dimension is... .
[0124] Step (4-2): The joint parameters are split into two parts: position offset and range factor. Then, they are combined through element-wise multiplication. This design ensures that the position offset is adaptively adjusted by the range factor, avoiding excessive offset.
[0125]
[0126]
[0127]
[0128] in, It is the original position offset tensor. It is a range factor (constrained to the range [0,1] by Sigmoid activation). This represents element-wise multiplication, and the adjusted offset dimension is output. .
[0129] Step (4-3): Generate a uniformly distributed base sampling grid in the horizontal and vertical directions. This grid provides the initial sampling point distribution for upsampling as a reference for adaptive adjustment.
[0130]
[0131]
[0132]
[0133] in, The function generates a two-dimensional coordinate grid, and the output shape is... Each dimension represents: number of channels, coordinate axes (x / y), number of vertical sampling points, number of horizontal sampling points, height dimension, and width dimension.
[0134] Step (4-4): Overlay the adjusted offset onto the base grid, and then perform normalization to adapt the sampling coordinates to the standard range of the deep learning framework, preparing for bilinear sampling.
[0135]
[0136]
[0137] in, It is a tensor composed of the width and height of the feature map. The normalization operation maps the coordinate values to the range [-1, 1], and the output sampling grid dimension is... .
[0138] In a preferred embodiment of the present invention, step (5) includes the following steps:
[0139] Step (5-1): Copy the input feature map along the channel dimension. This operation allows each feature group to access complete feature information, creating conditions for subsequent grouping processing.
[0140]
[0141] in, The operation replicates the original feature map along the channel dimension. Next, the output dimension is from Expand to .
[0142] Step (5-2): Reorganize the dimensions of the sampling grid tensor. This transformation converts the multidimensional grid data into the input format required by the standard sampling function, ensuring that the sampling operation can process it correctly.
[0143]
[0144] in, The operation changes the shape of the tensor without altering the data content, converting the original six-dimensional grid into a four-dimensional coordinate graph with a height of [missing information]. Width is Finally, the dimension stores (x, y) coordinate pairs.
[0145] Step (5-3): Perform differentiable bilinear interpolation sampling, which extracts feature values from the input feature map based on the sampling grid, achieving geometric deformation-aware feature transformation:
[0146]
[0147] in, It is a bilinear interpolation sampling function that uses boundary padding (padding_mode='border') to handle out-of-bounds coordinates, and its output dimension is... Spatial resolution has been expanded times.
[0148] In a preferred embodiment of the present invention, step (6) of the efficient neighborhood aggregation includes the following steps:
[0149] Step (6-1): Reorganize the presampled features according to the number of groups, and adjust the shape of the semantic kernel weights. These transformations prepare a suitable data format for subsequent convolution operations, ensuring that the grouping process can be executed correctly.
[0150]
[0151]
[0152] in, The operation modifies the tensor view without copying the data. , The grouping feature dimension is The semantic kernel dimension is .
[0153] Step (6-2): Perform grouped depthwise convolution to achieve neighborhood aggregation. This special form of convolution allows each spatial location to have an independent weighted kernel, enabling efficient variable kernel convolution computation.
[0154]
[0155]
[0156] The convolution operation uses a grouping method where the number of groups equals the total number of channels (B·G·C), allowing each channel to be processed independently, resulting in an output dimension of... .
[0157] Step (6-3): Reshape the aggregation result into a standard feature map format. This operation restores the batch and channel dimensions, forming the final upsampled feature representation:
[0158]
[0159] The output feature map has a dimension of 1. Spatial resolution has been achieved. Double upsampling.
[0160] In a preferred embodiment of the present invention, step (7) of the deep feature refining flow includes the following steps:
[0161] Step (7-1): Apply the first layer of depthwise separable convolution processing. This operation first extracts spatial features, and then enhances the nonlinear expressive power through batch normalization and GELU activation.
[0162]
[0163] in, It is a depthwise convolution with the number of groups equal to the number of input channels. It is a batch normalization operation. It is a Gaussian error linear unit activation function, and the dimension of the output feature map remains unchanged.
[0164] Step (7-2): Recalibrate the importance of feature channels using the channel attention module. This module learns the weight coefficients of each channel, strengthening the contribution of key feature channels.
[0165]
[0166]
[0167] in, The channel attention module is similar in structure to the channel attention enhancement in step (2), with the input and output dimensions remaining unchanged.
[0168] Step (7-3): Apply a second layer of depthwise separable convolution to deepen spatial feature extraction and further optimize the quality of feature representation.
[0169]
[0170] Step (7-4): Use 1×1 convolution to integrate channel information. This operation fuses the feature responses of different channels to form a richer feature representation.
[0171]
[0172] Step (7-5): Fuse refined features and original aggregated features through residual connections. This strategy preserves the original information while introducing refined features, ensuring information integrity.
[0173]
[0174] When the number of input and output channels is the same, the residual path can use an identity mapping, and the final output upsampled feature map dimension is... .
[0175] The above embodiments detail the specific implementation process of the present invention, with each step clearly defined by formulas and textual descriptions regarding operational details and parameter requirements. Those skilled in the art can adjust the specific parameter settings according to actual application scenarios without departing from the core principles of the present invention. The scope of protection of the present invention is defined by the claims.
[0176] Beneficial effects
[0177] This invention generates dynamic semantic kernel weights through semantically guided branches, achieving content-aware adaptive neighborhood aggregation and effectively overcoming the edge blurring and texture distortion problems caused by traditional fixed kernel upsampling. A channel attention mechanism strengthens the representation of key feature channels, and combined with the offset adjustment of the adaptive sampling branch, enables the sampling grid to intelligently fit the image structural features, significantly improving the detail reconstruction quality of geometrically deformed regions. The deep feature refinement stream employs lightweight depthwise separable convolution and residual connections, maintaining the integrity of high-frequency information while eliminating artifacts, breaking through the structure preservation bottleneck of traditional interpolation methods.
[0178] This method innovatively integrates semantic guidance and geometric adaptation mechanisms, achieving pixel-level semantic weight allocation based on bilinear sampling, thus ensuring both global semantic consistency and local structural accuracy during reconstruction. The grouped deep convolutional architecture significantly reduces the computational complexity of the dynamic kernel, while global average pooling and channel compression techniques eliminate spatial redundancy, ensuring real-time performance even at 4K resolution. The final output image exhibits good visual fidelity in texture-dense regions and at sub-pixel edges, providing a reliable solution for upsampling methods in convolutional neural networks.
Claims
1. A semantically guided adaptive image upsampling method, characterized in that, Includes the following steps: S1: Input the image feature map to be upsampled, the feature map has dimensions of B×C×H×W, where B is the batch size, C is the number of channels, H is the height, and W is the width; S2: Channel attention enhancement is applied to the input features to increase the response weights of important feature channels; S3: Generate dynamic semantic kernel weights through semantic guidance branches for subsequent neighborhood feature aggregation; S4: Generate position offset and range factor through adaptive sampling branch to construct sampling grid; S5: Use a sampling grid to perform bilinear sampling on the input features to obtain presampled features; S6: Efficient neighborhood aggregation of presampled features based on dynamic semantic kernel weights; S7: Enhance aggregated features through deep feature refinement stream and output upsampled feature map.
2. The semantically guided adaptive image upsampling method according to claim 1, characterized in that, In S2, the channel attention enhancement includes: Perform global average pooling on the input feature map to obtain channel-level statistical features; The channel dependencies are learned by two 1×1 convolutional layers. The first convolutional layer compresses the number of channels by a reduction factor, and the second layer restores the original number of channels. Use the Sigmoid activation function to generate a channel weight map; The formula for enhancing channel attention is as follows: ,in, For the input feature map, This indicates channel-by-channel multiplication. For the Sigmoid function, and It is a 1×1 convolutional layer, and reduction is the channel compression ratio parameter.
3. The semantically guided adaptive image upsampling method according to claim 1, characterized in that, In S3, the semantic guidance branch includes: The features enhanced by channel attention are compressed using 1×1 convolution with a compression ratio of 1 / 4. The compressed features are concatenated with the original input features along the channel dimension; The concatenated features are processed by two consecutive 3×3 convolutional layers. The number of output channels in the first layer is C / 2, and the number of output channels in the second layer is groups×kernel_size². The output features are subjected to Softmax normalization to generate dynamic semantic kernel weights; The formula for normalizing semantic kernel weights is as follows: ,in, For batch indexing, For grouped indexes, For kernel element index, For spatial location, It is kernel_size.
4. The semantically guided adaptive image upsampling method according to claim 1, characterized in that, In S4, the adaptive sampling branch includes: Spatial features are extracted using a 3×3 convolutional layer, with a total output channel count of C / 2. Use the ReLU activation function to enhance nonlinear expressiveness; The joint offset parameters are generated through a 1×1 convolutional layer, and the number of output channels is 3×groups×scale². The Tanh activation function is applied to constrain the position offset within the range of [-1, 1]. The joint parameters are split into two parts: position offset and range factor. Multiply the position offset by the range factor after Sigmoid activation; The final sampling points are constructed by overlaying a pre-defined base grid. The coordinates of the sampling points are normalized to the range of [-1, 1].
5. The semantically guided adaptive image upsampling method according to claim 4, characterized in that, The method for generating the basic mesh includes: Generate a uniformly distributed array of sampling points in both the horizontal and vertical directions; The formula for calculating the basic grid coordinates is as follows: , ,in, Function generation from arrive The arithmetic sequence with a step size of 2 eventually forms a basic grid tensor of shape [1,2,scale,scale,1,1].
6. The semantically guided adaptive image upsampling method according to claim 4, characterized in that, The coordinate normalization process includes: Transform the sampling point coordinates from the image pixel coordinate system to the normalized coordinate system; The coordinate normalization formula is as follows: ,in, These are the original coordinate values. For the corresponding dimension (width or height).
7. The semantically guided adaptive image upsampling method according to claim 1, characterized in that, In S5, the bilinear sampling includes: Duplicate the input feature map by channel dimension (groups); Implement differentiable bilinear interpolation using PyTorch's grid_sample function; The sampling formula is as follows: The sampling mode is bilinear interpolation, and the edge processing uses border padding.
8. The semantically guided adaptive image upsampling method according to claim 1, characterized in that, In S6, the efficient neighborhood aggregation includes: The presampled features are divided into groups according to the channel dimension; The semantic kernel weights are reorganized into convolutional kernels. Implement variable kernel weighted aggregation using grouped depthwise convolution; The aggregation operation formula is as follows: ,in, For kernel_size, The operation extracts local neighborhood feature blocks.
9. The semantically guided adaptive image upsampling method according to claim 1, characterized in that, In S7, the deep feature refining flow includes: The first depthwise separable convolutional layer is used for spatial feature extraction; Batch normalization layer standardized feature distribution; The GELU activation function enhances nonlinearity; The channel attention module recalibrates the channel weights; The second deep separable convolutional layer deepens feature extraction; 1×1 convolutional layers integrate channel information; Residual connections preserve the original feature information; The refining formula is as follows: ,in, This represents depthwise separable convolution. Indicates channel attention. This indicates batch normalization.
10. The semantically guided adaptive image upsampling method according to claim 1, characterized in that, The upsampling factor (scale), the number of feature groups (groups), and the neighborhood aggregation kernel size (kernel_size) of the method are configurable parameters, where: The value of scale is an integer from 2 to 8, the value of groups is a positive integer from 1 to the number of channels C, the kernel_size is an odd number greater than or equal to 3, and the channel compression ratio reduction is an integer from 4 to 16.