A multi-scale image feature extraction processing method

Abstract

The application belongs to the field of optoelectronic information technology, and discloses a multi-scale image feature extraction processing method. The method comprises the following steps: inputting an image after data preprocessing into a lightweight backbone network, outputting basic feature maps of each scale through different levels of the lightweight backbone network; the lightweight backbone network comprises multiple levels corresponding to preset scale levels; setting a differentiated channel attenuation coefficient for the basic feature maps of each scale, pruning, adopting a bidirectional fusion strategy combining from top to bottom and from bottom to top, realizing complementary fusion of high and low scale features, and outputting a multi-scale fusion feature map.

Description

Technical Field

[0001] This invention belongs to the field of optoelectronic information technology, and more specifically, relates to a multi-scale image feature extraction and processing method. Background Technology

[0002] In existing technologies, multi-scale feature extraction often employs a fixed number of channels. High-resolution feature maps can easily lead to memory overload. For example, the image feature point extraction method disclosed in CN102222228A extracts feature points through multi-scale wavelet transform, but it does not involve memory optimization design, resulting in high memory pressure when processing high-resolution images. Some patents attempt to optimize memory through channel pruning, such as the deep neural network channel pruning method in CN110210620A, which only designs a fixed pruning ratio for single-scale networks and does not consider the memory consumption patterns of multi-scale feature transfer; the channel attention-guided pruning method in CN112949840B does not involve memory optimization for cross-scale feature fusion and cannot adapt to multi-scale extraction scenarios.

[0003] Meanwhile, existing technologies suffer from computational redundancy. Traditional standard convolutional backbone networks have a large number of parameters and often employ unidirectional feature fusion strategies, resulting in low feature reuse rates. The traditional transposed convolutions used in the upsampling / downsampling process further increase memory overhead. In addition, existing methods are mostly general-purpose designs, lacking adaptability to special scenarios such as medical images and remote sensing images. When processing such high-resolution, large-format images, they are prone to memory overload, making it difficult to meet the application requirements of mobile terminals and embedded devices. Summary of the Invention

[0004] In response to the above-mentioned defects or improvement needs of existing technologies, this invention provides a multi-scale image feature extraction and processing method. Its purpose is to solve the core contradiction of the incompatibility between memory consumption, processing accuracy, and scene adaptability, and to address the technical problem that limits the engineering application of multi-scale feature extraction technology.

[0005] To achieve the above objectives, the present invention provides a multi-scale image feature extraction and processing method, comprising: The pixel values ​​of the image are normalized to the range [0, 1], and ImageNet is used for mean and variance adjustment; N×scale = original image resolution / feature map resolution is defined, where N is the scale coefficient, N = 1, 2, 4, 8...; according to the preset scale coefficient set, the target resolution of the feature map at each scale is calculated, and the target resolution = original image resolution / N. The adjusted image is input into a lightweight backbone network, and basic feature maps at various scales are output through different layers of the lightweight backbone network; the lightweight backbone network includes multiple layers corresponding to preset scale levels; Differentiated channel attenuation coefficients are set for the basic feature maps at each scale. The channel attenuation coefficient α is defined as the number of retained channels of the feature map / the original number of channels of the feature map. Based on the SENet channel attention mechanism, the channel importance weights are calculated separately for the basic feature maps at each scale. The channels with the highest weights are retained (α×100%). The number of retained channels is taken as the integer closest to α×the original number of channels, and the pruned feature maps at each scale are obtained. After pruning, feature maps at each scale are aligned by channel, and the number of channels in all feature maps is unified to the preset target number of channels C. The feature maps are then fused from high-scale coefficients to low-scale coefficients and then from low-scale coefficients to high-scale coefficients. When fusing from top to bottom, only the feature map of the current target scale and the high-scale feature map to be upsampled are retained, and the rest of the data is released in real time after fusion. When fusing from bottom to top, only the feature map of the current target scale and the low-scale feature map to be downsampled are retained, and the rest of the data is released in real time after fusion. The fused feature maps are sequentially processed by BN and GELU activation to output a multi-scale fused feature map.

[0006] Preferably, the mathematical expression for mean and variance adjustment is: in, X Let be the pixel value after normalizing the image to [0,1], μ be the pixel mean of the corresponding channel, and σ be the pixel variance of the corresponding channel. X norm The adjusted pixel values ​​are shown below. The core principle and function of this operation can be broken down into four points, which are fully compatible with the multi-scale feature extraction scenario of this invention. It eliminates differences in dimensions and pixel distribution. Although the pixel values ​​of natural images, medical images, and remote sensing images have been normalized to [0,1], there are significant differences in pixel distribution between different images (e.g., the blue channel pixel values ​​of remote sensing images are generally high, while the gray channel mean of medical CT images is moderate). By subtracting the mean to center the pixel distribution (mean approaches 0) and dividing by the variance to standardize the distribution (variance approaches 1), the distribution differences between different images are eliminated, allowing the backbone network to focus on extracting the essential features of the image without learning individual differences in data distribution.

[0007] Preferably, the lightweight backbone network comprises 11 core layers and employs a depthwise separable convolution and hierarchical activation design. If the input data has a large distribution range and the mean deviates from 0, the network's weight updates will experience gradient oscillations and slow convergence. After mean and variance adjustment, the data distribution is concentrated in a small range near 0, resulting in smoother gradient updates. This significantly improves the training convergence speed of the lightweight backbone network, adapting to the rapid deployment needs of various scenarios such as edge computing, medicine, and remote sensing.

[0008] To improve the robustness of feature extraction, in multi-scale feature extraction, feature maps of different scales have varying sensitivities to pixel noise (high-resolution 1× / 2× feature maps are more sensitive to noise). Mean-variance adjustment can reduce the impact of random fluctuations in pixel values ​​on feature extraction, allowing the SENet attention mechanism to more accurately identify core feature channels when calculating channel weights, avoiding misjudgments of redundant channels caused by noise, and improving the accuracy of channel pruning.

[0009] Matching the Input Distribution of Pre-trained Networks: The MobileNetV3-Small and EfficientNet-Lite1 backbone networks of this invention are both pre-trained on the ImageNet dataset, employing the same mean-variance adjustment strategy during pre-training. If new input data does not follow this distribution, a distribution shift problem will occur, causing the pre-trained weight parameters to fail to transfer effectively, significantly reducing the accuracy of multi-scale feature extraction. Mean-variance adjustment, on the other hand, can align the input distribution and ensure the effective reuse of pre-trained weights.

[0010] Furthermore, the structure of the 11 levels is as follows: Conv2d(3→16,3×3,step 1,fill 1)+BN→resolution H×W; Depth-separable Conv2d(16→16→24, 3×3, stride 2, padding 1) + BN (depth convolution channels = 16, pointwise convolution output = 24) → resolution H / 2×W / 2; Depth-separable Conv2d(24→24→40,5×5, step size 2, fill size 2)+BN→resolution H / 4×W / 4; Depth-separable Conv2d(40→40→48,3×3, step size 1, fill 1)+BN→resolution H / 4×W / 4; Depth-separable Conv2d(48→48→96,5×5, step size 2, fill size 2)+BN→resolution H / 8×W / 8; Depth-separable Conv2d(96→96→128,3×3, step size 1, fill 1)+BN→resolution H / 8×W / 8; Depth-separable Conv2d(128→128→160,3×3, step size 2, fill 1)+BN→resolution H / 16×W / 16; Depth-separable Conv2d(160→160→192,5×5, step size 1, fill size 2)+BN→resolution H / 16×W / 16; Depth-separable Conv2d(192→192→224,3×3, step size 1, fill 1)+BN→resolution H / 16×W / 16; Depth-separable Conv2d(224→224→256,3×3, step size 1, fill 1)+BN→resolution H / 16×W / 16; 1×1Conv2d(256→128,1×1,step 1)+BN→resolution H / 16×W / 16.

[0011] Furthermore, the SENet-based channel attention mechanism calculates channel importance weights for each scale's base feature map individually, including: The channel statistical vector S is calculated from the basic feature maps at each scale using global average pooling. Channel weights Z = σ(W2 × δ(W1 × S + b1) + b2) are generated using two fully connected layers and activation functions. The two fully connected layers include a dimensionality reduction fully connected layer and a dimensionality increase fully connected layer. W1 represents the weights of the dimensionality reduction fully connected layer, which reduces the dimension from the original number of channels to max(original number of channels / / 16, 1)). W2 represents the weights of the dimensionality increase fully connected layer, which increases the dimension from max(original number of channels / / 16, 1) back to the original number of channels. δ is the ReLU activation function, and σ is the Sigmoid activation function. Here, max(original number of channels / / 16, 1)) represents dividing the number of channels by 16 and rounding down to the nearest integer, but not less than 1. / / represents integer division (only the quotient is taken, not the decimal).

[0012] Furthermore, the process of fusing from high-scale coefficients to low-scale coefficients and then from low-scale coefficients to high-scale coefficients includes: The high-scale feature map is upsampled to the target scale through a transpose depthwise convolution with a 3×3 kernel, stride of 2, padding of 1, and output padding of 1. If the size of the upsampled feature map is inconsistent with the target scale feature map, it is matched by center pruning or edge zero padding, and then added element by element with the feature map after corresponding scale pruning and channel alignment. The low-scale feature map is downsampled to the target scale by a depthwise separable convolution with a kernel size of 3×3, padding of 1, and stride of 2. The size after downsampling follows the specific calculation standard of conventional convolution output: first, add twice the padding value to the input size, then subtract the convolution kernel size, divide the result by the stride and round down, and finally add 1 to get the final output size. Then, multiply it element-wise with the feature map after corresponding scale pruning and channel alignment.

[0013] Furthermore, the channel attenuation coefficients of the basic feature maps at each scale in step S3 satisfy: The attenuation coefficient α_min of the highest resolution feature map (corresponding to the minimum scale coefficient N_min) is ∈ [0.3, 0.5], and the attenuation coefficient α_max of the lowest resolution feature map (corresponding to the maximum scale coefficient N_max) is ∈ [0.8, 1.0]. For medical imaging scenarios, a fixed channel attenuation coefficient is used. For remote sensing image scenarios and other general scenarios, logarithmic linear interpolation is used to calculate the attenuation coefficient, where: Remote sensing scenario: N_min=0.5, N_max=4; General scenario: N_min=1, N_max=8.

[0014] Furthermore, the preset target number of channels C in step S4 can be adjusted according to the application scenario: Mobile terminal / embedded device scenario: C=32; Medical / Remote Sensing High-Precision Scene: C=128; Channel alignment is achieved using the Xavier uniform initialization strategy for weight initialization of 1×1 pointwise convolutions, with the gain value for the GELU activation function set to 1.0.

[0015] Furthermore, adjust parameters for specific scenarios: A. Medical imaging scenario: The scale levels are set to 1×, 2×, 4×, 8×, and 16×. The 16× scale base feature map is generated by downsampling the 8× scale feature map through a depthwise separable convolution with a kernel of 3×3, padding of 1, and stride of 2. Channel attenuation coefficient mapping rules: 1×→α1=0.5, 2×→α2=0.625, 4×→α3=0.8, 8×→α4=0.9, 16×→α_max=1.0; B. Remote sensing image scene: The scale levels are set to 0.5×, 1×, 2×, and 4×. The 0.5× scale basic feature map is generated by downsampling the original image through a depth-separable convolution with a kernel of 3×3, padding of 1, and stride of 2. The backbone network outputs 1× in the first layer, 2× in the second layer, and 4× in the third layer. Channel attenuation coefficient calculation: α_min=0.5 (corresponding to N_min=0.5), α_max=0.95 (corresponding to N_max=4), intermediate scale according to: α_inter = α_min + (α_max - α_min) × (log2(N) - log2(N_min)) / (log2(N_max) - log2(N_min)) is calculated (N is the current scale coefficient). The result is rounded to two decimal places, and the number of channels is processed according to S3 integerization and lower limit rules.

[0016] Furthermore, supplementary rules for generating 16× scale feature maps in medical imaging scenes: The activation function for depthwise separable convolution is GELU. The number of output channels is set according to the integer number of channels after 8× scale pruning. If the number of channels after 8× pruning is odd, add 1 to the nearest even number upwards; if it is even, leave it unchanged. Supplementary rules for 0.5× scale preprocessing in remote sensing image scenes: The target integer size of 0.5× scale is defined as the result of dividing the width / height of the original image resolution by 2. If size matching is required after downsampling, only the right and bottom sides of the image are zero-filled. If the fill amount is negative, the image is cropped to the target size from the center to ensure that the core area in the upper left corner of the image is not destroyed.

[0017] The present invention also provides an electronic device, comprising: a computer-readable storage medium and a processor; The computer-readable storage medium is used to store executable instructions; The processor is used to read executable instructions stored in the computer-readable storage medium and execute the above-described method.

[0018] The present invention also provides a computer-readable storage medium storing computer instructions for causing a processor to perform the above-described method.

[0019] The present invention also provides a computer program product, including a computer program or instructions that, when executed by a processor, implement the above-described method.

[0020] In summary, compared with the prior art, the multi-scale image feature extraction and processing method provided by this invention has the following advantages: 1. This invention is based on a shared storage and real-time release mechanism. During the bidirectional fusion process, only the feature maps required for the current fusion are retained. After the hierarchical fusion is completed, unnecessary intermediate data is released immediately, avoiding memory redundancy caused by storing all intermediate feature maps at multiple scales. This significantly reduces peak memory consumption. Combined with the differentiated channel decay strategy of the SENet channel attention mechanism, highly important channels at each scale are accurately selected. Redundant channels are pruned while ensuring that core features are not lost, effectively reducing the storage volume of feature maps. The lightweight backbone network uses depthwise separable convolutions instead of traditional convolutions, which greatly reduces the memory occupation of network parameters and intermediate calculation results, further reducing memory overhead. 1×1 pointwise convolutions unify the number of channels in feature maps at each scale, avoiding redundant storage and calculation in the fusion stage due to differences in the number of channels, and improving memory utilization efficiency.

[0021] 2. The improved lightweight backbone network of this invention adopts a hierarchical activation strategy (the first 3 ReLU layers ensure the stability of shallow features, and the last 8 GELU layers enhance the nonlinear expression of deep features). Combined with a standardized "convolution / depthseparable convolution + BN + activation" layer structure, it reduces the amount of computation while ensuring the extraction accuracy of multi-scale features. The bidirectional fusion strategy combining top-down and bottom-up approaches compensates for the information loss of fusion in a single direction, achieving complementary fusion of high and low scale features, improving the integrity and representational ability of multi-scale features. Channel pruning is based on the channel importance weight selection of SE attention to avoid feature loss caused by blind pruning, maintaining the discriminative ability of features while reducing channels. The combination of BN processing and GELU activation after fusion optimizes the stability of feature distribution and nonlinear expression, taking into account both lightweight design and feature extraction accuracy, balancing computational efficiency and representational ability, and further improving the feature extraction effect.

[0022] 3. This invention features scenario-specific scale customization: Medical imaging scenarios are expanded to a 16× scale to accommodate fine texture analysis needs; remote sensing scenarios add a 0.5× scale to match the need for large-scale coarse-grained feature analysis; and general scenarios adapt to basic resolution analysis needs, accurately matching the feature analysis characteristics of images from different fields. Differentiated channel attenuation rules: Medical scenarios use fixed coefficients to adapt to the need for fine feature preservation; remote sensing / general scenarios calculate coefficients through logarithmic linear interpolation to match the feature importance distribution at different scales. Target channel number adjustment based on scenario: Mobile / embedded device scenarios are set to 32 to adapt to computing power / memory limitations; medical / remote sensing high-precision scenarios are set to 128 to meet high-precision analysis needs, balancing edge deployment and professional applications. Specific scale generation supplementary rules: The 16× scale in medical imaging ensures an even number of channels for efficient convolution calculations; the 0.5× scale in remote sensing only fills / crops the lower right side and retains the core area in the upper left corner, adapting to the analysis needs of core geographic information in remote sensing images. It possesses strong scenario adaptability, meeting the differentiated application needs of multiple fields.

[0023] 4. The data preprocessing of this invention adopts ImageNet mean-variance normalization to unify the data distribution and improve the stability of network training and inference. The scale coefficient, channel attenuation coefficient, feature map size matching, etc. are all given clear mathematical definitions and rules, reducing the subjectivity of parameter adjustment and improving the reproducibility of the method. The 11-layer core structure of the backbone network is standardized, and the convolution type, parameters and resolution changes of each layer are clearly defined, reducing the difficulty of engineering implementation. The size matching rules (center cropping / edge zero padding) ensure the size consistency of multi-scale feature fusion, avoid feature misalignment and improve the stability of the processing flow.

[0024] 5. The lightweight design of this invention, including depthwise separable convolution and channel pruning, significantly reduces floating-point operations and improves inference speed. The downsampling module uses depthwise separable convolution with a stride of 2, and the number of output channels is controlled at 1.25-2 times that of the input. This ensures feature amplification while controlling computational complexity. The entire process has no redundant computational steps. Combined with memory optimization strategies, it can be efficiently adapted to edge deployment scenarios with limited computing power and memory, such as mobile terminals and embedded devices. Attached Figure Description

[0025] Figure 1 This is a flowchart of the multi-scale image feature extraction and processing method provided by the present invention.

[0026] Figure 2 This is a schematic diagram of a lightweight backbone network.

[0027] Figure 3 It is a single-scale operation flowchart.

[0028] Figure 4 This is a flowchart of the incentive operation process. Detailed Implementation

[0029] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0030] This invention provides a multi-scale image feature extraction and processing method, such as... Figure 1 As shown, it includes: The pixel values ​​of the image are normalized to the range [0, 1], and ImageNet is used for mean and variance adjustment; N×scale = original image resolution / feature map resolution is defined, where N is the scale coefficient, N = 1, 2, 4, 8...; according to the preset scale coefficient set, the target resolution of the feature map at each scale is calculated, and the target resolution = original image resolution / N. The adjusted image is input into a lightweight backbone network, and basic feature maps at various scales are output through different layers of the lightweight backbone network; the lightweight backbone network includes multiple layers corresponding to preset scale levels; Differentiated channel attenuation coefficients are set for the basic feature maps at each scale. The channel attenuation coefficient α is defined as the number of retained channels of the feature map / the original number of channels of the feature map. Based on the SENet channel attention mechanism, the channel importance weights are calculated separately for the basic feature maps at each scale. The channels with the highest weights are retained (α×100%). The number of retained channels is taken as the integer closest to α×the original number of channels, and the pruned feature maps at each scale are obtained. After pruning, feature maps at each scale are aligned by channel, and the number of channels in all feature maps is unified to the preset target number of channels C. The feature maps are then fused from high-scale coefficients to low-scale coefficients and then from low-scale coefficients to high-scale coefficients. When fusing from top to bottom, only the feature map of the current target scale and the high-scale feature map to be upsampled are retained, and the rest of the data is released in real time after fusion. When fusing from bottom to top, only the feature map of the current target scale and the low-scale feature map to be downsampled are retained, and the rest of the data is released in real time after fusion. The fused feature maps are sequentially processed by BN and GELU activation to output a multi-scale fused feature map.

[0031] Specifically, in this invention, the mean (0.485, 0.456, 0.406) and variance (0.229, 0.224, 0.225) are channel-by-channel statistical values ​​of the RGB three channels (corresponding to the R, G, and B channels respectively). These values ​​are pixel statistics from over one million natural images in the ImageNet-1k dataset. The selection of these values ​​is not random, but rather a dual choice combining the technical design of this invention and industry practices in deep learning. The specific reasons are as follows: 1. Adapt to the pre-training foundation of the backbone network to achieve efficient weight transfer.

[0032] The lightweight backbone network of this invention is an improvement on MobileNetV3-Small and EfficientNet-Lite1. The official pre-trained models of these two networks use ImageNet-1k as the training set, and the data preprocessing strictly adopts this set of means and variances during pre-training. In deep learning, the weights of the pre-trained model are learned based on a specific data distribution. If this invention uses other means and variances, the image data distribution input to the network will be inconsistent with the pre-training distribution, which will cause the low-level weights of feature extraction to fail (e.g., shallow convolutional layers cannot effectively extract basic features such as edges and textures). However, by using this set of values, the improved 11-layer backbone network can directly reuse the pre-trained weights. On this basis, fine-tuning can be performed for scenarios such as medicine and remote sensing, which greatly improves the training efficiency and feature extraction accuracy of the model.

[0033] 2. This value is the statistically optimal value for large-scale natural images and has generalization properties.

[0034] The ImageNet-1k dataset contains over 1.2 million natural images across 1000 classes, covering everyday scenes, flora and fauna, vehicles, and architecture. Its pixel distribution exhibits strong statistical representativeness, and the mean and variance calculated channel-by-channel represent common statistical characteristics of natural images. R channel mean 0.485, variance 0.229: The pixel values ​​of the red channel in natural images are relatively concentrated overall, with moderate variance; The G channel has a mean of 0.456 and a variance of 0.224. The green channel is the core channel of natural images (such as vegetation, sky, and skin color, which contain a lot of green information). Its pixel values ​​are most concentrated and its variance is the smallest. The B channel has a mean of 0.406 and a variance of 0.225. The blue channel mainly corresponds to the sky, water, etc., with generally low pixel values, the smallest mean, and a slightly higher variance than the G channel. This set of values ​​has been verified by industry and academia over a long period of time and is the universally optimal value for normalizing natural images. It is not only applicable to general scenarios, but also has good generalization for medical images (after conversion to 3 channels) and remote sensing images (RGB three channels). In this invention, it is only necessary to convert single-channel medical CT images to 3 channels and directly use RGB three channels for remote sensing satellite images. Effective normalization can be completed by using this set of values, without the need to separately calculate the mean and variance for each scene, thus simplifying the scene adaptation process.

[0035] 3. Matching the lightweight design of this invention avoids additional computational overhead.

[0036] One of the core objectives of this invention is to reduce memory consumption and improve computational efficiency. However, separately calculating the mean and variance of images for each scenario—general, medical, remote sensing, or edge computing—would lead to two major problems: Increased computational overhead for data preprocessing: The mean and variance of each channel of the dataset for each scenario need to be calculated. In particular, the datasets for remote sensing and medical images are large in scale and high in resolution, which consumes a lot of computing power and time. Increasing the complexity of model deployment: Edge devices (such as mobile phones and embedded devices) have limited computing power and memory. Deploying different mean and variance parameters for different scenarios would increase the model's parameter storage, which contradicts the original intention of lightweight design. However, by using the universal mean and variance of ImageNet, no additional statistics are needed, and it can be directly adapted to all scenarios. While ensuring the normalization effect, it fully meets the memory optimization and computational efficiency requirements of this invention.

[0037] 4. Industry-standard approach to improve the reproducibility and engineering feasibility of the method.

[0038] This invention explicitly highlights the advantages of standardized processes and high feasibility. The mean and variance of ImageNet are industry-standard in computer vision, used by almost all CNN-based image feature extraction, classification, and detection models (such as ResNet, VGG, and MobileNet series). Choosing this standard value makes the method easier for those skilled in the art to reproduce, reducing the difficulty of engineering implementation—developers do not need to readjust normalization parameters and can directly use existing code to implement data preprocessing, thus improving the method's applicability and scalability.

[0039] Specifically, such as Figure 2As shown, the lightweight backbone network consists of 11 core layers. These 11 core layers are an improved lightweight backbone network structure based on MobileNetV3-Small / EfficientNet-Lite1. They employ a standardized design of convolutional / depthmology-separable convolutions, batch normalization (BN) layers, and activation functions. The first three layers use ReLU activation, and the last eight layers use GELU activation. Downsampling layers progressively reduce resolution with a stride of 2, ultimately outputting a high-dimensional feature map with fixed channels. The specific convolution types, parameters, channel transformations, and resolution changes of each layer are as follows, serving as the core execution unit for feature extraction: Core Layer 1 Convolution type: Standard Conv2d convolution Core parameters: 3 to 16 channels, 3×3 convolution kernel, stride 1, padding 1, BN layer added. Activation function: ReLU Output resolution: Original image resolution H×W Function: Shallow basic feature extraction, mapping a 3-channel RGB image to a 16-channel primary feature map. Core Layer 2 Convolution type: Depthwise separable convolution Core parameters: 16→16→24 channels (16 channels for depthwise convolution, 24 channels for pointwise convolution output), 3×3 convolution kernel, stride 2, padding 1, BN layer added. Activation function: ReLU Output resolution: H / 2 × W / 2 Function: Initial downsampling, channel amplification by 1.5 times, extraction of 2× scale basic features. Core Layer 3 Convolution type: Depthwise separable convolution Core parameters: 24→24→40 channels, 5×5 convolution kernel, stride 2, padding 2, BN layer added. Activation function: ReLU Output resolution: H / 4 × W / 4 Function: Secondary downsampling, channel amplification by 1.67 times, extraction of 4× scale basic features (activation function switching the last layer). Core Layer 4 Convolution type: Depthwise separable convolution Core parameters: 40→40→48 channels, 3×3 convolution kernel, stride 1, padding 1, BN layer added. Activation function: GELU (Activation function officially switched) Output resolution: H / 4 × W / 4 Function: Without downsampling, channel amplification and deep nonlinear expression enhancement are performed on 4× scale features. Core Layer 5 Convolution type: Depthwise separable convolution Core parameters: 48→48→96 channels, 5×5 convolution kernel, stride 2, padding 2, BN layer added. Activation function: GELU Output resolution: H / 8 × W / 8 Function: Triple downsampling, 2x channel amplification, extraction of 8× scale basic features (core feature output layer for general scenarios) Core layer 6 Convolution type: Depthwise separable convolution Core parameters: 96→96→128 channels, 3×3 convolution kernel, stride 1, padding 1, BN layer added. Activation function: GELU Output resolution: H / 8 × W / 8 Function: Performs channel amplification and fine feature extraction on 8× scale features without downsampling. Core layer 7 Convolution type: Depthwise separable convolution Core parameters: 128→128→160 channels, 3×3 convolution kernel, stride 2, padding 1, with BN layer. Activation function: GELU Output resolution: H / 16×W / 16 Function: Four downsampling operations amplify channels by 1.25 times, providing basic features for 16× scale medical scenarios. Core layer 8 Convolution type: Depthwise separable convolution Core parameters: 160→160→192 channels, 5×5 convolution kernel, stride 1, padding 2, with BN layer. Activation function: GELU Output resolution: H / 16×W / 16 Function: Performs channel amplification and deep feature mining on 16× scale features without downsampling. Core layer 9 Convolution type: Depthwise separable convolution Core parameters: 192→192→224 channels, 3×3 convolution kernel, stride 1, padding 1, with BN layer. Activation function: GELU Output resolution: H / 16×W / 16 Function: Without downsampling, continuously enhances the representational power of 16× scale features. Core layer 10 Convolution type: Depthwise separable convolution Core parameters: 224→224→256 channels, 3×3 convolution kernel, stride 1, padding 1, with BN layer. Activation function: GELU Output resolution: H / 16×W / 16 Function: Without downsampling, it amplifies the 16× scale feature channels to 256, forming a high-dimensional feature representation. Core layer 11 Convolution type: 1×1 pointwise convolution Core parameters: 256 to 128 channels, 1×1 convolutional kernel, stride 1, with BN layer. Activation function: GELU Output resolution: H / 16×W / 16 Function: Channel dimensionality reduction, compressing 256-channel high-dimensional features into 128-channel core features, reducing subsequent computational overhead and preparing for feature fusion.

[0040] Core design features of the 11 core layers: Resolution variation: Only layers 2, 3, 5, and 7 are downsampling layers, with the resolution gradually decreasing from H×W to H / 16×W / 16, while the resolution of the remaining layers remains unchanged, focusing on feature mining; Channel changes: The number of channels is gradually increased from 3 to 256, and finally reduced to 128 through 1×1 convolution, achieving "incrementing features and reducing overhead by increasing dimensionality"; Convolutional pairings: shallow layers use standard 3×3 convolutions, followed by depthwise separable convolutions, which significantly reduces the number of parameters and memory usage, fitting the lightweight design. Scale output: In general scenarios, basic features of four scales, 1×, 2×, 4×, and 8×, are directly output from layers 1, 2, 3, and 5 respectively, without additional calculations, thus improving efficiency.

[0041] Specifically, the SENet channel attention mechanism is used to calculate channel importance weights for the base feature maps at each scale individually, including: The channel statistical vector S is calculated from the basic feature maps at each scale using global average pooling. Channel weights Z = σ(W2 × δ(W1 × S + b1) + b2) are generated through two fully connected layers and activation functions. The two fully connected layers include a dimensionality reduction fully connected layer and a dimensionality increase fully connected layer. W1 is the weight of the dimensionality reduction fully connected layer, which is used to reduce the dimension from the original number of channels to max(original number of channels / / 16, 1)). W2 is the weight of the dimensionality increase fully connected layer, which is used to increase the dimension from max(original number of channels / / 16, 1) back to the original number of channels. δ is the ReLU activation function, and σ is the Sigmoid activation function.

[0042] Specifically, the process of fusing from high-scale coefficients to low-scale coefficients and then from low-scale coefficients to high-scale coefficients includes: The high-scale feature map is upsampled to the target scale through a transpose depthwise convolution with a 3×3 kernel, stride of 2, padding of 1, and output padding of 1. If the size of the upsampled feature map is inconsistent with the target scale feature map, it is matched by center pruning or edge zero padding, and then added element by element with the feature map after corresponding scale pruning and channel alignment. The low-scale feature map is downsampled to the target scale by a depthwise separable convolution with a kernel size of 3×3, padding of 1, and stride of 2. The size after downsampling follows the specific calculation standard of conventional convolution output: first, add twice the padding value to the input size, then subtract the convolution kernel size, divide the result by the stride and round down, and finally add 1 to get the final output size. Then, multiply it element-wise with the feature map after corresponding scale pruning and channel alignment.

[0043] Figure 3 It is a single-scale operation flowchart. Figure 4 This is a flowchart of the incentive operation process.

[0044] This invention discloses a multi-scale image feature extraction and processing method that reduces memory consumption. Its core is a complete process design that addresses the trade-off between memory consumption, extraction accuracy, and scene adaptability in multi-scale feature extraction. This method is implemented through lightweight backbone network construction, SENet attention-based differential channel pruning, bidirectional feature fusion and shared storage, and scene-specific parameter customization. It is applicable to various scenarios such as general image analysis, medical image detection, remote sensing image classification, and real-time mobile terminal recognition. The following detailed embodiments illustrate this method. All embodiments adhere to the strict rule of retaining ≥8 channels. If the number of retained channels calculated by the attenuation coefficient is <8, it is forcibly set to 8, and the attenuation coefficient is adjusted accordingly.

[0045] Example 1: General Scenario (Natural Image Classification Task) 1.1 Experimental Environment Hardware: Intel Core i7-12700H CPU, NVIDIA RTX 3060 GPU (6GB VRAM), 16GB DDR4 RAM; Software: PyTorch 2.0, Python 3.9, OpenCV 4.8.0.

[0046] 1.2 Experimental Parameter Configuration Original images: A subset of the ImageNet-1k dataset was used, with a uniform resolution of 224×224 (H×W) and 3-channel RGB images; Scale levels: 1×, 2×, 4×, 8×, corresponding to target resolutions of 224×224, 112×112, 56×56, and 28×28, respectively; Channel attenuation coefficient: In general scenarios, logarithmic linear interpolation is used for calculation. N_min=1, N_max=8, α_min=0.3, α_max=0.8, and the calculated values ​​are 1×→0.3, 2×→0.41, 4×→0.61, and 8×→0.8. The calculation results are rounded to two decimal places. Preset target number of channels C=64 (adapts to the computing power requirements of mid-range devices); Backbone network: Improved MobileNetV3-Small, with ReLU activation in the first 3 layers and GELU activation in the last 8 layers. The activation functions are all attached after the last BN layer of each layer.

[0047] 1.3 Specific Implementation Steps S1. Data preprocessing: Normalize the image pixel values ​​to [0,1], and use ImageNet mean (0.485, 0.456, 0.406) and variance (0.229, 0.224, 0.225) for mean and variance adjustment; calculate the target resolution for each scale according to the scale coefficient: 224×224 (1×), 112×112 (2×), 56×56 (4×), 28×28 (8×).

[0048] S2. Backbone Network Feature Extraction: Features are extracted using an improved MobileNetV3-Small backbone network (11 core layers). The first layer outputs a 1× basic feature map (16 channels), the second layer outputs a 2× basic feature map (24 channels), the third layer outputs a 4× basic feature map (40 channels), and the fifth layer outputs an 8× basic feature map (96 channels).

[0049] S3. Differentiated Channel Pruning: Based on the SENet channel attention mechanism, channel importance weights are calculated separately for each scale. The squeezing operation generates a channel statistical vector S, and the activation operation generates channel weights using the formula Z=σ(W2×δ(W1×S+b1)+b2). ​​The top N% of channels by weight are retained according to the attenuation coefficient. The number of channels is first rounded to the nearest integer, and if it is less than 8, it is forcibly set to 8. 1×: 16×0.3=4.8 → Rounding down to 5 < 8, forcibly retaining 8 channels, corresponding to α adjusted to 0.5; 2×: 24×0.41=9.84 → Round to the nearest integer (≥8, keep); 4×: 40×0.61=24.4 → Rounded to the nearest integer, 24 channels (≥8, retained); 8×: 96×0.8=76.8 → Round to the nearest integer 77 (≥8, keep).

[0050] S4. Channel Alignment and Bidirectional Fusion: The number of channels in feature maps at each scale is unified to 64 using 1×1 pointwise convolution (GELU activation, no BN layer, Xavier uniform initialization, gain=1.0); a bidirectional fusion strategy is adopted, and a shared storage mechanism is enabled during the fusion process, retaining only the feature maps required for the current fusion, and releasing intermediate data not required for subsequent fusions immediately after the layer fusion is completed. Top-down fusion: from 8×→4×→2×→1×, the high-scale feature map is upsampled step by step through the transpose of the convolution kernel 3×3, stride 2, padding 1, and output padding 1. When the size does not match, it is matched by center cropping or edge zero padding, and then added element by element with the corresponding scale feature map. Bottom-up fusion: From 1×→2×→4×→8×, the low-scale feature map is downsampled by a depthwise separable convolution with a kernel of 3×3, padding of 1, and stride of 2. The size is calculated as Output=floor((input size+2×padding-kernel size) / stride)+1, and then multiplied element-wise with the feature map of the corresponding scale.

[0051] S5. Post-fusion processing: Perform BN processing and GELU activation operations on the bidirectional fused feature map in sequence to output a multi-scale fused feature map with 64 channels and a resolution of 224×224.

[0052] S6. Scene Adaptation: No additional adjustments are needed for general scenes. The fused feature map can be directly input into the subsequent classification network to complete the image classification task.

[0053] Example 2: Medical Imaging Scene (Nodule Detection in Lung CT Images) 2.1 Experimental Environment Hardware: Intel Xeon Gold 6330 CPU, NVIDIA A100 GPU (40GB VRAM), 64GB DDR4 RAM; Software: PyTorch 2.1, Python 3.10, SimpleITK 2.2.1.

[0054] 2.2 Experimental Parameter Configuration Original images: Lung CT image dataset, resolution 512×512 (H×W), single-channel grayscale images converted to 3-channel input network; Scale levels: 1×, 2×, 4×, 8×, 16×, with target resolutions of 512×512, 256×256, 128×128, 64×64, and 32×32, respectively. Channel attenuation coefficients: fixed coefficients are used, 1×→α1=0.5, 2×→α2=0.625, 4×→α3=0.8, 8×→α4=0.9, 16×→α_max=1.0; The preset target number of channels is C=128 (to meet the requirements of high-precision detection). Backbone network: Improved EfficientNet-Lite1, with the first 3 layers of ReLU activation and the last 8 layers of GELU activation. The depthwise separable convolution is a cascaded structure of "depthwise convolution + BN + pointwise convolution + BN".

[0055] 2.3 Specific Implementation Steps S1. Data preprocessing: Normalize the pixel values ​​of the CT images to [0,1] and adjust them using ImageNet mean and variance; calculate the target resolution at each scale, where the target resolution at the 16× scale is 32×32.

[0056] S2. Backbone Network and 16× Feature Generation: The improved EfficientNet-Lite1 outputs 1× (16 channels), 2× (24 channels), 4× (40 channels), and 8× (96 channels) basic feature maps; the 16× scale feature map is generated by downsampling the 8× feature map through a depthwise separable convolution (GELU activation) with a kernel of 3×3, padding of 1, and stride of 2. The number of output channels is set according to the number of channels after 8× pruning. The number of channels is first rounded to the nearest integer, and then it is determined to be odd or even. Odd numbers are added by 1 to get even numbers, and even numbers are directly retained.

[0057] S3. Differentiated Channel Pruning: Based on the SENet channel attention mechanism, channel weights at each scale are calculated, and channels are retained with a fixed attenuation coefficient. The rule of retaining ≥8 channels is strictly followed, and the number of channels is rounded to an even number (only for 16×). 1×: 16×0.5=8 → Round to 8 channels (≥8, retain); 2×: 24×0.625=15 → Round to 15 channels (≥8, keep); 4×: 40×0.8=32 → Rounded down to 32 channels (≥8, retained); 8×: 96×0.9=86.4 → Round to the nearest integer 86 (≥8, keep, even number); 16×: 86×1.0=86 → Round to 86 channels (≥8, keep, consistent with 8×).

[0058] S4. Channel Alignment and Bidirectional Fusion: The number of channels in all scale feature maps is unified to 128 by 1×1 pointwise convolution (GELU activation, no BN layer, Xavier initialization, gain=1.0); bidirectional fusion covers 16×→8×→4×→2×→1× (from top to bottom) and 1×→2×→4×→8×→16× (from bottom to top). The fusion rules are the same as in Example 1. The 16× scale size matching adopts center clipping. The fusion process enables the shared storage mechanism to release intermediate data in real time.

[0059] S5. Post-fusion processing: Perform BN processing and GELU activation on the fused feature map to output a multi-scale fused feature map with 128 channels and a resolution of 512×512.

[0060] S6. Scene Adaptation: Implement medical imaging scene supplementation rules, with an even number of channels (86) at the 16× scale to ensure efficient convolution calculation. The fused feature map is then input into the subsequent detection network to complete lung nodule detection.

[0061] Example 3: Remote Sensing Image Scene (Satellite Image Land Cover Classification) 3.1 Experimental Environment Hardware: Intel Core i9-13900HX CPU, NVIDIA RTX 4090 GPU (16GB VRAM), 32GB DDR5 RAM; Software: TensorFlow 2.10, Python 3.9, GDAL 3.6.2.

[0062] 3.2 Experimental Parameter Configuration Original image: High-resolution satellite imagery dataset, 1024×1024 (H×W) resolution, 3-channel RGB image; Scale levels: 0.5×, 1×, 2×, 4×, with target resolutions of 512×512, 1024×1024, 512×512, and 256×256, respectively. The target size of 0.5× is the result of dividing the width and height of the original image by 2. Channel attenuation coefficients: α_min=0.5 (N_min=0.5), α_max=0.95 (N_max=4). The intermediate scale is calculated as α_inter=α_min+(α_max-α_min)×(log2 (N)-log2 (N_min)) / (log2 (N_max)-log2 (N_min)), and the result is rounded to two decimal places: 1×→0.65, 2×→0.8; The preset target number of channels is C=128 (to meet the requirements of high-precision classification). Backbone network: Improved MobileNetV3-Small, 0.5× scale features generated by direct downsampling of the original image.

[0063] 3.3 Specific Implementation Steps S1. Data preprocessing: Normalize the pixel values ​​of the satellite image to [0,1] and adjust them using ImageNet mean and variance; calculate the target resolution at each scale, 0.5 × target size = 1024 / / 2 = 512 × 512.

[0064] S2, Backbone Network and 0.5× Feature Generation: The improved MobileNetV3-Small backbone network outputs a 1× basic feature map (16 channels) in layer 1, a 2× (24 channels) feature map in layer 2, and a 4× (40 channels) feature map in layer 3. The 0.5× scale feature map is generated by downsampling the original image through a depthwise separable convolution with a kernel of 3×3, padding of 1, and stride of 2 (16 channels). If the size does not match after downsampling, zero padding is only applied to the right and bottom sides of the image. When the padding amount is negative, the center is cropped to the target size to ensure that the core geographic information in the upper left corner is not destroyed.

[0065] S3. Differentiated Channel Pruning: Based on the SENet channel attention mechanism, channel weights at each scale are calculated, and channels are retained according to the attenuation coefficient. The number of channels is rounded to the nearest integer and is ≥8. 0.5×: 16×0.5=8 → Round to 8 channels (≥8, retain); 1×: 16×0.65=10.4 → Round to the nearest integer for 10 channels (≥8, keep); 2×: 24×0.8=19.2 → Round to the nearest integer 19 (≥8, keep); 4×: 40×0.95=38 → Round to 38 channels (≥8, keep).

[0066] S4. Channel Alignment and Bidirectional Fusion: The number of channels at each scale is unified to 128 through 1×1 pointwise convolution, and the convolution parameters are the same as in the previous embodiment. Bidirectional fusion covers 0.5×→1×→2×→4× (from bottom to top) and 4×→2×→1×→0.5× (from top to bottom). The 0.5× scale size matching only processes the lower right side of the image. The fusion process enables a shared storage mechanism to release unnecessary intermediate data in real time.

[0067] S5. Post-fusion processing: Perform BN+GELU activation on the fused feature map to output a multi-scale fused feature map with 128 channels and a resolution of 1024×1024.

[0068] S6. Scene Adaptation: Execute remote sensing scene supplementation rules to ensure the integrity of the core geographic information in the upper left corner at the 0.5× scale, and integrate the feature map into the subsequent classification network to complete land cover classification.

[0069] Example 4: Mobile Terminal Deployment Scenario (Real-time Image Recognition on Mobile Phone) 4.1 Experimental Environment Hardware: Mobile phone; Software: TensorFlow Lite 2.10, Android 14 system.

[0070] 4.2 Experimental Parameter Configuration Original image: Image taken with mobile phone, resolution 640×480 (H×W), 3-channel RGB image; Scale levels: general scene 1×, 2×, 4×, 8×, with target resolutions of 640×480, 320×240, 160×120, and 80×60 respectively; Channel attenuation coefficient: Log-linear interpolation calculation for general scenarios, 1×→0.3, 2×→0.41, 4×→0.61, 8×→0.8, the result is rounded to two decimal places; Preset target number of channels C=32 (adapts to edge memory / computing power limitations); Backbone network: Improved MobileNetV3-Small, quantized to INT8 precision for edge deployment, with the number of output channels of the downsampling module controlled to be 1.25 times that of the input.

[0071] 4.3 Specific Implementation Steps S1. Data preprocessing: Images captured by mobile phones are normalized to [0,1], adjusted according to ImageNet mean and variance, and the target resolution at each scale is calculated to adapt to the resolution ratio of mobile phone images.

[0072] S2. Backbone Network Feature Extraction: Features are extracted using the improved MobileNetV3-Small (INT8 quantization). Layers 1, 2, 3, and 5 output 1× (16 channels), 2× (24 channels), 4× (40 channels), and 8× (96 channels) basic feature maps, respectively. Quantization rules: Convolutional / fully connected layer weights are quantized with INT8, BN layer parameters are normalized and redundant parameters are removed, and the activation function output retains FP32 precision.

[0073] S3. Differentiated Channel Pruning: Channel weights are calculated based on SENet, and channels are retained according to the attenuation coefficient, following the rule that the number of retained channels is ≥8, with the number of channels rounded to the nearest integer. 1×: 16×0.3=4.8 → Forcefully retain 8 channels, adjust α to 0.5; 2×: 24×0.41=9.84 → rounded down to 10 channels; 4×: 40×0.61=24.4 → rounded down to 24 channels; 8×: 96×0.8=76.8→rounded to 77 channels.

[0074] S4. Channel Alignment and Bidirectional Fusion: The number of channels at each scale is unified to 32 through 1×1 pointwise convolution (GELU activation, no BN layer), and the convolution weights are designed with lightweight features to adapt to the device side; the bidirectional fusion rules are the same as in Example 1, and the fusion process strictly implements shared storage and real-time release to minimize the memory usage on the mobile device.

[0075] S5. Post-fusion processing: After the fused feature map is activated by BN+GELU, a 32-channel, 640×480 resolution fused feature map is output. The feature map is then lightweight compressed to adapt to mobile inference.

[0076] S6. End-to-end adaptation: The model removes all redundant parameters and retains only the necessary inference layers; the channel amplification ratio of the downsampling module is controlled at 1.25 times to reduce computational complexity; the feature map input is fused to the end-to-end lightweight classification network to achieve real-time image recognition on mobile devices.

[0077] Comparative Example 1: General Scenario (No Shared Storage Mechanism) 1.1 Experimental Environment Completely identical to Example 1: Intel Core i7-12700H CPU, NVIDIA RTX 3060 GPU (6GB VRAM), 16GB DDR4 RAM; PyTorch 2.0, Python 3.9, OpenCV 4.8.0.

[0078] 1.2 Experimental Parameter Configuration Except for removing the "shared storage and real-time release mechanism", the other parameters are completely consistent with those in Example 1: original image 224×224, 3 channels, scale level 1× / 2× / 4× / 8×, channel attenuation coefficient 1×→0.3, 2×→0.41, 4×→0.61, 8×→0.8, target number of channels C=64, improved MobileNetV3-Small backbone network.

[0079] 1.3 Specific Implementation Steps Key difference: In step S4, during the bidirectional fusion process, all basic feature maps at all scales and all intermediate fusion feature maps are retained until the entire bidirectional fusion process is completely completed before all intermediate data are released; the remaining steps (data preprocessing, backbone network feature extraction, channel pruning, channel alignment, and post-fusion processing) are completely consistent with Example 1.

[0080] 1.4 Experimental Results and Comparative Analysis Experimental results: Peak memory consumption reached 1.8GB, an increase of 119.5% compared to Example 1 (0.82GB); inference speed was 28FPS, a decrease of 30% compared to Example 1 (40FPS); ImageNet subset classification accuracy was 89.1%, basically the same as Example 1 (89.2%).

[0081] Comparative conclusion: After removing the shared storage mechanism, the feature extraction accuracy did not change significantly, but memory consumption increased dramatically and inference speed decreased significantly, proving that the shared storage and real-time release mechanism of the present invention can effectively reduce memory redundancy and improve storage and computing efficiency.

[0082] Comparative Example 2: Medical Imaging Scenarios (Channel-less Pruning Strategy) 2.1 Experimental Environment Completely identical to Example 2: Intel Xeon Gold 6330 CPU, NVIDIA A100 GPU (40GB VRAM), 64GB DDR4 RAM; PyTorch 2.1, Python 3.10, SimpleITK 2.2.1.

[0083] 2.2 Experimental Parameter Configuration Except for canceling channel pruning (attenuation coefficient α=1.0 for all scale channels, retaining all original channels), the other parameters are completely consistent with Example 2: lung CT images 512×512, 3 channels, scale level 1× / 2× / 4× / 8× / 16×, target number of channels C=128, improved EfficientNet-Lite1 backbone network.

[0084] 2.3 Specific Implementation Steps The core difference is that in step S3, the channel importance weight calculation and pruning operation based on SENet are not performed. The basic feature maps at each scale directly retain all the original channels (1×=16 channels, 2×=24 channels, 4×=40 channels, 8×=96 channels, 16×=86 channels) and enter step S4; the remaining steps are completely consistent with Example 2.

[0085] 2.4 Experimental Results and Comparative Analysis Experimental results: Peak memory consumption reached 4.2GB, an increase of 121% compared to Example 2 (1.9GB); inference speed was 15FPS, a decrease of 42.3% compared to Example 2 (26FPS); nodule detection sensitivity was 94.8% and specificity was 92.2%, which were basically the same as in Example 2.

[0086] Comparative conclusion: After canceling channel pruning, feature integrity was not affected, but memory consumption and computational burden increased significantly, proving that the differentiated channel pruning strategy of the present invention can effectively reduce storage volume and computational load while ensuring that core features are not lost.

[0087] Comparative Example 3: Mobile Terminal Scenario (Single-Directional Fusion Strategy) 3.1 Experimental Environment Completely identical to Example 4: mobile phone; TensorFlow Lite 2.10, Android 14 system.

[0088] 3.2 Experimental Parameter Configuration Except for changing bidirectional fusion to "top-down fusion only", the other parameters are completely the same as in Example 4: mobile phone captured image 640×480, 3 channels, scale level 1× / 2× / 4× / 8×, channel attenuation coefficient logarithmic linear interpolation calculation, target number of channels C=32, improved MobileNetV3-Small backbone network (INT8 quantization).

[0089] 3.3 Specific Implementation Steps Key difference: In step S4, only top-down fusion (8×→4×→2×→1×) is performed, and bottom-up fusion is not performed. After fusion, the process proceeds directly to step S5; the remaining steps are completely consistent with Example 4.

[0090] 3.4 Experimental Results and Comparative Analysis Experimental results: Peak memory consumption was 175MB, slightly lower than Example 4 (180MB) (2.8%); inference speed was 37FPS, slightly higher than Example 4 (35FPS) (5.7%); image recognition accuracy was 81.2%, lower than Example 4 (85.6%) (5.1%).

[0091] Comparative conclusion: While single-direction fusion can slightly reduce memory usage and improve speed, the lack of feature complementarity from low-scale to high-scale leads to a decrease in feature representation ability and a significant reduction in accuracy. The bidirectional fusion strategy of this invention can significantly improve feature integrity and recognition accuracy with only a slight increase in memory / computational costs, achieving a balance between efficiency and accuracy.

[0092] Comparative Example 4: Remote Sensing Image Scene (Fixed Scale Level, No Scene Adaptation) 4.1 Experimental Environment Completely identical to Example 3: Intel Core i9-13900HX CPU, NVIDIA RTX 4090 GPU (16GB VRAM), 32GB DDR5 RAM; TensorFlow 2.10, Python 3.9, GDAL 3.6.2.

[0093] 4.2 Experimental Parameter Configuration Key differences: The general scenario's "1× / 2× / 4× / 8×" scale levels are used, and a 0.5× scale feature map is not generated; the remote sensing 0.5× scale fill / cropping supplementation rules are not executed; the channel attenuation coefficient uses the general scenario's log-linear interpolation (N_min=1, N_max=8); the remaining parameters are the same as in Example 3.

[0094] 4.3 Specific Implementation Steps Key differences: Step S1 does not calculate the target resolution at a scale of 0.5×, and Step S2 does not generate a feature map at a scale of 0.5×; Step S4 uses center clipping / zero padding for general scenes when matching sizes, and does not execute the rule of "processing only the lower right side and retaining the core area at the upper left corner"; the remaining steps are completely consistent with Example 3.

[0095] 4.4 Experimental Results and Comparative Analysis Experimental results: Peak memory consumption was 2.6GB, an increase of 18.2% compared to Example 3 (2.2GB); the overall accuracy of land cover classification was 86.3%, a decrease of 5.2% compared to Example 3 (91.5%); the Kappa coefficient was 0.83, a decrease of 6.7% compared to Example 3 (0.89).

[0096] Comparative conclusion: Without scaling and rule adaptation for remote sensing scenes, not only does memory consumption increase, but it also fails to meet the needs of large-scale coarse-grained feature analysis of remote sensing images, resulting in incomplete extraction of core geographic information and a significant decrease in classification accuracy. The scene customization strategy of this invention can accurately match the characteristics of images in different domains, improving the targeting and effectiveness of feature extraction.

[0097] Example 5 The present invention also relates to an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the method described above.

[0098] The electronic device can be a desktop computer, laptop, handheld computer, or cloud server, etc. The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The memory can be used to store computer programs and / or modules. The processor performs various functions of the electronic device by running or executing the computer programs and / or modules stored in the memory, and by accessing data stored in the memory.

[0099] Example 6 The present invention also relates to a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described above.

[0100] Specifically, the memory may include high-speed random access memory, as well as non-volatile memory, such as hard disks, RAM, plug-in hard disks, smart media cards (SMC), secure digital (SD) cards, flash cards, at least one disk storage device, flash memory device, or other volatile solid-state storage devices.

[0101] Example 7 This invention provides a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the steps of the method described in the above embodiments of this invention.

[0102] The technical features of the embodiments described above can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as the combination of these technical features does not contradict each other, it should be considered within the scope of this specification. It should be noted that the terms "in one embodiment," "for example," and "again" in this invention are intended to illustrate the invention and are not intended to limit the invention.

[0103] The embodiments described above are merely examples of several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention.

Claims

1. A multi-scale image feature extraction and processing method, characterized in that, include: The pixel values ​​of the image are normalized to the range [0, 1], and mean and variance are adjusted using ImageNet. Define N×scale = original image resolution / feature map resolution, where N is the scale coefficient, N = 1, 2, 4, 8...; Based on the preset set of scale coefficients, calculate the target resolution of the feature map at each scale. Target resolution = original image resolution / N; The adjusted image is input into a lightweight backbone network, and basic feature maps at various scales are output through different layers of the lightweight backbone network; the lightweight backbone network includes multiple layers corresponding to preset scale levels; Differentiated channel attenuation coefficients are set for the basic feature maps at each scale. The channel attenuation coefficient α is defined as the number of retained channels of the feature map / the original number of channels of the feature map. Based on the SENet channel attention mechanism, the channel importance weights are calculated separately for the basic feature maps at each scale. The channels with the highest weights are retained (α×100%). The number of retained channels is taken as the integer closest to α×the original number of channels, and the pruned feature maps at each scale are obtained. After pruning, feature maps at each scale are aligned by channel, and the number of channels in all feature maps is unified to the preset target number of channels C. The feature maps are then fused from high-scale coefficients to low-scale coefficients and then from low-scale coefficients to high-scale coefficients. When fusing from top to bottom, only the feature map of the current target scale and the high-scale feature map to be upsampled are retained, and the rest of the data is released in real time after fusion. When fusing from bottom to top, only the feature map of the current target scale and the low-scale feature map to be downsampled are retained, and the rest of the data is released in real time after fusion. The fused feature maps are sequentially processed by BN and GELU activation to output a multi-scale fused feature map.

2. The multi-scale image feature extraction and processing method as described in claim 1, characterized in that, The lightweight backbone network consists of 11 layers, all of which adopt a standardized design of convolution / depthmality separable convolution, BN layers, and activation functions. The first 3 layers use the ReLU activation function, and the last 8 layers use the GELU activation function, ultimately outputting a high-dimensional feature map with fixed channels.

3. The multi-scale image feature extraction and processing method as described in claim 1, characterized in that, The 11-level structure is as follows: The first layer is a standard two-dimensional convolutional layer with 3 input channels and 16 output channels. The kernel size is 3×3, the stride is 1, and the padding is 1. It is followed by BN processing and ReLU activation function. The output feature map resolution is the same as the input image, with a height of H and a width of W. The second layer is a depthwise separable convolutional layer. The depthwise convolutional part has 16 input channels and 16 output channels, with a kernel size of 3×3, a stride of 2, and padding of 1. The pointwise convolutional part has 16 input channels and 24 output channels, followed by BN processing and ReLU activation function. The output feature map resolution is H / 2 in height and W / 2 in width. The third layer is a depthwise separable convolutional layer. The depthwise convolutional part has 24 input channels and 24 output channels, with a kernel size of 5×5, a stride of 2, and padding of 2. The pointwise convolutional part has 24 input channels and 40 output channels, followed by BN processing and ReLU activation function. The output feature map resolution is H / 4 in height and W / 4 in width. The fourth layer is a depthwise separable convolutional layer. The depthwise convolutional part has 40 input channels and 40 output channels, with a kernel size of 3×3, a stride of 1, and padding of 1. The pointwise convolutional part has 40 input channels and 48 output channels, followed by BN processing and the GELU activation function. The output feature map resolution is H / 4 in height and W / 4 in width. The fifth layer is a depthwise separable convolutional layer. The depthwise convolutional part has 48 input channels and 48 output channels, with a kernel size of 5×5, a stride of 2, and padding of 2. The pointwise convolutional part has 48 input channels and 96 output channels, followed by BN processing and the GELU activation function. The output feature map resolution is H / 8 in height and W / 8 in width. The sixth layer is a depthwise separable convolutional layer. The depthwise convolutional part has 96 input channels and 96 output channels, with a kernel size of 3×3, a stride of 1, and padding of 1. The pointwise convolutional part has 96 input channels and 128 output channels, followed by BN processing and the GELU activation function. The output feature map resolution is H / 8 in height and W / 8 in width. Layer 7: This is a depthwise separable convolutional layer. The depthwise convolutional part has 128 input channels and 128 output channels, with a kernel size of 3×3, a stride of 2, and padding of 1. The pointwise convolutional part has 128 input channels and 160 output channels, followed by BN processing and the GELU activation function. The output feature map resolution is H / 16 in height and W / 16 in width. Layer 8: This is a depthwise separable convolutional layer. The depthwise convolutional part has 160 input channels and 160 output channels, with a kernel size of 5×5, a stride of 1, and padding of 2. The pointwise convolutional part has 160 input channels and 192 output channels, followed by BN processing and the GELU activation function. The output feature map resolution is H / 16 in height and W / 16 in width. Layer 9: This is a depthwise separable convolutional layer. The depthwise convolutional part has 192 input channels and 192 output channels, with a kernel size of 3×3, a stride of 1, and padding of 1. The pointwise convolutional part has 192 input channels and 224 output channels, followed by BN processing and the GELU activation function. The output feature map resolution is H / 16 in height and W / 16 in width. Layer 10: This is a depthwise separable convolutional layer. The depthwise convolutional part has 224 input channels and 224 output channels, with a kernel size of 3×3, a stride of 1, and padding of 1. The pointwise convolutional part has 224 input channels and 256 output channels, followed by BN processing and the GELU activation function. The output feature map resolution is H / 16 in height and W / 16 in width. The eleventh layer is a 1×1 two-dimensional convolutional layer with 256 input channels and 128 output channels. The kernel size is 1×1, the stride is 1, and it is followed by BN processing and GELU activation function. The output feature map resolution is H / 16 in height and W / 16 in width.

4. The multi-scale image feature extraction and processing method as described in claim 1, characterized in that, The activation function of the lightweight improved backbone network is configured as follows: The first three core layers use ReLU activation, and the last eight core layers use GELU activation; the activation functions are all attached after the last BN layer of each layer. The structure of depthwise separable convolution is a concatenation of depthwise convolution, BN layer and pointwise convolution, BN layer; The downsampling module consists of depthwise separable convolutional layers with a stride of 2 and BN layers, with the number of output channels being 1.25 to 2 times the number of input channels.

5. The multi-scale image feature extraction and processing method as described in claim 1, characterized in that, The SENet-based channel attention mechanism calculates channel importance weights separately for each scale of the base feature map, including: The channel statistical vector S is calculated from the basic feature maps at each scale using global average pooling. Channel weights Z = σ(W2 × δ(W1 × S + b1) + b2) are generated through two fully connected layers and activation functions. The two fully connected layers include a dimensionality reduction fully connected layer and a dimensionality increase fully connected layer. W1 is the weight of the dimensionality reduction fully connected layer, which is used to reduce the dimension from the original number of channels to max(original number of channels / / 16, 1)). W2 is the weight of the dimensionality increase fully connected layer, which is used to increase the dimension from max(original number of channels / / 16, 1) back to the original number of channels. δ is the ReLU activation function, and σ is the Sigmoid activation function.

6. The multi-scale image feature extraction and processing method as described in claim 1, characterized in that, The process of fusing high-scale coefficients to low-scale coefficients and then fusing low-scale coefficients back to high-scale coefficients includes: The high-scale feature map is upsampled to the target scale through a transpose depthwise convolution with a 3×3 kernel, stride of 2, padding of 1, and output padding of 1. If the size of the upsampled feature map is inconsistent with the target scale feature map, it is matched by center pruning or edge zero padding, and then added element by element with the feature map after corresponding scale pruning and channel alignment. The low-scale feature map is downsampled to the target scale by a depthwise separable convolution with a kernel size of 3×3, padding of 1, and stride of 2. The size after downsampling follows the specific calculation standard of conventional convolution output: first, add twice the padding value to the input size, then subtract the convolution kernel size, divide the result by the stride and round down, and finally add 1 to get the final output size. Then, multiply it element-wise with the feature map after corresponding scale pruning and channel alignment.

7. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1 to 6.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 6.

9. A computer program product, comprising a computer program or instructions, characterized in that, When the computer program or instructions are executed by a processor, they implement the steps of the method as described in any one of claims 1 to 6.

Images