Single-scan mamba feature extraction method and system for image global modeling

Abstract

The application provides a single-scan Mamba feature extraction method and system for image global modeling, comprising: acquiring an input image, extracting a multi-channel feature map with a channel dimension and a spatial dimension through a convolution operation; performing state space modeling of single forward calculation on the multi-channel feature map to model the global dependency relationship between channels, wherein the state space modeling is realized by performing matrix operation on the feature after spatial dimension flattening and a channel interaction matrix constructed based on state space model parameters; and outputting the feature map after global dependency modeling, and applying the feature map to a downstream visual task. The application eliminates redundant calculation caused by multiple spatial scans, simplifies the model structure, ensures the performance of image classification and other downstream tasks, and significantly reduces the computational complexity and inference delay.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of computer vision and deep learning, and particularly relates to a single-scan Mamba feature extraction method and system for image global modeling. BACKGROUND

[0002] In recent years, using convolutional neural networks and visual Transformers to extract image features has become the mainstream technical route. Convolutional neural networks can effectively capture local spatial features through local receptive fields and convolution operations, but have limitations in capturing the feature relationships of distant regions in an image. The effective receptive field is difficult to cover the full image range, especially in high-resolution images, and the modeling capability of long-distance semantic association is limited. Visual Transformers divide the image into consecutive local pixel blocks and flatten them into one-dimensional sequences, establish global relationships using self-attention mechanisms, and have strong expression capabilities. However, the computational complexity increases quadratically with the image resolution, which limits the efficiency in high-resolution scenarios and lightweight deployment requirements, making it difficult to balance performance and resource consumption.

[0003] To solve the above problems, a visual Mamba feature extraction method based on state space model is proposed. This method first divides the input image into several pixel blocks and flattens them into a one-dimensional label sequence, then models the long sequence through a linear state transition mechanism, thus establishing a global range of feature relationships while maintaining linear time complexity, and showing significant advantages in parameter efficiency and long sequence processing capability. However, existing visual Mamba methods usually rely on spatial scanning strategies (such as row, column, Z-type or two-dimensional plane sequential expansion) to convert images into label sequences, and then perform state propagation or convolution calculation. Although this kind of spatial scanning strategy can explicitly encode spatial position information, it has the following limitations in practice: First, there is an essential adaptation conflict between the two-dimensional spatial structure inherent in the image and the one-dimensional sequence processing after scanning. Single-direction scanning easily destroys the spatial adjacency between regions, resulting in insufficient modeling of cross-region dependencies. Therefore, the model often needs to perform repeated scanning in multiple directions or multiple scales, and the scanning results in different directions have high similarity, causing the label sequence to be processed redundantly multiple times, significantly increasing the training and inference cost. Second, spatial scanning usually combines spatial position priors, multi-path fusion or complex position encoding strategies. Although these designs can compensate for the loss of spatial structure to some extent and improve feature expression ability, they significantly increase the complexity of the model structure and the difficulty of parameter adjustment, and may also introduce the risk of overfitting. Third, the causal feature of traditional spatial scanning makes the model only rely on the information of the previous pixels when processing the sequence, and cannot effectively utilize the semantic association of the subsequent unscanned regions. Moreover, the interaction of long-distance pixels will decay with the increase of sequence length, further limiting the complete capture of global context. At the same time, in the high-resolution input and real-time inference scenarios, multiple scanning and multi-branch fusion will significantly increase the delay and memory occupancy, making it difficult to meet the lightweight deployment requirements of mobile terminals, edge computing and other scenarios, limiting the practical application range of the method. SUMMARY

[0004] To address the shortcomings and deficiencies of existing technologies, this invention provides a single-scan visual Mamba feature extraction method and system for efficient global image modeling. It aims to solve the problems of redundant computation, structural complexity, and deployment limitations caused by multiple scans in the spatial dimension in existing visual Mamba methods. This method constructs a marker sequence along the channel dimension and achieves global context modeling using a single scan. Specifically, it includes: first, mapping the input image to a high-dimensional feature map through convolution operations; then, extracting local spatial features and optimizing feature propagation through initial convolutional units and convolutional modules; the core lies in the channel scan feature modeling part, which dynamically generates the parameter matrix and discretized time step parameters of the state space model through layer normalization and convolution operations. Combined with learnable state weight parameters, a discretized state matrix is ​​obtained after normalization. Then, a channel interaction matrix is ​​constructed through element-wise multiplication. After flattening the high-dimensional feature map, matrix multiplication is performed with the transpose of the channel interaction matrix and the output projection parameters. Global information is aggregated, and the final features are output through residual connections and regularization. By adjusting the channel dimension, network depth, and state space parameter configuration, multiple model variants can be generated to adapt to different computing power platforms. This invention eliminates redundant computation caused by multiple spatial scans, simplifies the model structure, and significantly reduces computational complexity and inference latency while ensuring the performance of downstream tasks such as image classification. The training process uses a public dataset and achieves end-to-end optimization through the cross-entropy loss function and gradient descent optimizer, which has good practicality and deployment flexibility.

[0005] The specific technical solution adopted by this invention to solve its technical problem is as follows:

[0006] A single-scan Mamba feature extraction method for global image modeling includes:

[0007] The input image is acquired, and a multi-channel feature map with channel and spatial dimensions is extracted through convolution operations.

[0008] A state space modeling process is performed on the multi-channel feature map using a single forward computation to model the global dependencies between channels. The state space modeling is achieved by performing matrix operations on the features after flattening the spatial dimensions and the channel interaction matrix constructed based on the state space model parameters.

[0009] The output feature map, after global dependency modeling, is applied to downstream vision tasks.

[0010] Furthermore, the extraction process of the multi-channel feature map is as follows: the input image is sequentially mapped and optimized through at least two cascaded convolutional layers, and each convolutional layer is spatially downsampled to finally obtain a multi-channel feature map with a preset number of channels and spatial size.

[0011] Furthermore, the specific process of state space modeling includes:

[0012] After performing layer normalization on the multi-channel feature map, the input projection matrix, output projection matrix, and discretized time step parameters are generated by sequentially performing 1×1 convolution and group convolution operations. The state space model also includes a learnable autoregressive weight matrix.

[0013] After adding the discretized time step parameter to the autoregressive weight matrix, the discretized state matrix is ​​obtained by processing it through a normalization function. Then, the discretized state matrix is ​​multiplied element-wise with the input projection matrix to obtain the channel interaction matrix.

[0014] First, the features flattened in the spatial dimension are multiplied by the transpose of the channel interaction matrix to obtain intermediate features; then, the intermediate features are multiplied by the output projection matrix to generate a flattened global feature sequence.

[0015] After reshaping the global feature sequence back to its original spatial dimension, feature transformation is performed through convolution, and then residual connection is performed with the multi-channel feature map. Combined with path dropout regularization, the feature map after global dependency modeling is obtained.

[0016] Furthermore, the state space modeling is achieved through a network architecture consisting of multiple core stages. Each core stage includes a convolution module and a single forward computation state space modeling for the channel dimension. The convolution module is a residual block structure that sequentially performs 1×1 convolution for channel mapping, group convolution, and 1×1 convolution for channel restoration on the input features. The output is fused with the original input features through residual connections, and a path dropping mechanism is set at the output end of the main convolution path.

[0017] Furthermore, by adjusting the channel dimension, network block number, and state space parameter configuration of each core stage, model variants adapted to different computing power platforms are formed, with different configuration combinations for the channel dimension, network block number, and state space parameters of each model variant.

[0018] Furthermore, the downstream visual tasks include image classification and / or image retrieval.

[0019] Furthermore, the difference between the predicted results and the true labels is calculated using the cross-entropy loss function, and the learnable parameters of the state-space model are optimized end-to-end using the AdamW optimizer. A dynamic learning rate adjustment strategy is adopted during the training process.

[0020] And, a single-scan Mamba feature extraction system for global image modeling, comprising:

[0021] The feature extraction module is used to acquire the input image and extract multi-channel feature maps with channel and spatial dimensions through convolution operations;

[0022] The global modeling module is used to perform a single forward computation on the multi-channel feature map to model the state space, so as to model the global dependencies between channels. The state space modeling is achieved by performing matrix operations on the features after flattening the spatial dimension and the channel interaction matrix constructed based on the state space model parameters.

[0023] The output application module is used to output feature maps after global dependency modeling and adapt them to downstream vision tasks.

[0024] And a computer device including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the method described above.

[0025] A non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described above.

[0026] Compared with the prior art, the present invention and its preferred embodiments have at least the following beneficial effects:

[0027] First, by using channel-dimensional scanning as the sequential scanning direction and performing a single forward computation for state space modeling, this invention effectively avoids the redundant computations caused by multi-directional, multi-scale spatial scanning, which is a limitation of existing visual Mamba-like methods. This significantly simplifies the model structure and parameter tuning, reduces computational complexity and time latency during training and inference, and is more suitable for the needs of high-resolution image processing and real-time inference scenarios. Second, the channel-dimensional modeling approach eliminates the dependence on fixed spatial priors, ensuring that each channel feature is equally important. This allows for a more comprehensive aggregation of global contextual information across regions and channels. Combined with the convolutional module's ability to extract local spatial features and textures, it achieves an organic fusion of local features and global dependencies, improving the discriminative and expressive power of image features. Third, the multi-stage network architecture and flexibly adjustable model variant design enable this invention to adapt to different channel dimensions, network block numbers, and state space parameter configurations, meeting the deployment requirements of various computing platforms from mobile devices to high-performance servers, thus enhancing the practicality and scalability of the technical solution. Furthermore, the combination of end-to-end training strategies with optimization mechanisms such as residual connections and path dropout regularization ensures the stability and generalization ability of model training. The trained model can be flexibly applied to various downstream visual tasks such as image classification and image retrieval, which enhances the application value of the technical solution while taking into account feature extraction performance. Attached Figure Description

[0028] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments:

[0029] Figure 1This is a block diagram of the overall structure of the visual Mamba network according to an embodiment of the present invention;

[0030] Figure 2 This is a schematic diagram of the convolution module structure according to an embodiment of the present invention;

[0031] Figure 3 This is a flowchart illustrating the structure of the channel scanning feature modeling section in an embodiment of the present invention. Detailed Implementation

[0032] To make the features and advantages of the present invention more apparent and understandable, specific embodiments are described below in detail:

[0033] It should be noted that the following detailed descriptions are exemplary and intended to provide further explanation of this application. Unless otherwise specified, all technical and scientific terms used in this specification have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains.

[0034] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the exemplary embodiments according to this application. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0035] To address the problem that existing visual Mamba models rely on multiple selective scanning mechanisms to capture sufficient spatial features during image feature extraction, resulting in a significant reduction in training and inference efficiency, this invention proposes a novel visual Mamba feature extraction method and a corresponding novel and efficient single-scan visual Mamba feature extraction network. While maintaining global context modeling capabilities, it avoids reliance on multiple scans and complex location priors, thereby improving the efficiency and deployability of image feature extraction while ensuring image feature discriminability.

[0036] This scheme significantly improves computational efficiency while maintaining feature representation capabilities by eliminating redundant scanning branches. Existing visual Mamba-like methods typically flatten spatial features into a sequence of labels and rely on prior spatial locations to perform multiple selective scans; however, this invention constructs the label sequence along the channel dimension. In deep learning networks, different channels of feature maps often correspond to semantically independent abstract patterns (such as textures, edges, or object parts). These patterns are shared across the entire spatial plane and do not depend on fixed spatial order information. Therefore, when constructing a sequence along the channel dimension, channel features naturally do not possess prior spatial locations, all labels are equally important, and effective feature modeling can be achieved with only one scan.

[0037] The implementation process of this invention involves the following steps:

[0038] Step 1: Prepare the image dataset; preprocess the ImageNet-1K image dataset used for network model training, including image size normalization, data augmentation, and other operations.

[0039] Step 2: As Figure 1 As shown, a multi-stage visual Mamba network consisting of a state-space model with channel dimension scanning is constructed and trained using the dataset prepared in step 1.

[0040] Step 3: Apply the trained network model to downstream visual tasks to verify its performance; for example, in image classification tasks, after inputting an image to be classified, the model outputs the corresponding category prediction result. The discriminative ability of the proposed method in extracting features is verified by the accuracy of the classification prediction; simultaneously, the computational efficiency of the proposed method while maintaining accuracy is verified by comparing the number of model parameters and the computational cost.

[0041] In a preferred embodiment, in step 1, the image dataset required for network training in step 2 is established. Specifically:

[0042] Download the publicly available ImageNet-1K dataset. The ImageNet-1K dataset is a large-scale visual recognition challenge dataset containing a large number of high-resolution images covering a wide range of object categories. This dataset is primarily used for training and performance validation of image feature extraction networks.

[0043] As a preferred embodiment, step 2 is as follows:

[0044] Step 2.1: Construct a state-space model based on channel scanning; First, unlike the state-space model that typically uses multiple selective scanning, the input image... Dividing the spatial dimension into multiple pixel blocks, this invention directly utilizes convolution operations to process the input image. Mapping to high-dimensional features This allows for explicit decoupling of representations of more visual channel dimensions. The intrinsic parameter matrix of the state-space model includes (A, B, c, ...). Then, convolution operations are used to generate the necessary parameter matrix suitable for channel scan feature modeling; finally, the input features are used... Drive a channel-scan-based state-space model to model the correlation between information from different visual channels. Obtain output features containing global information. .

[0045] Step 2.2: Construct a method for processing the input image The initial convolutional unit performs preliminary feature extraction and high-dimensional mapping; as the first part of the network, this module consists of two concatenated convolutional layers, used to extract basic features from the image and perform downsampling twice, preparing input for subsequent layers. The first convolutional layer maps the three RGB channels of the image to higher feature channels, capturing the most basic features; the second convolutional layer further expands the number of channels and combines and optimizes the features extracted by the first layer. Thus, the original input image... This is then transformed into high-dimensional features. .

[0046] Step 2.3: Constructing features for extraction Convolutional modules that perform feature transformation and refinement, such as Figure 2 As shown;

[0047] In this embodiment, the convolution module is designed as a residual block, specifically responsible for extracting local spatial information and texture. Its internal data flow path is as follows: first, it passes through a... Convolution performs channel mapping; then, a grouped convolutional layer is used to extract features at a lower computational cost; finally, another convolutional layer is used... Convolution maps features back to the target channel dimension. At the output of the convolutional module, random path dropout regularization is introduced, which randomly discards a path during training with a certain probability, thereby enhancing the model's generalization ability. Furthermore, this module retains a bypass path with an identity mapping at the output, which is added to the output of the convolutional backbone to form a residual connection, ensuring effective gradient propagation.

[0048] Step 2.4: Construct the core channel scanning feature modeling part of this invention, such as... Figure 3 As shown;

[0049] Step 2.4.1: Construct the convolutional part of the necessary parameter matrix for generating the channel scan;

[0050] To achieve state-space modeling based on channel scanning, this invention designs a set of convolutional modules at the input end of the state-space model for dynamically generating the necessary parameter matrix. Specifically, firstly, the input features... The execution layer normalizes to stabilize the feature distribution and eliminate scale differences between different channels; then it passes through a... The convolutional layer normalizes the feature maps. Channel dimension Mapped to state space dimension Three times the size, generating intermediate feature maps .

[0051] Next, a grouped convolutional layer is used to model local feature relationships while maintaining channel independence; finally, the processed feature maps are... The matrix is ​​uniformly divided into three parts along the channel dimension, with the first two parts serving as the parameter matrix. and One part is responsible for the input and output projection of the state space; the other part serves as the discretization time step parameter. Where L = H×W. Meanwhile, the parameter matrix of the state-space model... Defined as a learnable parameter, its initial shape is .

[0052] Step 2.4.2: Correlation modeling of channel scanning.

[0053] After determining the parameter matrix, the features of the input image are used. Drive the state-space model to achieve dynamic correlation modeling across channels and calculate output features. First, use dynamically generated right Discretize and apply The function guarantees stability, resulting in a discretized state matrix. :

[0054]

[0055] in, Its function is to normalize the discretized state matrix, ensuring the numerical stability of the state propagation process.

[0056] Next, With dynamically generated Perform element-wise multiplication to obtain the weighted channel interaction matrix. :

[0057]

[0058] in This indicates bitwise multiplication. Next, the input feature map will be... Flattened in spatial dimension The aggregation of global information is accomplished through the following matrix multiplication sequence:

[0059]

[0060] in Represents matrix multiplication. This indicates transpose. and Matrix multiplication yields Then, the intermediate results With parameter matrix Multiply to obtain the flattened output. :

[0061]

[0062] These two matrix multiplication steps are equivalent to scanning the input spatial feature sequence along the channel dimension. As a mathematical implementation of a single channel scan, performing matrix multiplication essentially involves allowing the spatial information of each channel to interact with the spatial information of other channels sequentially. Therefore, this step is used to aggregate spatial feature information across different visual channels, unfolding the two-dimensional channel-space features into a sequence along the channel dimension, achieving progressive scanning and global information aggregation along the channel direction.

[0063] After performing matrix multiplication, the calculated result will be flattened and output. Reshaping to the spatial dimensions of the original image And then through a Convolution is used for the final feature transformation. Finally, the result of the convolution transformation is connected to the original input features via residual connections. The results are added together and regularized using a path discarding mechanism to obtain the final output of the module. .

[0064] Step 3: Network model training and application;

[0065] The final feature map after all stages of processing The network uses a global average pooling layer and a classification head to predict the image's category. Classification is used as a proxy task to supervise training the network and improve its image feature representation capabilities. The training process involves calculating the difference between the predicted and true labels using the cross-entropy loss function, and then using the gradient descent optimizer AdamW to optimize all learnable parameters of the network (including those in the channel scan feature modeling part). The network (including matrix and parameter generation, network weights, etc.) is optimized end-to-end. After training, the network can be used for downstream tasks such as classification and retrieval that require feature extraction from images.

[0066] Compared with existing technologies, this invention significantly reduces redundant calculations by eliminating multiple spatial scans and avoids position-dependent priors, achieving a simpler channel labeling processing method and demonstrating excellent performance in tasks such as image classification.

[0067] This invention proposes an image feature extraction method based on the visual Mamba architecture, which involves scanning sequential information along the channel dimension. Benefiting from the spatial independence and weight equality of channel-dimensional features, this method achieves fast and effective global modeling with a single scan, reducing the time delay and computational complexity of image feature extraction and promoting the deployment of image feature extraction systems.

[0068] The following is a more specific test example to further demonstrate and introduce the solution of the present invention:

[0069] First, the following definitions and explanations are provided:

[0070] Input image It is a three-channel RGB image format, which is sent to the network input after high-dimensional mapping.

[0071] : Represents the input feature map of any module in the network, with dimensions (B, C, H, W), representing batch size, number of channels, height, and width, respectively.

[0072] N, L: N is the dimension of the state space, representing the implicit state length of each channel when modeling the state; L is the total number of spatial locations, which is equal to the height × width of the input feature map, i.e., L = H × W.

[0073] A, B, c, The core parameters in the state-space model. The autoregressive weights representing the channel states are used to control the changing trend of channel features with each scan step. (Matrix) These correspond to the input projection and output projection matrices, respectively, and are used to establish the mapping relationship between input features and state updates. This represents the discretization time step, and its value determines the rate and stability of state updates.

[0074] q: , as input features Stratigraphic normalization and The intermediate feature map obtained after convolution is used to generate the state space parameters (B, c, ...). The intermediate features of ).

[0075] The basic steps of the implementation process are as follows:

[0076] Step S1: Create the image dataset required for network training.

[0077] This embodiment uses the publicly available large-scale image classification dataset ImageNet-1K. This dataset contains approximately 1.28 million training images and 50,000 validation images, covering a total of 1,000 categories. All images are preprocessed, including uniformly adjusting the input images to 224×224 pixels, applying data augmentation strategies such as random cropping and random horizontal flipping to improve the model's generalization ability, and normalizing pixel values.

[0078] Step S2: Construct and train the visual Mamba network. This invention adopts a hierarchical hybrid architecture, and its detailed construction process is as follows:

[0079] Step S2.1: Construct the overall network architecture.

[0080] The overall architecture of the visual Mamba network in this embodiment is composed of the following parts connected in sequence, such as... Figure 1 As shown:

[0081] 1. Initial Convolutional Layer: Consists of two cascaded convolutional layers. The first convolutional layer maps the input three-channel image to a feature representation of a preset dimension and performs spatial downsampling through convolution with a stride of 2. The second convolutional layer further increases the number of feature channels to a higher dimension while performing spatial downsampling again. This part completes the initial feature extraction and two downsampling operations on the original image.

[0082] 2. Three core stages: Each stage begins with a convolutional module to further compress the spatial dimensions and increase the number of channels. Following this, a channel-scanning feature modeling section enables the network to extract local spatial features through convolution at each layer, while also achieving fast and efficient global context modeling through a single channel scan. This design avoids the multiple spatial scans found in traditional visual Mamba methods, significantly improving computational efficiency. Furthermore, as shown in Table 1, by adjusting the network depth, channel dimension, and state space parameter configurations in each stage, this invention also generates several variants (M1~M4) to flexibly adapt to the needs of different computing platforms, allowing deployment from lightweight mobile devices to high-performance servers, demonstrating excellent scalability.

[0083] Table 1. Configuration parameter table for different model variants (M1~M4) of this method

[0084]

[0085] Step S2.2: Construct the convolutional module, such as... Figure 2 As shown.

[0086] This module is a residual structure, specifically implemented as follows:

[0087] Input feature map Firstly, through a The number of channels is adjusted using convolution; then grouped convolution is used (the number of convolution groups is set proportionally to the number of intermediate channels). Convolutional kernels are used to extract spatial features; this design maintains the ability to model spatial information while reducing the number of parameters. Subsequently, another... The number of channels is recovered from the convolution. After this, random path dropout regularization is applied to the output of the main convolution path, randomly discarding the output of the entire convolution path with a certain probability. Finally, the output of this path is combined with the original input passed through the bypass residual connection. Add them together.

[0088] Step S2.3: Construct the core channel scan state space, such as... Figure 3 As shown.

[0089] This module is the core technology of this invention, and its internal workflow is as follows:

[0090] 1. Input Feature Processing: Input a two-dimensional feature map First, layer normalization is performed to stabilize the feature distribution.

[0091] 2. Dynamic parameter generation: The normalized feature map is generated through a... Convolutional layers, which convert channel dimensions Mapping to three times the dimension N of the state space generates intermediate feature maps. Next, a grouped convolutional layer is used to model local feature relationships while maintaining channel independence.

[0092] 3. Parameter decoupling and allocation: The processed feature map It is evenly divided into three parts along the channel dimension:

[0093] The first two parts serve as parameter matrices. and (L = H×W, representing the total number of spatial locations), together responsible for the input and output projection of the state space. The latter part serves as the discretization time step parameter. Meanwhile, the parameter matrix of the state-space model Defined as a learnable parameter, its initial shape is .

[0094] 4. Correlation modeling of channel scanning: After determining the parameter matrix, the input features are used... Drive the state-space model to achieve dynamic correlation modeling across channels and calculate output features. By using the following formula Discretize and ensure numerical stability:

[0095]

[0096] in The purpose of this operation is to limit the state propagation coefficient to remain stable within a numerical range, so that the state updates of each channel are dynamically balanced between 0 and 1, thereby preventing gradient explosion or vanishing problems when multiple layers are stacked.

[0097] After discretization, the obtained In essence, this can be understood as a dynamic weight distribution among a set of channels, characterizing the degree of dependence between different channel features. Next, we will... and Perform element-wise multiplication to obtain the weighted channel interaction matrix:

[0098]

[0099] in," " indicates bitwise multiplication. This operation is equivalent to adaptively adjusting the input projection intensity in each state dimension, thereby constructing channel association weights for subsequent global scanning.

[0100] During the scanning phase, the input feature map is... Flattened into a sequence At this time, the channel dimension Considered as the main axis of the scan, while the spatial dimension These are mapped to sequence elements. Global information aggregation and propagation are achieved through the following matrix multiplication sequence:

[0101]

[0102]

[0103] in" " indicates matrix multiplication. First step" The first step is to perform weighted fusion of the input channel features and the channel interaction matrix to achieve aggregation of full-space features from each channel; the second step... This completes the projection of the state into the output space, generating a feature sequence containing global semantic information. Through the above two steps, each channel completes the interaction with all other channels in a single scan, forming a global feature dependency relationship.

[0104] Intuitively, a single channel scan in this embodiment can be viewed as "recursive state propagation along the channel dimension," meaning the model sequentially uses channel features as state inputs, and through... The matrix controls its changing trend, through The propagation rate is adaptively adjusted to achieve dynamic global modeling within the channel domain. This design breaks away from the fixed path of traditional visual Mamba methods that rely on two-dimensional spatial sequence expansion, avoiding redundant computations caused by multiple scans and direction dependence, while retaining the global modeling characteristics of the state-space model. A comparison of the scanning mechanisms of this method with existing Mamba methods is shown in Table 2.

[0105] Table 2. Comparison of scanning mechanisms between existing visual Mamba series methods and our proposed method.

[0106]

[0107] 5. Feature Transformation and Residual Connection: This involves transforming the output... Through a Convolution performs the final feature transformation. Finally, the transformed result is connected to the original input features via a residual concatenation. The results are added together and regularized using a path discarding mechanism to obtain the final output of the module. .

[0108] Step S2.4: Model training.

[0109] In this embodiment, the model is trained using a classification task as a proxy, with ImageNet-1K as the training dataset. During training, the feature maps processed through all stages are input into a global average pooling layer to obtain a one-dimensional feature vector, which is then fed into an MLP classifier to output the final class prediction. This embodiment uses the cross-entropy loss function as the optimization objective and employs the AdamW optimizer to update all learnable parameters end-to-end. The initial learning rate is set to 0.001, the total batch size is set to 1024, the training epochs are 300, and a cosine annealing strategy is used for dynamic adjustment. With this training configuration, the model can converge stably on a large-scale dataset and learn robust feature representations.

[0110] Step S3: Model application and performance verification.

[0111] The trained network was then used to verify its performance using an image classification task.

[0112] Taking the ImageNet-1K classification task as an example, the model takes an image as input and outputs a 1000-dimensional probability vector. The category corresponding to the highest probability is the prediction result.

[0113] The experimental results are shown in Table 3. The present invention outperforms the existing mainstream models in terms of the balance between accuracy, number of parameters and floating-point operation.

[0114] Table 3 compares the accuracy, parameter count, and floating-point number of our proposed method with existing mainstream CNN / Transformer / Mamba models on the ImageNet-1K dataset.

[0115]

[0116] In summary, this embodiment achieves a highly efficient visual backbone network for a single scan by shifting global dependency modeling to the channel dimension. It combines structural simplicity, computational efficiency, and cross-task adaptability, and has significant research and application value.

[0117] Based on the same inventive concept, this invention also provides a computer device, comprising: one or more processors, and a memory for storing one or more computer programs; the programs include program instructions, and the processor executes the program instructions stored in the memory. The processor may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. It is the computing and control core of the terminal, used to implement one or more instructions, specifically for loading and executing one or more instructions stored in a computer storage medium to implement the above-described method.

[0118] It should be further explained that, based on the same inventive concept, the present invention also provides a computer storage medium storing a computer program, which, when executed by a processor, performs the above-described method. This storage medium can be any combination of one or more computer-readable media. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In the present invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0119] It should be noted that, unless otherwise defined, the technical or scientific terms used in this invention should have the ordinary meaning understood by one of ordinary skill in the art to which this invention pertains. The terms "first," "second," and similar terms used in this invention do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0120] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention in any other way. Any person skilled in the art may make changes or modifications to the above-disclosed technical content to create equivalent embodiments. However, any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the protection scope of the present invention.

[0121] This invention is not limited to the preferred embodiment described above. Anyone inspired by this invention can derive other forms of a single-scan Mamba feature extraction method and system for global image modeling. All equivalent variations and modifications made within the scope of the claims of this invention shall fall within the scope of this invention.

Claims

1. A single-scan Mamba feature extraction method for global image modeling, characterized in that, include: The input image is acquired, and a multi-channel feature map with channel and spatial dimensions is extracted through convolution operations. A state space modeling process is performed on the multi-channel feature map using a single forward computation to model the global dependencies between channels. The state space modeling is achieved by performing matrix operations on the features after flattening the spatial dimensions and the channel interaction matrix constructed based on the state space model parameters. Output the feature map after global dependency modeling, and apply it to downstream vision tasks; The specific process of state-space modeling includes: After performing layer normalization on the multi-channel feature map, the input projection matrix, output projection matrix, and discretized time step parameters are generated by sequentially performing 1×1 convolution and group convolution operations. The state space model also includes a learnable autoregressive weight matrix. After adding the discretized time step parameter to the autoregressive weight matrix, the discretized state matrix is ​​obtained by processing it through a normalization function. Then, the discretized state matrix is ​​multiplied element-wise with the input projection matrix to obtain the channel interaction matrix. First, the features flattened in the spatial dimension are multiplied by the transpose of the channel interaction matrix to obtain intermediate features; then, the intermediate features are multiplied by the output projection matrix to generate a flattened global feature sequence. After reshaping the global feature sequence back to its original spatial dimension, feature transformation is performed through convolution, and then residual connection is performed with the multi-channel feature map. Combined with path dropout regularization, the feature map after global dependency modeling is obtained.

2. The single-scan Mamba feature extraction method for global image modeling according to claim 1, characterized in that: The extraction process of the multi-channel feature map is as follows: the input image is sequentially mapped and optimized through at least two cascaded convolutional layers, and each convolutional layer is spatially downsampled to finally obtain a multi-channel feature map with a preset number of channels and spatial size.

3. The single-scan Mamba feature extraction method for global image modeling according to claim 1, characterized in that: The state space modeling is achieved through a network architecture consisting of multiple core stages. Each core stage includes a convolution module and a single forward computation of the channel dimension for state space modeling. The convolution module is a residual block structure that sequentially performs 1×1 convolution for channel mapping, group convolution, and 1×1 convolution for channel restoration on the input features. The output is fused with the original input features through residual connections, and a path dropping mechanism is set at the output end of the main convolution path.

4. The single-scan Mamba feature extraction method for global image modeling according to claim 3, characterized in that: By adjusting the channel dimension, number of network blocks, and state space parameter configuration of each core stage, model variants adapted to different computing power platforms are formed. The channel dimension, number of network blocks, and state space parameters of each model variant adopt different configuration combinations.

5. The single-scan Mamba feature extraction method for global image modeling according to claim 1, characterized in that: The downstream visual tasks include image classification and / or image retrieval.

6. The single-scan Mamba feature extraction method for global image modeling according to claim 1, characterized in that: The difference between the predicted result and the true label is calculated using the cross-entropy loss function. The learnable parameters of the state space model are optimized end-to-end using the AdamW optimizer, and a dynamic learning rate adjustment strategy is adopted during training.

7. A single-scan Mamba feature extraction system for global image modeling, used to implement the method as described in claim 1, characterized in that, include: The feature extraction module is used to acquire the input image and extract multi-channel feature maps with channel and spatial dimensions through convolution operations; The global modeling module is used to perform a single forward computation on the multi-channel feature map to model the state space, so as to model the global dependencies between channels. The state space modeling is achieved by performing matrix operations on the features after flattening the spatial dimension and the channel interaction matrix constructed based on the state space model parameters. The output application module is used to output feature maps after global dependency modeling and adapt them to downstream vision tasks.

8. A computer device, characterized in that, It includes a processor and a memory, the memory storing a computer program, and the processor, when executing the computer program, implements the method of any one of claims 1-6.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-6.

Images