Image feature extraction method and related product

Abstract

The invention discloses an image feature extraction method and related products, and the method comprises the steps: carrying out the feature modeling of an initial feature map obtained through the embedding processing of a to-be-processed image, obtaining a modeling feature map, and obtaining an output feature map of the to-be-processed image according to the modeling feature map, wherein the feature modeling processing comprises the steps of splitting an initial feature map into N spatial sub-feature maps, and splitting each spatial sub-feature map into M channel sub-feature maps; carrying out at least one time of equal mapping processing, local feature modeling processing and global feature modeling processing on the M channel sub-feature maps corresponding to each space sub-feature map to obtain sub-modeling feature maps of the M channel sub-feature maps, splicing the sub-modeling feature maps of the M channel sub-feature maps to obtain a fusion sub-modeling feature map of the spatial sub-feature map; and splicing the fusion sub-modeling feature maps of the N space sub-feature maps to obtain a modeling feature map. According to the invention, the extraction performance and calculation complexity of image feature extraction can be effectively balanced.

Description

Technical Field

[0001] This application relates to the field of computer technology, specifically to an image feature extraction method, an image feature extraction device, an electronic device, a computer-readable storage medium, and a computer program product. Background Technology

[0002] In practical applications, it is often necessary to deploy visual task models on devices with limited processing resources, such as mobile devices and IoT devices, to endow these devices with visual task processing capabilities. For example, by deploying an image classification model on a smartphone, the smartphone can be given the ability to recognize objects in a photograph.

[0003] Visual task models typically consist of two main parts: a base model (also known as the backbone network, feature extraction network, etc.) and a task head. When performing a visual task, the base model first extracts features from the input image to obtain an output feature map. The task head then uses this output feature map to perform specific visual task processing, such as image classification, semantic segmentation, and object detection. Overall, the visual task processing capability of a visual task model primarily depends on the feature extraction capability of its base model, and its computational complexity is also mainly limited by the computational complexity of the base model.

[0004] Therefore, given the above practical deployment requirements, it becomes crucial to provide a visual foundation model with high feature extraction performance and low computational complexity. Summary of the Invention

[0005] This application provides an image feature extraction method, an image feature extraction device, an electronic device, a computer-readable storage medium, and a computer product.

[0006] Firstly, the image feature extraction method provided in this application includes:

[0007] Embedding is performed on the image to be processed to obtain an initial feature map of the image to be processed. The resolution of the initial feature map is smaller than the resolution of the image to be processed.

[0008] The initial feature map is processed by feature modeling to obtain a modeled feature map, and the output feature map of the image to be processed is obtained based on the modeled feature map;

[0009] Feature modeling processing includes:

[0010] In the spatial dimension, the initial feature map is split into N spatial sub-feature maps, and in the channel dimension, each spatial sub-feature map is split into M channel sub-feature maps, where N is a positive integer greater than or equal to 2 and M is a positive integer greater than or equal to 3.

[0011] For each spatial sub-feature map corresponding to M channel sub-feature maps, the first channel sub-feature map is processed by one of identity mapping, local feature modeling, or global feature modeling to obtain the sub-modeling feature map of the first channel sub-feature map. The i-th channel sub-feature map is added to the sub-modeling feature map of the (i-1)-th channel sub-feature map to obtain the fused feature map. The fused feature map is then processed by one of identity mapping, local feature modeling, or global feature modeling to obtain the sub-modeling feature map of the i-th channel sub-feature map, i∈[2,M]. This includes at least one identity mapping, local feature modeling, and global feature modeling process. The sub-modeling feature maps of the M channel sub-feature maps are concatenated along the channel dimension to obtain the fused sub-modeling feature map of the spatial sub-feature map.

[0012] The modeling feature map is obtained by merging N spatial sub-feature maps in the spatial dimension into a fusion sub-modeling feature map.

[0013] Secondly, the image feature extraction apparatus provided in this application includes:

[0014] The image embedding module is used to embed the image to be processed to obtain an initial feature map of the image to be processed. The resolution of the initial feature map is smaller than the resolution of the image to be processed.

[0015] The feature extraction module is used to perform feature modeling processing on the initial feature map to obtain a modeled feature map, and to obtain the output feature map of the image to be processed based on the modeled feature map;

[0016] Feature modeling processing includes:

[0017] In the spatial dimension, the initial feature map is split into N spatial sub-feature maps, and in the channel dimension, each spatial sub-feature map is split into M channel sub-feature maps, where N is a positive integer greater than or equal to 2 and M is a positive integer greater than or equal to 3.

[0018] For each spatial sub-feature map corresponding to M channel sub-feature maps, the first channel sub-feature map is processed by one of identity mapping, local feature modeling, or global feature modeling to obtain the sub-modeling feature map of the first channel sub-feature map. The i-th channel sub-feature map is added to the sub-modeling feature map of the (i-1)-th channel sub-feature map to obtain the fused feature map. The fused feature map is then processed by one of identity mapping, local feature modeling, or global feature modeling to obtain the sub-modeling feature map of the i-th channel sub-feature map, i∈[2,M]. This includes at least one identity mapping, local feature modeling, and global feature modeling process. The sub-modeling feature maps of the M channel sub-feature maps are concatenated along the channel dimension to obtain the fused sub-modeling feature map of the spatial sub-feature map.

[0019] The modeling feature map is obtained by merging N spatial sub-feature maps in the spatial dimension into a fusion sub-modeling feature map.

[0020] Optionally, in one embodiment, the feature extraction module is used to perform identity mapping on the first channel sub-feature map to obtain a mapped feature map, and use the mapped feature map as the sub-modeling feature map of the first channel sub-feature map; add the i-th channel sub-feature map and the sub-modeling feature map of the (i-1)-th channel sub-feature map to obtain a first fused feature map, and perform local feature modeling on the first fused feature map to obtain the sub-modeling feature map of the i-th channel sub-feature map, i∈[2,M-1]; add the M-th channel sub-feature map and the sub-modeling feature map of the (M-1)-th channel sub-feature map to obtain a second fused feature map, and perform global feature modeling on the second fused feature map to obtain the sub-modeling feature map of the M-th channel sub-feature map.

[0021] Optionally, in one embodiment, the feature extraction module is used to sequentially perform depthwise convolution, batch normalization, linear rectified activation, pointwise convolution, and batch normalization on the first fused feature map to obtain the sub-modeling feature map of the i-th channel sub-feature map.

[0022] Optionally, in one embodiment, the feature extraction module is used to perform global modeling processing on the second fused feature map through the Mamba module to obtain the sub-modeling feature map of the Mth channel sub-feature map.

[0023] Optionally, in one embodiment, when M is greater than or equal to 4, the kernel size of the depthwise convolution process in the later local feature modeling process is greater than the kernel size of the depthwise convolution process in the earlier local feature modeling process.

[0024] Optionally, in one embodiment, the feature extraction module is used to perform at least one optimization process on the feature modeling map to obtain an output feature map; the optimization process includes downsampling processing and feature modeling processing performed sequentially.

[0025] Optionally, in one embodiment, the downsampling process includes sequentially performing depthwise convolution processing, batch normalization processing, feedforward processing, pointwise convolution processing, batch normalization processing, linear rectification activation processing, depthwise convolution processing, batch normalization processing, channel attention enhancement processing, pointwise convolution processing, batch normalization processing, depthwise convolution processing, batch normalization processing, and feedforward processing, wherein the stride of the second depthwise convolution processing is S1, and the stride of the other convolution processing is 1, where S1 is a positive integer greater than or equal to 2.

[0026] Optionally, in one embodiment, the feedforward processing includes sequentially performing pointwise convolution processing, batch normalization processing, linear rectification activation processing, pointwise convolution processing, and batch normalization processing.

[0027] Optionally, in one embodiment, the image embedding module is used to perform L mapping processes, one standard convolution process, and one batch normalization process on the image to be processed in sequence to obtain the initial feature map of the image to be processed; the mapping process includes the sequential standard convolution process, batch normalization process, and linear rectified activation process; wherein, the stride of all standard convolution processes in the embedding process is S2, where S2 is a positive integer greater than or equal to 2, and L is a positive integer greater than or equal to 3.

[0028] Optionally, in one embodiment, the feature extraction module is used to sequentially perform depthwise convolution, batch normalization, feedforward processing, and residual connection processing on the initial feature map to obtain the processed feature map; and to split the processed feature map into N spatial sub-feature maps in the spatial dimension.

[0029] Optionally, in one embodiment, the feature extraction module is used to stitch together N spatial sub-feature maps in the spatial dimension to obtain a fused sub-modeling feature map; add the stitched feature map to the processed feature map to obtain an intermediate feature map; and perform depthwise convolution, batch normalization, and feedforward processing on the intermediate feature map in sequence to obtain a modeling feature map.

[0030] Thirdly, the electronic device provided in this application includes a memory and a processor. The memory stores a computer program, and the processor runs the computer program in the memory to implement the steps in the image feature extraction method provided in this application.

[0031] Fourthly, the computer-readable storage medium provided in this application stores a computer program adapted for processor execution to implement the steps in the image feature extraction method provided in this application.

[0032] Fifthly, the computer program product provided in this application includes a computer program adapted for processor execution to implement the steps in the image feature extraction method provided in this application.

[0033] The image feature extraction scheme provided in this application first obtains an initial feature map with a resolution smaller than that of the image to be processed through embedding processing. Then, in the spatial dimension, the initial feature map is split into N spatial sub-feature maps, and in the channel dimension, each spatial sub-feature map is further split into M channel sub-feature maps. Each channel sub-feature map is used as the object of feature modeling. In this way, the amount of data for a single feature modeling is reduced from both the spatial and channel dimensions, thereby reducing the computational complexity. On the other hand, in the feature modeling process, at least one identity mapping process, local feature modeling process, and global feature modeling process are performed on the M channel sub-feature maps of the same spatial sub-feature map to obtain a sub-modeling feature map. The M sub-modeling feature maps are then concatenated to obtain a fused sub-modeling feature map of the spatial sub-feature map. Finally, the fused sub-modeling feature map of the N spatial sub-feature maps is concatenated to obtain a modeling feature map. Thus, based on the modeling feature map, an output feature map that can represent both local spatial characteristics and global spatial characteristics is obtained. Therefore, by setting the architecture of the visual basic model according to the image feature extraction scheme provided in this application, the visual basic model can have both high feature extraction performance and low computational complexity, thereby enabling the visual task model based on the visual basic model to be effectively deployed on devices with limited processing resources. Attached Figure Description

[0034] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0035] Figure 1a This is a schematic diagram of a scene of the image feature extraction system provided in an embodiment of this application;

[0036] Figure 1b This is a flowchart illustrating an image feature extraction method provided in an embodiment of this application;

[0037] Figure 1c This is an example architecture diagram of the visual basic model provided in the embodiments of this application;

[0038] Figure 1d This is an example diagram of the architecture of the embedded module in the embodiments of this application;

[0039] Figure 1e This is a schematic diagram of the feature modeling process in the embodiments of this application;

[0040] Figure 1f This is an example architecture diagram of the feature modeling module in an embodiment of this application;

[0041] Figure 1g This is an example diagram of the architecture of the local feature modeling branch in the embodiments of this application;

[0042] Figure 1h This is an example diagram of the architecture of the Mamba module in the embodiments of this application;

[0043] Figure 1i This is another architectural example diagram of the visual basic model provided in the embodiments of this application;

[0044] Figure 1j This is an example diagram of the architecture of the downsampling module in an embodiment of this application;

[0045] Figure 1k This is an example diagram of the architecture of the feedforward layer in the embodiments of this application;

[0046] Figure 11 This is another architectural example diagram of the feature modeling module in the embodiments of this application;

[0047] Figure 1m This is another architectural example diagram of the feature modeling module in the embodiments of this application;

[0048] Figure 2a This is another schematic diagram of the architecture of the visual basic model provided in the embodiments of this application;

[0049] Figure 2b This is another schematic flowchart of the image feature extraction method provided in the embodiments of this application;

[0050] Figure 3 This is a schematic diagram of the image feature extraction device provided in an embodiment of this application;

[0051] Figure 4 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0052] It should be noted that the principles of this application are illustrated by example in a suitable computing environment. The following description is based on the specific embodiments of this application that are illustrated, and should not be regarded as limiting other specific embodiments not detailed herein.

[0053] In the following description of this application, "some embodiments" are referred to, which describe a subset of all possible embodiments. However, it is understood that "some embodiments" may be the same subset or different subset of all possible embodiments, and may be combined with each other without conflict.

[0054] In the following description of this application, the terms "first, second, third" are used only to distinguish similar objects and do not represent a specific order of objects. It is understood that "first, second, third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0055] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0056] To reduce the computational complexity of feature extraction while ensuring its quality, this application provides an image feature extraction method, an image feature extraction apparatus, an electronic device, a computer-readable storage medium, and a computer program product. The image feature extraction method can be executed by the image feature extraction apparatus or by an electronic device integrating the image feature extraction apparatus.

[0057] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described below are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0058] Please refer to the reference. Figure 1aThis application also provides an image feature extraction system, which includes an electronic device 100 for executing the image feature extraction method provided in this application. The electronic device 100 can be any device equipped with a processor and having processing capabilities, such as mobile devices with processors, such as smartphones, tablets, PDAs, laptops, virtual reality devices, augmented reality devices, or mixed reality devices, or fixed devices with processors, such as desktop computers, televisions, servers, and industrial equipment. The process begins by acquiring the image to be processed for feature extraction and embedding it to obtain an initial feature map. The resolution of the initial feature map is lower than that of the image to be processed. Feature modeling is then performed on the initial feature map to obtain a modeled feature map, and the output feature map of the image to be processed is obtained based on the modeled feature map. The feature modeling process includes: splitting the initial feature map into N spatial sub-feature maps in the spatial dimension, and splitting each spatial sub-feature map into M channel sub-feature maps in the channel dimension, where N is a positive integer greater than or equal to 2, and M is a positive integer greater than or equal to 3. For each spatial sub-feature map corresponding to the M channel sub-feature maps, the first channel sub-feature map undergoes identity mapping and local feature construction. The process involves either modeling or global feature modeling to obtain a sub-modeling feature map of the first channel sub-feature map. The sub-modeling feature map of the i-th channel sub-feature map is then added to the sub-modeling feature map of the (i-1)-th channel sub-feature map to obtain a fused feature map. The fused feature map is then subjected to either identity mapping, local feature modeling, or global feature modeling to obtain a sub-modeling feature map of the i-th channel sub-feature map, where i ∈ [2, M]. This includes at least one instance of identity mapping, local feature modeling, or global feature modeling. The sub-modeling feature maps of M channel sub-feature maps are concatenated along the channel dimension to obtain a fused sub-modeling feature map of the spatial sub-feature map. The fused sub-modeling feature map of N spatial sub-feature maps is concatenated along the spatial dimension to obtain a modeling feature map.

[0059] In addition, such as Figure 1a As shown, the image feature extraction system may also include a memory 200 for storing relevant data during the image feature extraction process, such as the original image to be processed, the initial feature map obtained by embedding processing, the spatial sub-feature map, channel sub-feature map, fusion feature map, sub-modeling feature map, fusion sub-modeling feature map, modeling feature map obtained by feature modeling processing, and the final output feature map.

[0060] It should be noted that the image feature extraction system described above is merely an example, intended to more clearly illustrate the technical solutions of the embodiments of this application, and does not constitute a limitation on the technical solutions provided in the embodiments of this application. As those skilled in the art will know, with the evolution of image feature extraction systems and the emergence of new business scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0061] The following sections provide detailed descriptions of each example. It should be noted that the sequence numbers of the following embodiments are not intended to limit the preferred order of the embodiments.

[0062] Please refer to Figure 1b , Figure 1b This is a flowchart illustrating the image feature extraction method provided in this embodiment, as shown below. Figure 1b As shown, the process of this image feature extraction method can be as follows:

[0063] In step 110, the image to be processed is embedded to obtain an initial feature map of the image to be processed. The resolution of the initial feature map is smaller than the resolution of the image to be processed.

[0064] The image to be processed is the image that needs to be processed by a visual task. The visual task can be any type of visual task, including but not limited to image classification tasks, semantic segmentation tasks, object detection tasks, and so on.

[0065] In this embodiment of the application, in response to a task processing request, the image to be processed for visual task processing can be obtained according to the task processing request. For example, the electronic device that performs the image feature extraction method can be a smartphone. After the smartphone takes a picture and generates a new image, it uses the image as the image to be classified and generates an image classification request for the image, instructing that feature extraction be performed on the image, thereby using the extracted output feature map to complete the image classification.

[0066] It should be noted that the image feature extraction scheme provided in this application can be embedded into a visual task model to perform feature extraction operations, thereby reducing the computational complexity of feature extraction while ensuring the quality of feature extraction. This ensures that the visual task model can effectively complete visual task processing even when deployed on resource-constrained devices. Accordingly, this application provides a novel visual foundation model, which is configured to take an image as input, perform feature extraction on the input image to obtain an output feature map, and use the output feature map for downstream visual tasks.

[0067] Please refer to Figure 1cThe visual basic model provided in this application embodiment may include an embedding module and a feature modeling module. The embedding module is configured to perform embedding processing on the input image to obtain an initial feature map of the input image, and the resolution of the initial feature map of the input image is smaller than the resolution of the input image. The feature modeling module is configured to perform feature modeling processing on the initial feature map of the input image to obtain a modeled feature map of the input image, and to obtain an output feature map based on the modeled feature map.

[0068] Accordingly, in this embodiment, the obtained image to be processed can be input into the embedding module of the visual basic model, and then the image to be processed can be embedded through the embedding module to obtain an initial feature map with a resolution lower than that of the image to be processed.

[0069] Optionally, in one embodiment, embedding processing is performed on the image to be processed to obtain an initial feature map of the image to be processed, including:

[0070] The image to be processed is subjected to L mapping processes, one standard convolution process, and one batch normalization process in sequence to obtain the initial feature map of the image to be processed.

[0071] The mapping process includes standard convolution processing, batch normalization processing, and linear rectified activation processing performed sequentially.

[0072] In the embedding process, the stride of all standard convolution processes is S2, where S2 is a positive integer greater than or equal to 2, and L is a positive integer greater than or equal to 3.

[0073] This application provides an optional embedding processing scheme, and correspondingly provides an optional architecture for the embedding module. Please refer to... Figure 1d The embedding module includes L layers of standard convolutional layers + batch normalization + linear rectified activation layers, and 1 layer of standard convolutional layers + batch normalization layers. The stride of the standard convolutions is S2, and the kernel size is k. As an example, in this embodiment, L is 3, S2 is 2, and k is 3, meaning the embedding module includes 3 layers of standard convolutional layers + batch normalization + linear rectified activation layers and 1 layer of standard convolutional layers + batch normalization layers. In other embodiments, S2 is a positive integer greater than or equal to 2, and L is a positive integer greater than or equal to 3 as constraints. L, S2, and k can be chosen by those skilled in the art according to actual needs.

[0074] Accordingly, when embedding the image to be processed, the image is input into the first layer of the embedding module: standard convolution + batch normalization + linear rectified activation layer. This first layer sequentially performs standard convolution, batch normalization, and linear rectified activation on the input image, thus completing the first mapping process. In this way, through L layers of standard convolution + batch normalization + linear rectified activation layers, L mapping processes can be completed on the image to be processed, yielding the mapping result.

[0075] The mapping result is input into the standard convolution + batch normalization layer of the embedding module. The standard convolution + batch normalization layer sequentially performs standard convolution and batch normalization processing on the mapping result to obtain the output result of the embedding module, which is the initial feature map of the image to be processed.

[0076] It is understood that in this embodiment, the stride of all standard convolutions in the embedding module is set to 2. Therefore, after embedding processing, the resolution of the initial feature map of the image to be processed is 1 / 16 of the resolution of the image to be processed; that is, the height of the initial feature map is 1 / 16 of the height of the image to be processed, and the width of the initial feature map is 1 / 16 of the width of the image to be processed. Furthermore, since multiple standard convolutions are performed on the image to be processed during the embedding process, the number of channels in the initial feature map will be greater than the number of channels in the image to be processed. For example, the image to be processed can be represented as X∈R. HxWx3 The initial feature map can be represented as Where H represents the height of the image to be processed, W represents the width of the image to be processed, 3 represents the number of channels of the image to be processed, and C1 represents the number of channels of the initial feature map. C1>3, and its value depends on the specific configuration of the standard convolution in the embedding module.

[0077] In step 120, the initial feature map is processed by feature modeling to obtain a modeled feature map, and the output feature map of the image to be processed is obtained based on the modeled feature map.

[0078] After embedding the image to be processed and obtaining its initial feature map, the initial feature map is further processed by feature modeling to obtain a modeled feature map. The output feature map is then derived from this modeled feature map. For example, depending on the actual feature extraction requirements, the modeled feature map can be directly used as the output feature map, or it can be post-processed with specific configurations to obtain the final output feature map. Once the output feature map is obtained, it can be used to perform the visual task processing on the image.

[0079] Please refer to Figure 1e Feature modeling processing includes:

[0080] 1210, in the spatial dimension, the initial feature map is split into N spatial sub-feature maps, and in the channel dimension, each spatial sub-feature map is split into M channel sub-feature maps, where N is a positive integer greater than or equal to 2 and M is a positive integer greater than or equal to 3;

[0081] 1220. For each spatial sub-feature map corresponding to M channel sub-feature maps, perform one of the following processing on the first channel sub-feature map: identity mapping, local feature modeling, or global feature modeling, to obtain the sub-modeling feature map of the first channel sub-feature map; add the sub-modeling feature map of the i-th channel sub-feature map to the sub-modeling feature map of the (i-1)-th channel sub-feature map to obtain the fused feature map, and perform one of the following processing on the fused feature map: identity mapping, local feature modeling, or global feature modeling, to obtain the sub-modeling feature map of the i-th channel sub-feature map, i∈[2,M], including at least one identity mapping, local feature modeling, and global feature modeling; concatenate the sub-modeling feature maps of the M channel sub-feature maps in the channel dimension to obtain the fused sub-modeling feature map of the spatial sub-feature map;

[0082] 1230. The modeling feature map is obtained by merging N spatial sub-feature maps in the spatial dimension.

[0083] This application provides an optional feature modeling processing scheme and a corresponding optional architecture for a feature modeling module.

[0084] Please refer to Figure 1fThe feature modeling module may include a spatial splitting layer, a channel splitting layer, M modeling branches, a channel concatenation layer, and a spatial concatenation layer. The spatial splitting layer is configured to split the input feature map into N spatial sub-feature maps of the same resolution in the spatial dimension. The channel splitting layer is configured to further split the input spatial sub-feature map into M channel sub-feature maps of the same length in the channel dimension. The first modeling branch is configured to perform one of the following processing on the input channel sub-feature maps: identity mapping, local feature modeling, or global feature modeling, to obtain a sub-modeling feature map of the first channel sub-feature map. The ith modeling branch is configured to process the ith channel sub-feature map and the (i-1)th channel sub-feature map... The fused feature map obtained by adding the sub-modeling feature maps of the feature map is subjected to one of the following processing: identity mapping, local feature modeling, or global feature modeling, to obtain the sub-modeling feature map of the i-th channel sub-feature map, i∈[2,M]. The M modeling branches are subjected to at least one identity mapping, at least one local feature modeling, and at least one global feature modeling. The channel splicing layer is configured to splice the M sub-modeling feature maps corresponding to the same spatial sub-feature map in the channel dimension and then perform pointwise convolution processing to obtain the fused sub-modeling feature map. The spatial splicing layer is configured to splice the fused sub-modeling feature maps corresponding to the N spatial sub-feature maps in the spatial dimension to form the modeling feature map of the input feature map.

[0085] Accordingly, when performing feature modeling processing on the initial feature map, the initial feature map is input into the spatial splitting layer of the feature modeling module. This spatial splitting layer splits the initial feature map into N spatial sub-feature maps in the spatial dimension. For each spatial sub-feature map, it is input into the channel splitting layer of the feature modeling module. This channel splitting layer splits the spatial sub-feature map into M channel sub-feature maps in the channel dimension. The first channel sub-feature map is input into the first modeling branch of the feature modeling module. The first modeling branch performs one of the following processing on the first channel sub-feature map: identity mapping, local feature modeling, or global modeling, to obtain the sub-modeling feature map of the first channel sub-feature map. The sub-modeling feature map of the i-th channel sub-feature map is added to the sub-modeling feature map of the (i-1)-th channel sub-feature map to obtain the fused feature map, which is then input into the feature modeling module. The i-th modeling branch performs one of the following processing on the fused feature map: identity mapping, local feature modeling, or global modeling, to obtain the sub-modeling feature map of the i-th channel sub-feature map, i∈[2,M]. Among them, the M modeling branches perform at least one identity mapping, at least one local feature modeling, and at least one global feature modeling. The sub-modeling feature maps of the M channel sub-feature maps are input into the channel stitching layer of the feature modeling module. The sub-modeling feature maps of the M channel sub-feature maps are stitched together by the channel stitching layer and then subjected to pointwise convolution to obtain the fused sub-modeling feature map of the spatial sub-feature map. The fused sub-modeling feature maps of the N spatial sub-feature maps are input into the spatial stitching layer of the feature modeling module. The fused sub-modeling feature maps of the N spatial sub-feature maps are stitched together by the spatial stitching layer to obtain the modeling feature map of the initial feature map.

[0086] Optionally, in one embodiment, the first channel sub-feature map is processed by one of identity mapping, local feature modeling, or global feature modeling to obtain a sub-modeling feature map of the first channel sub-feature map. The i-th channel sub-feature map is added to the sub-modeling feature map of the (i-1)-th channel sub-feature map to obtain a fused feature map. The fused feature map is then processed by one of identity mapping, local feature modeling, or global feature modeling to obtain a sub-modeling feature map of the i-th channel sub-feature map, i∈[2,M], including:

[0087] The first channel sub-feature map is subjected to identity mapping to obtain a mapped feature map, which is then used as the sub-modeling feature map of the first channel sub-feature map.

[0088] The first fused feature map is obtained by adding the sub-modeling feature map of the i-th channel sub-feature map to the sub-modeling feature map of the (i-1)-th channel sub-feature map, and the first fused feature map is processed by local feature modeling to obtain the sub-modeling feature map of the i-th channel sub-feature map, i∈[2,M-1].

[0089] The second fused feature map is obtained by adding the sub-modeling feature map of the Mth channel sub-feature map to the sub-modeling feature map of the (M-1)th channel sub-feature map, and then performing global feature modeling processing on the second fused feature map to obtain the sub-modeling feature map of the Mth channel sub-feature map.

[0090] This application embodiment further provides an optional feature modeling processing scheme, and correspondingly provides an optional architecture for a feature modeling module.

[0091] In this embodiment, the first modeling branch of the feature modeling module is an identity mapping branch, the ith modeling branch is a local feature modeling branch (i∈[2,M-1]), and the Mth modeling branch is a global feature modeling branch. The identity mapping branch is configured to perform identity mapping processing on the input ith channel sub-feature map to obtain a mapped feature map, which serves as the sub-modeling feature map of the ith channel sub-feature map. The ith local feature modeling branch is configured to perform local feature modeling processing on the fused feature map obtained by adding the ith channel sub-feature map and the sub-modeling feature map of the (i-1)th channel sub-feature map to obtain the sub-modeling feature map of the ith channel sub-feature map. The global feature modeling branch is configured to perform global feature modeling processing on the fused feature map obtained by adding the Mth channel sub-feature map and the sub-modeling feature map of the (M-1)th channel sub-feature map to obtain the sub-modeling feature map of the Mth channel sub-feature map.

[0092] Correspondingly, for each spatial sub-feature map corresponding to M channel sub-feature maps, the first channel sub-feature map is input into the identity mapping branch of the feature modeling module. The identity mapping branch performs identity mapping on the first channel sub-feature map to obtain a mapped feature map, which is then used as the sub-modeling feature map of the first channel sub-feature map. The i-th channel sub-feature map is added to the sub-modeling feature map of the (i-1)-th channel sub-feature map to obtain the first fused feature map. The first fused feature map is then input into the (i-1)-th local feature modeling module. The model branch performs local feature modeling processing on the first fused feature map through the (i-1)th local feature modeling branch to obtain the sub-modeling feature map of the i-th channel sub-feature map, i∈[2,M]. The M-th channel sub-feature map is added to the sub-modeling feature map of the (M-1)th channel sub-feature map to obtain the second fused feature map. The second fused feature map is then input into the global feature modeling branch of the feature modeling module. The global feature modeling branch performs global feature modeling processing on the second fused feature map to obtain the sub-modeling feature map of the M-th channel sub-feature map.

[0093] Optionally, in one embodiment, local feature modeling processing is performed on the first fused feature map to obtain a sub-modeling feature map of the i-th channel sub-feature map, including:

[0094] The first fused feature map is sequentially processed by depthwise convolution, batch normalization, linear rectified activation, pointwise convolution, and batch normalization to obtain the sub-modeling feature map of the i-th channel sub-feature map.

[0095] This application provides an optional local feature modeling processing scheme, and correspondingly provides an optional architecture for a local feature modeling branch.

[0096] Please refer to Figure 1g The local feature modeling branch can include depthwise convolutional layers, a first batch normalization layer, a linear rectified activation layer, a pointwise convolutional layer, and a second batch normalization layer. The depthwise convolutional layer is configured to perform depthwise convolution on the input feature map to obtain the depthwise convolution result. The first batch normalization layer is configured to perform batch normalization on the input depthwise convolution result to obtain the first batch normalized result. The linear rectified activation layer is configured to perform linear rectified activation on the input first batch normalized result to obtain the linear rectified activation result. The pointwise convolutional layer is configured to perform pointwise convolution on the input linear rectified activation result to obtain the pointwise convolution result. The second batch normalization layer is configured to perform batch normalization on the input pointwise convolution result to obtain the sub-modeling feature map of the input feature map. All convolutions in the local feature modeling branch have a 3x3 kernel size. In other embodiments, those skilled in the art may also set the architecture of local feature modeling branches according to actual needs, and this application does not impose specific limitations.

[0097] Accordingly, when performing local feature modeling processing on the first fused feature map, the first fused feature map is input into the depthwise convolutional layer of the corresponding local feature modeling branch. This depthwise convolutional layer performs depthwise convolution processing on the input first fused feature map to obtain the depthwise convolution result. This depthwise convolution result is then input into the first batch normalization layer of the local feature modeling branch. This batch normalization layer performs batch normalization processing on the input depthwise convolution result to obtain the first batch normalization result. Finally, this first batch normalization result is input into the linear rectified activation layer of the local feature modeling branch. The activation layer performs linear rectified activation processing on the first batch normalized result of the input to obtain a linear rectified activation result. This linear rectified activation result is then input into the pointwise convolutional layer of the local feature modeling branch. The pointwise convolutional layer performs pointwise convolution processing on the input linear rectified activation result to obtain a pointwise convolutional result. This pointwise convolutional result is then input into the second batch normalization layer of the local feature modeling branch. The batch normalization layer performs batch normalization processing on the input pointwise convolutional result to obtain the second batch normalized result, which is the sub-modeling feature map obtained by performing local feature modeling processing on the first fused feature map.

[0098] Optionally, in one embodiment, a sub-modeling feature map of the Mth channel sub-feature map is obtained by performing global modeling processing on the second fused feature map, including:

[0099] The Mamba module is used to perform global modeling processing on the second fused feature map to obtain the sub-modeling feature map of the Mth channel sub-feature map.

[0100] This application provides an optional global feature modeling processing scheme, and correspondingly provides an optional architecture for a global feature modeling branch. In this embodiment, the Mamba module is used as the global feature modeling branch. The specific architecture of the Mamba module is not limited here and can be selected by those skilled in the art according to actual needs, including but not limited to architectures such as Mamba-1 and Mamba-2.

[0101] As an example, please refer to Figure 1h The Mamba module consists of linear layers and channel splitting layers from top to bottom. Two branches emerge from the channel splitting layers: one branch includes activation layers, and the other includes depthwise convolutional layers, activation layers, and a state-space model layer. These two branches are connected to the second linear layer after element-wise multiplication. The two linear layers can be implemented using convolutions; the type of convolution used is not restricted, for example, a standard convolution can be used. Similarly, the two activation layers can be implemented using activation functions; the type of activation function used is not restricted, for example, the Silu activation function can be used.

[0102] When performing global modeling processing on the second fused feature map using the Mamba module to obtain the sub-modeling feature map of the Mth channel sub-feature map, the second fused feature map is input into the first linear layer of the Mamba module for linear mapping to obtain the first linear mapping result. The linear mapping result is then input into the channel splitting layer of the Mamba module, which splits the linear mapping result into two linear mapping results in the channel dimension. One sub-linear mapping result is input into one branch, and after activation processing by the activation layer in that branch, the activation result is obtained. The other sub-linear mapping result is input into another branch, which first passes through a depthwise convolutional layer and then through an activation layer to obtain the four inputs A, X, B, and C (not shown in the figure) required by the state space model layer. The state space model layer processes the four inputs A, X, B, and C accordingly to obtain the processing result. Finally, the processing result of the state space model layer is multiplied element-wise with the activation result of the previous branch and then input into the second linear layer for linear mapping. The obtained linear mapping result is the sub-modeling feature map of the Mth channel sub-feature map.

[0103] It should be noted that the Mamba module has the global modeling capability of the Transformer module, and has only linear computational complexity. It can capture global information and long-distance dependencies in the image, and significantly reduce computational complexity while maintaining efficient feature extraction.

[0104] Optionally, in one embodiment, the (M-1)th local feature modeling branch in the feature modeling module can be replaced with a global feature modeling branch, and the two global feature modeling branches in the replaced feature modeling module have different global feature modeling methods. As an example, a transformer module can be used as the first global feature modeling branch, and a Mamba module can be used as the second global feature modeling branch.

[0105] Optionally, in one embodiment, when M is greater than or equal to 4, in order to further enhance the quality of feature extraction, the kernel size of the depthwise convolution process in the subsequent local feature modeling process is configured to be greater than the kernel size of the depthwise convolution process in the preceding local feature modeling process.

[0106] Optionally, in one embodiment, obtaining the output feature map of the image to be processed based on the modeled feature map includes:

[0107] The feature modeling map is optimized at least once to obtain the output feature map.

[0108] The optimization process includes sequential downsampling and feature modeling.

[0109] In this embodiment, the modeling feature map of the initial feature map is not directly used as the output feature map of the image to be processed. Instead, it is further post-processed to obtain the output feature map of the image to be processed. Accordingly, an optional post-processing scheme and the architecture of the corresponding visual basic model are provided.

[0110] Please refer to Figure 1i The visual basic model provided in this application embodiment may include an embedding module, a feature modeling module, and J optimization processing modules. The optimization processing modules include a downsampling module and a feature modeling module, where J is a positive integer greater than or equal to 1.

[0111] When obtaining the output feature map of the image to be processed based on the modeling feature map, the modeling feature map of the initial feature map can be optimized J times by J optimization processing modules of the visual basic model, and the optimization result of the last optimization processing is used as the output feature map of the image to be processed.

[0112] Accordingly, during the optimization process, the feature map to be optimized is input into the optimization module. First, the downsampling module in the optimization module downsamples the input feature map to obtain a downsampled feature map. Then, the feature modeling module in the optimization module performs feature modeling on the downsampled feature map to obtain a modeled feature map, which is the optimization result of this optimization module on the input feature map. The details of how feature modeling is performed are not elaborated here; please refer to the relevant descriptions in the above embodiments.

[0113] Optionally, in one embodiment, the downsampling process includes sequentially performing depthwise convolution processing, batch normalization processing, feedforward processing, pointwise convolution processing, batch normalization processing, linear rectification activation processing, depthwise convolution processing, batch normalization processing, channel attention enhancement processing, pointwise convolution processing, batch normalization processing, depthwise convolution processing, batch normalization processing, and feedforward processing, wherein the stride of the second depthwise convolution processing is S1, and the stride of the other convolution processing is 1, where S1 is a positive integer greater than or equal to 2.

[0114] This application provides an optional downsampling processing scheme and an optional architecture for the corresponding downsampling module.

[0115] Please refer to Figure 1j The downsampling module includes a first depthwise convolution + batch normalization layer, a first feedforward layer, a standard convolution + batch normalization + linear rectified activation layer, a second depthwise convolution + batch normalization layer, a channel attention enhancement layer (which can be implemented using a compression and activation network), a standard convolution layer + batch normalization layer, a third depthwise convolution + batch normalization layer, and a second feedforward layer. In the second depthwise convolution + batch normalization layer, the stride S1 of the depthwise convolutions is used, while the stride of other convolutions is 1, where S1 is a positive integer greater than or equal to 2, and the kernel size of all convolutions is 3x3. In other embodiments, those skilled in the art can, according to actual needs, split the depthwise convolution + batch normalization layer into separate depthwise convolution layers and batch normalization layers, or split the standard convolution + batch normalization + linear rectified activation layer into separate standard convolution layers, batch normalization layers, and linear rectified activation layers.

[0116] Accordingly, when performing downsampling processing through the downsampling module, the feature map to be downsampled is input into the first depthwise convolution + batch normalization layer of the downsampling module. This layer sequentially performs depthwise convolution and batch normalization on the input feature map, yielding the first batch normalization result. This batch normalization result is then input into the first feedforward layer of the downsampling module for feedforward processing, yielding the first feedforward result. This feedforward result is then input into the standard convolution + batch normalization + linear rectified activation layer of the downsampling module. This layer sequentially performs standard convolution, batch normalization, and linear rectified activation on the feedforward result, yielding the activation result. Finally, this activation result is input into the second depthwise convolution + batch normalization layer of the downsampling module. This layer sequentially performs depthwise convolution and batch normalization on the input activation result. The process is as follows: First, the second batch normalization result is obtained. This batch normalization result is then input into the channel attention enhancement layer of the downsampling module. This layer performs channel attention enhancement processing on the batch normalization result, resulting in a channel attention enhancement result. Next, this channel attention enhancement result is input into the standard convolution + batch normalization layer of the downsampling module. This layer then performs standard convolution and batch normalization processing sequentially on the channel attention enhancement result, resulting in a third batch normalization result. Finally, this batch normalization result is input into the third depthwise convolution + batch normalization layer of the downsampling module. This layer then performs depthwise convolution and batch normalization processing sequentially on the batch normalization result, resulting in a fourth batch normalization result. Finally, this batch normalization result is input into the second feedforward layer of the downsampling module for feedforward processing, resulting in the second feedforward result, which is the downsampled feature map of the input feature map. Since the stride of the second depthwise convolution + batch normalization layer is 2, the resolution of the downsampled feature map is 1 / 2 of the input feature map, that is, the width of the downsampled feature map is 1 / 2 of the width of the input feature map, and the height of the downsampled feature map is 1 / 2 of the height of the input feature map.

[0117] Optionally, in one embodiment, the feedforward processing includes sequentially performing pointwise convolution processing, batch normalization processing, linear rectification activation processing, pointwise convolution processing, and batch normalization processing.

[0118] This application provides an optional feedforward processing scheme and a corresponding optional architecture for the feedforward layer.

[0119] Please refer to Figure 1k The feedforward layer includes a first pointwise convolutional layer, a first batch normalization layer, a linear rectified activation layer, a second pointwise convolutional layer, and a second batch normalization layer, wherein the kernel size of all convolutions is 3x3. In other embodiments, the architecture of the feedforward layer can also be configured by those skilled in the art according to actual needs.

[0120] Accordingly, during feedforward processing, the feature map requiring feedforward processing is input into the first pointwise convolutional layer of the feedforward layer. This pointwise convolutional layer performs pointwise convolution on the input feature map, yielding the first convolution result. This convolution result is then input into the first batch normalization layer of the feedforward layer for batch normalization, yielding the first batch normalized result. This batch normalized result is then input into the linear rectified activation layer of the feedforward layer for linear rectified activation, yielding the activation result. This activation result is then input into the second pointwise convolutional layer of the feedforward layer for pointwise convolution, yielding the second convolution result. Finally, this convolution result is input into the second batch normalization layer of the feedforward layer for batch normalization, yielding the second batch normalized result, which is the feedforward result of the input feature map.

[0121] Optionally, in one embodiment, splitting the initial feature map into N spatial sub-feature maps in the spatial dimension includes:

[0122] The initial feature map is sequentially processed by depthwise convolution, batch normalization, feedforward processing, and residual connection processing to obtain the processed feature map.

[0123] The processed feature map is split into N spatial sub-feature maps in the spatial dimension.

[0124] In the embodiments of this application, please refer to Figure 11 The feature modeling module also includes a preprocessing layer, which comprises a depthwise convolutional layer with batch normalization and a feedforward layer. The kernel size of the depthwise convolutional layer is 3x3. In other embodiments, the depthwise convolutional layer with batch normalization can also be split into independent depthwise convolutional layers and batch normalization layers. The architecture of the feedforward layer is described in the relevant embodiments above and will not be repeated here.

[0125] Correspondingly, when splitting the initial feature map into N spatial sub-feature maps in the spatial dimension, the initial feature map is first input into the depthwise convolution + batch normalization layer in the preprocessing layer. The initial feature map is then subjected to depthwise convolution and batch normalization processing sequentially through this depthwise convolution + batch normalization layer to obtain the batch normalization result. The batch normalization result is then input into the feedforward layer in the preprocessing layer. The feedforward layer processes the batch normalization result through feedforward processing to obtain the feedforward result. The feedforward result is then joined with the initial feature map using residual connection processing, which is equivalent to adding the feedforward result to the initial feature map to obtain the processed feature map of the preprocessing layer. Finally, the processed feature map is input into the spatial splitting layer, which splits the processed feature map into N spatial sub-feature maps in the spatial dimension.

[0126] Optionally, in one embodiment, a modeling feature map is obtained by fusion of N spatial sub-feature maps in the spatial dimension, including:

[0127] The spliced ​​feature map is obtained by fusion sub-modeling feature map by splicing N spatial sub-feature maps in the spatial dimension;

[0128] The concatenated feature map is added to the processed feature map to obtain the intermediate feature map;

[0129] The intermediate feature maps are sequentially processed by depthwise convolution, batch normalization, and feedforward to obtain the modeling feature maps.

[0130] In the embodiments of this application, please refer to Figure 1m The feature modeling module also includes a post-processing layer, which comprises a depthwise convolutional layer with batch normalization and a feedforward layer. The kernel size of the depthwise convolutional layer is 3x3. In other embodiments, the depthwise convolutional layer with batch normalization can also be split into independent depthwise convolutional layers and batch normalization layers. The architecture of the feedforward layer is described in the relevant embodiments above and will not be repeated here.

[0131] Correspondingly, when concatenating the modeling feature map from N spatial sub-feature maps to obtain the modeling feature map, the fused modeling feature map of the N spatial sub-feature maps is first input into the spatial concatenation layer. The spatial concatenation layer concatenates the fused modeling feature map of the N spatial sub-feature maps in the spatial dimension to obtain the concatenated feature map. At this time, the concatenated feature map is not used as the modeling feature map of the initial feature map. Instead, the concatenated feature map is added to the processed feature map of the preprocessing layer to obtain the intermediate feature map. The intermediate feature map is input into the depthwise convolution + batch normalization layer of the post-processing layer. The depthwise convolution + batch normalization layer sequentially performs depthwise convolution and batch normalization processing on the intermediate feature map to obtain the batch normalization result. The batch normalization result is input into the feedforward layer in the post-processing layer. The feedforward layer performs feedforward processing on the batch normalization result to obtain the feedforward result, which is the modeling feature map of the initial feature map.

[0132] It should be noted that, unless otherwise specified, the stride of any convolutional processing involved in the above embodiments is 1.

[0133] Optionally, in one embodiment, for the visual foundation model provided in this application, an optional training scheme is provided, including:

[0134] Obtain sample images and their label categories;

[0135] Feature extraction is performed on the sample images using a visual basic model to obtain the output feature map of the sample images;

[0136] The output feature map is classified by the classification task head to obtain the predicted category of the sample image;

[0137] The training loss is obtained based on the difference between the predicted category and the label category, and the model parameters of the visual base model are updated based on the training loss until a preset stopping condition is met.

[0138] In this embodiment, no specific restrictions are placed on the architecture of the classification task head, the selection of training loss, or the configuration of the preset stopping condition. These can be configured by those skilled in the art according to actual needs. For example, the classification task head can adopt an architecture of fully connected layers and soft maximization layers, the training loss can be cross-entropy loss, and the preset stopping condition can be configured to the number of parameter updates of the visual base model reaching a preset number, or it can be configured to the convergence of the training loss, and so on.

[0139] As can be seen from the above, the image feature extraction scheme provided in this application first obtains an initial feature map with a resolution smaller than that of the image to be processed through embedding processing. Then, in the spatial dimension, the initial feature map is first split into N spatial sub-feature maps, and in the channel dimension, each spatial sub-feature map is further split into M channel sub-feature maps. Each channel sub-feature map is used as the object of feature modeling. In this way, the amount of data for a single feature modeling is reduced from both the spatial and channel dimensions, thereby reducing the computational complexity. On the other hand, in the feature modeling process, at least one identity mapping process, local feature modeling process, and global feature modeling process are performed on the M channel sub-feature maps of the same spatial sub-feature map to obtain a sub-modeling feature map. The M sub-modeling feature maps are then concatenated to obtain a fused sub-modeling feature map of the spatial sub-feature map. Finally, the fused sub-modeling feature map of the N spatial sub-feature maps is concatenated to obtain a modeling feature map. Thus, based on the modeling feature map, an output feature map that can represent both local spatial characteristics and global spatial characteristics is obtained. Therefore, by setting the architecture of the visual basic model according to the image feature extraction scheme provided in this application, the visual basic model can have both high feature extraction performance and low computational complexity, thereby enabling the visual task model based on the visual basic model to be effectively deployed on devices with limited processing resources.

[0140] Please refer to Figure 2a and Figure 2b The following describes the image feature extraction method provided in this application, using an electronic device as the execution subject. Figure 2aAs shown, the visual foundation model provided in this application sequentially includes one embedding module, one independent feature modeling module, and two optimization processing modules. The embedding module sequentially includes three layers of standard convolution + batch normalization + linear rectified activation layers and one layer of standard convolution + batch normalization layer, where the stride of all convolutions is 2 and the kernel size is 3x3. The feature modeling module sequentially includes one preprocessing layer, one separable moving state space model layer, and one postprocessing layer. The preprocessing layer sequentially includes one depthwise convolution layer. The system consists of a batch normalization layer and a feedforward layer. The feedforward layer comprises a pointwise convolutional layer, a batch normalization layer, a linear rectified activation layer, a pointwise convolutional layer, and a batch normalization layer. The separable moving state space model layer includes a spatial splitting layer, a channel splitting layer, four modeling branches, a channel stitching layer, and a spatial stitching layer. The four modeling branches are, in order, an identity mapping branch, two local feature modeling branches, and one global feature modeling branch based on the Mamba module. The local feature modeling branch consists of one depthwise convolutional layer, one batch normalization layer, one linear rectified activation layer, one pointwise convolutional layer, and one batch normalization layer. The post-processing layer consists of one depthwise convolutional layer + batch normalization layer and one feedforward layer. In the two local feature modeling branches, the stride of the depthwise convolution is 1, and the kernel sizes are 3x3 and 5x5, respectively. All other convolutions except the pointwise convolution have a kernel size of 3x3 and a stride of 1. The optimization module consists of one downsampling module and one feature optimization module. The feature modeling module and the downsampling module sequentially include one depthwise convolution + batch normalization layer, one feedforward layer, one standard convolution + batch normalization + linear rectified activation layer, one depthwise convolution + batch normalization layer, one channel attention enhancement layer, one standard convolution + batch normalization layer, one depthwise convolution + batch normalization layer, and one feedforward layer. In the second depthwise convolution + batch normalization layer, the stride of the depthwise convolution is 2, and the stride of the other convolutions is 1. The kernel size of all convolutions is 3x3.

[0141] based on Figure 2a The provided visual base model, such as Figure 2b As shown, the process of this image feature extraction method can also be as follows:

[0142] In 210, the electronic device acquires the image to be processed, which requires feature extraction.

[0143] In this embodiment, the electronic device can respond to a task processing request and obtain the image to be processed for visual task processing according to the task processing request. For example, the electronic device that performs the image feature extraction method can be a smartphone. After the smartphone takes a picture and generates a new image, it uses the image as the image to be classified and generates an image classification request for the image, instructing that feature extraction be performed on the image, thereby using the extracted output feature map to complete the image classification.

[0144] In 220, the electronic device performs embedding processing on the image to be processed through the embedding module to obtain an initial feature map of the image to be processed. The resolution of the initial feature map is smaller than the resolution of the image to be processed.

[0145] In this embodiment of the application, the electronic device can input the acquired image to be processed into the embedding module, and the embedding module performs embedding processing on the image to be processed to obtain an initial feature map with a resolution lower than that of the image to be processed.

[0146] The electronic device inputs the image to be processed into the first layer of the embedding module: a standard convolutional layer + batch normalization layer + linear rectified activation layer. This layer sequentially performs standard convolution, batch normalization, and linear rectified activation processing on the input image, completing the first mapping process. Thus, through three layers of standard convolutional layers + batch normalization layers, three mapping processes are completed to obtain the mapping result. The electronic device further inputs the mapping result into the standard convolutional layer + batch normalization layer of the embedding module. This layer sequentially performs standard convolution and batch normalization processing on the mapping result, yielding the output of the embedding module, which is the initial feature map of the image to be processed.

[0147] It is understood that in this embodiment, the stride of all standard convolutions in the embedding module is set to 2. Therefore, after embedding processing, the resolution of the initial feature map of the image to be processed is 1 / 16 of the resolution of the image to be processed; that is, the height of the initial feature map is 1 / 16 of the height of the image to be processed, and the width of the initial feature map is 1 / 16 of the width of the image to be processed. Furthermore, since the image to be processed undergoes multiple standard convolutions during the embedding process, the number of channels in the initial feature map will be greater than the number of channels in the image to be processed. For example, if the image to be processed is an image with height H and width W, it can be represented as X∈R. HxWx3 The corresponding initial feature map can be represented as Where C1 represents the number of channels in the initial feature map, C1>3, and its value depends on the specific configuration of the standard convolution in the embedding module.

[0148] In 230, the electronic device performs feature modeling processing on the initial feature map through the feature modeling module to obtain the modeled feature map.

[0149] The electronic device first inputs the initial feature map into the depthwise convolution + batch normalization layer in the preprocessing layer of the feature modeling module. This layer sequentially performs depthwise convolution and batch normalization on the initial feature map, yielding a batch normalized result. This batch normalized result is then input into the feedforward layer in the preprocessing layer. The feedforward layer performs feedforward processing on the batch normalized result, yielding a feedforward result. This feedforward result is then joined with the initial feature map using a residual connection, effectively adding the feedforward result to the initial feature map, resulting in the processed feature map of the preprocessing layer. Finally, this processed feature map is input into the spatial splitting layer, which splits the processed feature map into N spatial sub-feature maps in the spatial dimension. The feedforward processing includes sequential pointwise convolution, batch normalization, linear rectified activation, pointwise convolution, and batch normalization.

[0150] For each spatial sub-feature map, the electronic device splits it into four channel sub-feature maps in the channel dimension through the feature modeling module. The electronic device inputs the first channel sub-feature map into the identity mapping branch of the feature modeling module. The identity mapping branch performs identity mapping on the first channel sub-feature map to obtain a mapped feature map, which is then used as the sub-modeling feature map of the first channel sub-feature map. This can be represented as: f′0=f1, where f0 represents the first channel sub-feature map, and f0′ represents the mapped feature map, i.e., the sub-modeling feature map of the first channel sub-feature map. The second channel sub-feature map is added to the sub-modeling feature map of the first channel sub-feature map to obtain the first fused feature map. This first fused feature map is input into the first local feature modeling branch of the feature modeling module. The first local feature modeling branch performs local feature modeling on the first fused feature map to obtain the sub-modeling feature map of the second channel sub-feature map, which can be represented as: f′1=LE k＝3 (f1+f0′), where f′1 represents the sub-modeling feature map of the second channel sub-feature map, LE k＝3 () represents the local feature modeling branch with a 3x3 kernel size for depthwise convolution, and f1 represents the second channel sub-feature map. The third channel sub-feature map is added to the sub-modeling feature map of the second channel sub-feature map to obtain the second first fused feature map. This first fused feature map is input into the second local feature modeling branch of the feature modeling module. This branch performs local feature modeling processing on the first fused feature map to obtain the sub-modeling feature map of the third channel sub-feature map, which can be expressed as: f′2=LE k＝5 (f2+f1′), where f′2 represents the sub-modeling feature map of the third channel sub-feature map, LE k＝5() represents the local feature modeling branch with a 5x5 kernel size for depthwise convolution, and f2 represents the third channel sub-feature map. The fourth channel sub-feature map is added to the sub-modeling feature map of the third channel sub-feature map to obtain the second fused feature map. This second fused feature map is then input into the global feature modeling branch of the feature modeling module. The global feature modeling branch performs global feature modeling processing on the second fused feature map to obtain the sub-modeling feature map of the fourth channel sub-feature map, which can be expressed as: f′3=Mamba(f3+f2′), where f′3 represents the sub-modeling feature map of the fourth channel sub-feature map, Mamba() represents the global feature modeling branch based on the Mamba module, and f3 represents the fourth channel sub-feature map. Electronic devices concatenate the sub-modeling feature maps of four channel sub-feature maps along the channel dimension to obtain the fused sub-modeling feature map of the spatial sub-feature map.

[0151] The electronic device inputs the fusion sub-modeling feature map of N spatial sub-feature maps into the spatial stitching layer. The spatial stitching layer stitches the fusion sub-modeling feature map of N spatial sub-feature maps in the spatial dimension to obtain the stitched feature map. The stitched feature map is then added to the processed feature map of the preprocessing layer to obtain the intermediate feature map. The intermediate feature map is then input into the depthwise convolution + batch normalization layer of the post-processing layer. The depthwise convolution + batch normalization layer sequentially performs depthwise convolution and batch normalization processing on the intermediate feature map to obtain the batch normalization result. The batch normalization result is then input into the feedforward layer in the post-processing layer. The feedforward layer performs feedforward processing on the batch normalization result to obtain the feedforward result, which is the modeling feature map of the initial feature map.

[0152] For example, the initial feature map is represented as After performing feature modeling processing using the above independent feature modeling modules, the resulting modeled feature map can be represented as follows: C2>C1, and its value depends on the specific configuration of the convolution in the feature modeling module.

[0153] In step 240, the electronic device optimizes the modeling feature map through the first optimization processing module to obtain an optimized feature map.

[0154] The electronic device inputs the modeling feature map into the downsampling module in the first optimization processing module. The first depthwise convolution + batch normalization layer of this downsampling module sequentially performs depthwise convolution and batch normalization on the modeling feature map, obtaining the first batch normalization result. This batch normalization result is then input into the first feedforward layer of the downsampling module for feedforward processing, obtaining the first feedforward result. This feedforward result is then input into the standard convolution + batch normalization + linear rectified activation layer of the downsampling module. This standard convolution + batch normalization + linear rectified activation layer sequentially performs standard convolution, batch normalization, and linear rectified activation on the feedforward result, obtaining the activation result. This activation result is then input into the second depthwise convolution + batch normalization layer of the downsampling module. This depthwise convolution + batch normalization layer sequentially performs depthwise convolution and batch normalization on the input activation result, obtaining the second batch... The normalized results are then input into the channel attention enhancement layer of the downsampling module. This layer performs channel attention enhancement processing on the normalized results to obtain the channel attention enhancement results. The channel attention enhancement results are then input into the standard convolution + batch normalization layer of the downsampling module. This layer sequentially performs standard convolution and batch normalization processing on the channel attention enhancement results to obtain the third batch normalization result. This batch normalization result is then input into the third depthwise convolution + batch normalization layer of the downsampling module. This layer sequentially performs depthwise convolution and batch normalization processing on the batch normalization results to obtain the fourth batch normalization result. Finally, this batch normalization result is input into the second feedforward layer of the downsampling module for feedforward processing to obtain the second feedforward result, which is the downsampled feature map of the modeled feature map. The electronic device inputs the downsampled feature map into the feature modeling module in the first optimization module. This module performs feature modeling on the downsampled feature map, thus optimizing the feature modeling map and obtaining the optimized feature map. The method for performing feature modeling on the downsampled feature map using the feature modeling module can be implemented similarly to the method described above for performing feature modeling on the initial feature map; it will not be elaborated upon here.

[0155] For example, the modeled feature map obtained by performing feature modeling processing on the initial feature map through an independent feature modeling module is represented as follows: After the above optimization modules have been applied, the resulting optimized feature map can be represented as follows: C3 > C2, and its value depends on the specific configuration of the convolution in the first optimization module.

[0156] In step 250, the electronic device optimizes the feature map through the second optimization processing module to obtain the output feature map of the image to be processed.

[0157] Furthermore, the electronic device optimizes the optimized feature map through the second optimization processing module to obtain the output feature map of the image to be processed. The optimization of the optimized feature map through the second optimization processing module can be implemented in accordance with the method of optimizing the modeling feature map through the first optimization processing module in the above embodiment, and will not be elaborated here.

[0158] For example, the optimized feature map obtained by optimizing the modeling feature map through the first optimization processing module is represented as follows: After optimization by the second optimization module, the resulting output feature map can be represented as follows: C4 > C3, and its value depends on the specific configuration of the convolution in the second optimization module.

[0159] As shown above, after obtaining the output feature map of the image to be processed through the second optimization processing module, the visual task processing of the image to be processed can be completed using the output feature map.

[0160] To verify the feature extraction capability of the visual basic model (named MobileMamba) provided in the embodiments of this application, the following comparative experiment was conducted.

[0161] EdgeViT-XXS, MobileViT-XS, ShuffleNetV2 2.0×, MobileNetV3-Large, MobileViTV2-0.75, MobileOne-S1, GLiT-Tiny, EfficientNet-B0, EfficientViT-M5, and SHViT-S3 were selected as the baseline visual models for comparison. The feature extraction capability of the baseline visual model (MobileMamba) provided in this application was verified on the ImageNet-1K image dataset using a classification task. The experimental data are shown in Table 1 below. Table 1 is the comparison data table for the classification task.

[0162]

[0163]

[0164] Table 1

[0165] As shown in the table above, the visual base model (MobileMamba) provided in this application embodiment has the highest Top-1 accuracy in classification tasks compared to the control visual base model, and the number of floating-point operations per second required is at a low level. This indicates that the visual base model (MobileMamba) provided in this application embodiment has higher feature extraction performance and lower computational complexity compared to the control visual base model.

[0166] To facilitate better implementation of the above image feature extraction methods, this application also provides a corresponding image feature extraction device. The meanings of the terms used are the same as in the above image feature extraction methods; for specific implementation details, please refer to the descriptions in the above method embodiments.

[0167] Please refer to Figure 3 , Figure 3 This is a schematic diagram of the structure of the image feature extraction device provided in the embodiments of this application. The image feature extraction device may include an image embedding module 310 and a feature extraction module 320, wherein...

[0168] The image embedding module 310 is used to perform embedding processing on the image to be processed to obtain an initial feature map of the image to be processed. The resolution of the initial feature map is smaller than the resolution of the image to be processed.

[0169] The feature extraction module 320 is used to perform feature modeling processing on the initial feature map to obtain a modeled feature map, and to obtain the output feature map of the image to be processed based on the modeled feature map;

[0170] Feature modeling processing includes:

[0171] In the spatial dimension, the initial feature map is split into N spatial sub-feature maps, and in the channel dimension, each spatial sub-feature map is split into M channel sub-feature maps, where N is a positive integer greater than or equal to 2 and M is a positive integer greater than or equal to 3.

[0172] For each spatial sub-feature map corresponding to M channel sub-feature maps, the first channel sub-feature map is processed by one of identity mapping, local feature modeling, or global feature modeling to obtain the sub-modeling feature map of the first channel sub-feature map. The i-th channel sub-feature map is added to the sub-modeling feature map of the (i-1)-th channel sub-feature map to obtain the fused feature map. The fused feature map is then processed by one of identity mapping, local feature modeling, or global feature modeling to obtain the sub-modeling feature map of the i-th channel sub-feature map, i∈[2,M]. This includes at least one identity mapping, local feature modeling, and global feature modeling process. The sub-modeling feature maps of the M channel sub-feature maps are concatenated along the channel dimension to obtain the fused sub-modeling feature map of the spatial sub-feature map.

[0173] The modeling feature map is obtained by merging N spatial sub-feature maps in the spatial dimension into a fusion sub-modeling feature map.

[0174] Optionally, in one embodiment, the feature extraction module 320 is used to perform identity mapping processing on the first channel sub-feature map to obtain a mapped feature map, and use the mapped feature map as the sub-modeling feature map of the first channel sub-feature map; add the i-th channel sub-feature map and the sub-modeling feature map of the (i-1)-th channel sub-feature map to obtain a first fusion feature map, and perform local feature modeling processing on the first fusion feature map to obtain the sub-modeling feature map of the i-th channel sub-feature map, i∈[2,M-1]; add the M-th channel sub-feature map and the sub-modeling feature map of the (M-1)-th channel sub-feature map to obtain a second fusion feature map, and perform global feature modeling processing on the second fusion feature map to obtain the sub-modeling feature map of the M-th channel sub-feature map.

[0175] Optionally, in one embodiment, the feature extraction module 320 is used to sequentially perform depthwise convolution, batch normalization, linear rectified activation, pointwise convolution, and batch normalization on the first fused feature map to obtain the sub-modeling feature map of the i-th channel sub-feature map.

[0176] Optionally, in one embodiment, the feature extraction module 320 is used to perform global modeling processing on the second fused feature map through the Mamba module to obtain the sub-modeling feature map of the Mth channel sub-feature map.

[0177] Optionally, in one embodiment, when M is greater than or equal to 4, the kernel size of the depthwise convolution process in the later local feature modeling process is greater than the kernel size of the depthwise convolution process in the earlier local feature modeling process.

[0178] Optionally, in one embodiment, the feature extraction module 320 is used to perform at least one optimization process on the feature modeling map to obtain an output feature map; the optimization process includes downsampling processing and feature modeling processing performed sequentially.

[0179] Optionally, in one embodiment, the downsampling process includes sequentially performing depthwise convolution processing, batch normalization processing, feedforward processing, pointwise convolution processing, batch normalization processing, linear rectification activation processing, depthwise convolution processing, batch normalization processing, channel attention enhancement processing, pointwise convolution processing, batch normalization processing, depthwise convolution processing, batch normalization processing, and feedforward processing, wherein the stride of the second depthwise convolution processing is S1, and the stride of the other convolution processing is 1, where S1 is a positive integer greater than or equal to 2.

[0180] Optionally, in one embodiment, the feedforward processing includes sequentially performing pointwise convolution processing, batch normalization processing, linear rectification activation processing, pointwise convolution processing, and batch normalization processing.

[0181] Optionally, in one embodiment, the image embedding module 310 is used to perform L mapping processes, one standard convolution process, and one batch normalization process on the image to be processed in sequence to obtain an initial feature map of the image to be processed; the mapping process includes the sequential standard convolution process, batch normalization process, and linear rectified activation process; wherein, the step size of all standard convolution processes in the embedding process is S2, where S2 is a positive integer greater than or equal to 2, and L is a positive integer greater than or equal to 3.

[0182] Optionally, in one embodiment, the feature extraction module 320 is used to sequentially perform depthwise convolution, batch normalization, feedforward processing, and residual connection processing on the initial feature map to obtain the processed feature map; and to split the processed feature map into N spatial sub-feature maps in the spatial dimension.

[0183] Optionally, in one embodiment, the feature extraction module 320 is used to stitch together N spatial sub-feature maps in the spatial dimension to obtain a fused sub-modeling feature map; add the stitched feature map to the processed feature map to obtain an intermediate feature map; and sequentially perform depthwise convolution, batch normalization, and feedforward processing on the intermediate feature map to obtain a modeling feature map.

[0184] For details on the implementation of each of the above modules, please refer to the previous examples, which will not be repeated here.

[0185] This application also provides an electronic device, including a memory and a processor, wherein the processor executes the steps in the image feature extraction method provided in the above embodiments by calling a computer program stored in the memory.

[0186] Please refer to Figure 4 , Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0187] The electronic device may include components such as a processor 101 with one or more processing cores, a memory 102 with one or more computer-readable storage media, a power supply 103, and an input unit 104. Those skilled in the art will understand that... Figure 4 The electronic device structure shown does not constitute a limitation on the electronic device and may include more or fewer components than shown, or combine certain components, or have different component arrangements. Wherein:

[0188] The processor 101 is the control center of the electronic device, connecting various parts of the device via various interfaces and lines. It executes software programs and / or modules stored in the memory 102, and calls data stored in the memory 102, to perform various functions and process data. Optionally, the processor 101 may include one or more processing cores; alternatively, the processor 101 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, while the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 101.

[0189] The memory 102 can be used to store software programs and modules. The processor 101 executes various functional applications and data processing by running the software programs and modules stored in the memory 102. The memory 102 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 102 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 102 may also include a memory controller to provide the processor 101 with access to the memory 102.

[0190] The electronic device also includes a power supply 103 that supplies power to the various components. Optionally, the power supply 103 can be logically connected to the processor 101 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The power supply 103 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0191] The electronic device may also include an input unit 104, which can be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.

[0192] Although not shown, the electronic device may also include a display unit, an image acquisition component, etc., which will not be described in detail here. Specifically, in this embodiment, the processor 101 loads one or more executable codes corresponding to computer programs into the memory 102, and the processor 101 executes the steps in the image feature extraction method provided in this application, such as:

[0193] Embedding is performed on the image to be processed to obtain an initial feature map of the image to be processed. The resolution of the initial feature map is smaller than the resolution of the image to be processed.

[0194] The initial feature map is processed by feature modeling to obtain a modeled feature map, and the output feature map of the image to be processed is obtained based on the modeled feature map;

[0195] Feature modeling processing includes:

[0196] In the spatial dimension, the initial feature map is split into N spatial sub-feature maps, and in the channel dimension, each spatial sub-feature map is split into M channel sub-feature maps, where N is a positive integer greater than or equal to 2 and M is a positive integer greater than or equal to 3.

[0197] For each spatial sub-feature map corresponding to M channel sub-feature maps, the first channel sub-feature map is processed by one of identity mapping, local feature modeling, or global feature modeling to obtain the sub-modeling feature map of the first channel sub-feature map. The i-th channel sub-feature map is added to the sub-modeling feature map of the (i-1)-th channel sub-feature map to obtain the fused feature map. The fused feature map is then processed by one of identity mapping, local feature modeling, or global feature modeling to obtain the sub-modeling feature map of the i-th channel sub-feature map, i∈[2,M]. This includes at least one identity mapping, local feature modeling, and global feature modeling process. The sub-modeling feature maps of the M channel sub-feature maps are concatenated along the channel dimension to obtain the fused sub-modeling feature map of the spatial sub-feature map.

[0198] The modeling feature map is obtained by merging N spatial sub-feature maps in the spatial dimension into a fusion sub-modeling feature map.

[0199] It should be noted that the electronic device provided in this application embodiment and the image feature extraction method in the above embodiment belong to the same concept. The specific implementation process can be found in the above related embodiments, and will not be repeated here.

[0200] This application also provides a computer-readable storage medium storing a computer program thereon. When the computer program stored thereon is executed on the processor of the electronic device provided in the embodiments of this application, the processor of the electronic device implements the steps in the image feature extraction method provided in this application. The storage medium may be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0201] This application also provides a computer program product, which includes a computer program that, when executed on the processor of the electronic device provided in the embodiments of this application, causes the processor of the electronic device to implement the steps in the image feature extraction method provided in this application.

[0202] The image feature extraction method, image feature extraction device, electronic device, computer-readable storage medium, and computer program product provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

[0203] It should be noted that when the above embodiments of this application are applied to specific products or technologies, and user-related data is involved, user permission or consent is required, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

Claims

1. An image feature extraction method, characterized in that, include: Embedding processing is performed on the image to be processed to obtain an initial feature map of the image to be processed, wherein the resolution of the initial feature map is smaller than the resolution of the image to be processed; The initial feature map is subjected to feature modeling processing to obtain a modeled feature map, and the output feature map of the image to be processed is obtained based on the modeled feature map; The feature modeling process includes: The initial feature map is split into N spatial sub-feature maps in the spatial dimension, and each spatial sub-feature map is split into M channel sub-feature maps in the channel dimension, where N is a positive integer greater than or equal to 2 and M is a positive integer greater than or equal to 3. For each spatial sub-feature map corresponding to M channel sub-feature maps, the first channel sub-feature map is processed by one of identity mapping, local feature modeling, or global feature modeling to obtain the sub-modeling feature map of the first channel sub-feature map. The i-th channel sub-feature map is added to the sub-modeling feature map of the (i-1)-th channel sub-feature map to obtain the fused feature map. The fused feature map is then processed by one of identity mapping, local feature modeling, or global feature modeling to obtain the sub-modeling feature map of the i-th channel sub-feature map, i∈[2,M]. This includes at least one instance of identity mapping, local feature modeling, and global feature modeling. The sub-modeling feature maps of the M channel sub-feature maps are concatenated along the channel dimension to obtain the fused sub-modeling feature map of the spatial sub-feature map. The modeling feature map is obtained by splicing the N spatial sub-feature maps together in the spatial dimension to form a fused sub-modeling feature map.

2. The image feature extraction method according to claim 1, characterized in that, The process of performing one of the following processing on the first channel sub-feature map—identity mapping, local feature modeling, or global feature modeling—to obtain a sub-modeling feature map of the first channel sub-feature map, adding the sub-modeling feature map of the i-th channel sub-feature map and the sub-modeling feature map of the (i-1)-th channel sub-feature map to obtain a fused feature map, and then performing one of the following processing on the fused feature map—identity mapping, local feature modeling, or global feature modeling—to obtain a sub-modeling feature map of the i-th channel sub-feature map, i∈[2,M], includes: The first channel sub-feature map is subjected to identity mapping to obtain a mapped feature map, and the mapped feature map is used as the sub-modeling feature map of the first channel sub-feature map; The first fused feature map is obtained by adding the sub-modeling feature map of the i-th channel sub-feature map to the sub-modeling feature map of the (i-1)-th channel sub-feature map, and local feature modeling processing is performed on the first fused feature map to obtain the sub-modeling feature map of the i-th channel sub-feature map, i∈[2,M-1]. The second fused feature map is obtained by adding the sub-modeling feature map of the Mth channel sub-feature map to the sub-modeling feature map of the (M-1)th channel sub-feature map, and then performing global feature modeling processing on the second fused feature map to obtain the sub-modeling feature map of the Mth channel sub-feature map.

3. The image feature extraction method according to claim 2, characterized in that, The step of performing local feature modeling processing on the first fused feature map to obtain the sub-modeling feature map of the i-th channel sub-feature map includes: The first fused feature map is sequentially processed by depthwise convolution, batch normalization, linear rectified activation, pointwise convolution, and batch normalization to obtain the sub-modeling feature map of the i-th channel sub-feature map.

4. The image feature extraction method according to claim 2, characterized in that, The global modeling process of the second fused feature map to obtain the sub-modeling feature map of the Mth channel sub-feature map includes: The second fused feature map is globally modeled using the Mamba module to obtain the sub-modeling feature map of the Mth channel sub-feature map.

5. The image feature extraction method according to claim 2, characterized in that, When M is greater than or equal to 4, the kernel size of the convolution kernel in the depthwise convolution process during the later local feature modeling process is greater than the kernel size of the convolution kernel in the depthwise convolution process during the earlier local feature modeling process.

6. The image feature extraction method according to claim 1, characterized in that, The step of obtaining the output feature map of the image to be processed based on the modeling feature map includes: The feature modeling map is optimized at least once to obtain the output feature map; The optimization process includes sequential downsampling and feature modeling.

7. The image feature extraction method according to claim 6, characterized in that, The downsampling process includes the following sequential processes: depthwise convolution, batch normalization, feedforward, pointwise convolution, batch normalization, linear rectification activation, depthwise convolution, batch normalization, channel attention enhancement, pointwise convolution, batch normalization, depthwise convolution, batch normalization, and feedforward. The step size of the second depthwise convolution is S1, and the step size of the other convolution processes is 1, where S1 is a positive integer greater than or equal to 2.

8. The image feature extraction method according to claim 7, characterized in that, The feedforward processing includes sequential pointwise convolution processing, batch normalization processing, linear rectification activation processing, pointwise convolution processing, and batch normalization processing.

9. The image feature extraction method according to claim 1, characterized in that, The embedding process of the image to be processed to obtain the initial feature map of the image to be processed includes: The image to be processed is subjected to L mapping processes, one standard convolution process, and one batch normalization process in sequence to obtain the initial feature map of the image to be processed. The mapping process includes standard convolution processing, batch normalization processing, and linear rectified activation processing performed sequentially. In the embedding process, the stride of all standard convolution processes is S2, where S2 is a positive integer greater than or equal to 2, and L is a positive integer greater than or equal to 3.

10. The image feature extraction method according to claim 1, characterized in that, The step of splitting the initial feature map into N spatial sub-feature maps in the spatial dimension includes: The initial feature map is sequentially processed by depthwise convolution, batch normalization, feedforward processing, and residual connection processing to obtain the processed feature map. The processed feature map is split into N spatial sub-feature maps in the spatial dimension.

11. The image feature extraction method according to claim 10, characterized in that, The process of concatenating the N spatial sub-feature maps in the spatial dimension to obtain the fused sub-modeling feature map includes: The spliced ​​feature map is obtained by splicing the N spatial sub-feature maps together in the spatial dimension to form a fusion sub-modeling feature map. The spliced ​​feature map is added to the processed feature map to obtain the intermediate feature map; The intermediate feature map is sequentially subjected to depthwise convolution, batch normalization, and feedforward processing to obtain the modeling feature map.

12. An image feature extraction device, characterized in that, include: An image embedding module is used to embed the image to be processed to obtain an initial feature map of the image to be processed, wherein the resolution of the initial feature map is smaller than the resolution of the image to be processed. The feature extraction module is used to perform feature modeling processing on the initial feature map to obtain a modeled feature map, and to obtain the output feature map of the image to be processed based on the modeled feature map; The feature modeling process includes: The initial feature map is split into N spatial sub-feature maps in the spatial dimension, and each spatial sub-feature map is split into M channel sub-feature maps in the channel dimension, where N is a positive integer greater than or equal to 2 and M is a positive integer greater than or equal to 3. For each spatial sub-feature map corresponding to M channel sub-feature maps, the first channel sub-feature map is processed by one of identity mapping, local feature modeling, or global feature modeling to obtain the sub-modeling feature map of the first channel sub-feature map. The i-th channel sub-feature map is added to the sub-modeling feature map of the (i-1)-th channel sub-feature map to obtain the fused feature map. The fused feature map is then processed by one of identity mapping, local feature modeling, or global feature modeling to obtain the sub-modeling feature map of the i-th channel sub-feature map, i∈[2,M]. This includes at least one instance of identity mapping, local feature modeling, and global feature modeling. The sub-modeling feature maps of the M channel sub-feature maps are concatenated along the channel dimension to obtain the fused sub-modeling feature map of the spatial sub-feature map. The modeling feature map is obtained by splicing the N spatial sub-feature maps together in the spatial dimension to form a fused sub-modeling feature map.

13. An electronic device, characterized in that, The method includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program in the memory to implement the steps of the image feature extraction method according to any one of claims 1 to 11.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program adapted for execution by a processor to implement the steps of the image feature extraction method according to any one of claims 1 to 11.

15. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the image feature extraction method according to any one of claims 1 to 11.

Images