Cascaded strip-mixed multi-layer perceptron model image recognition method and related device
By using a cascaded strip hybrid multilayer perceptron model, image feature processing is performed through a cascaded structure of four network layers, which solves the problem of low efficiency in interaction between image blocks and improves the accuracy and efficiency of image recognition.
Patent Information
- Application Number
- CN202310466969.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-04-20
- Publication Date
- 2026-02-24
- Estimated Expiration
- 2043-04-20
AI Technical Summary
Existing multilayer perceptron models suffer from low efficiency in image patch interaction and poor aggregation performance.
A cascaded, bandgap hybrid multilayer perceptron model is adopted. Image features are mapped, information interacted, and features are aggregated through a cascaded structure of four network layers. This includes a first mapping operation, a first information interaction operation, and a first feature aggregation operation; a second convolution operation, a second information interaction operation, and a second feature aggregation operation; a third convolution operation, a third information interaction operation, and a third feature aggregation operation; and a fourth convolution operation and a fourth feature aggregation operation. Finally, pooling and the second mapping operation are performed.
It improves the interaction efficiency and aggregation effect between image patches in the multilayer perceptron model, thereby enhancing the model's expressive power and recognition accuracy.
Smart Images

Figure CN116630687B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of computer vision technology, and in particular to a method and related equipment for image recognition of a cascaded strip hybrid multilayer perceptron model. Background Technology
[0002] In the field of computer vision, Convolutional Neural Networks (CNNs) are the mainstream deep learning models, achieving a series of breakthroughs in algorithmic performance. Self-attention methods have achieved great success in natural language processing. Inspired by this, Transformer models have also been applied to the vision domain, achieving performance comparable to CNNs. Although ViT (Vision Transformer) models and their related methods generally outperform traditional CNNs, recent research shows that self-attention is not essential in vision algorithms, and attention-free models can achieve similar or even better performance. In recent years, Multi-Layer Perceptron (MLP) models, such as MLP-Mixer, MAXIM, and gMLP, have constructed attention-free deep learning models using only MLP network layers, demonstrating excellent model potential.
[0003] In visual models, both global and local information significantly impact recognition accuracy. CNNs (Convolutional Neural Networks) aggregate local information by sharing kernel weights and performing sliding convolutions on the image using a series of kernels. However, limited by kernel size, this method cannot capture long-range image information. Transformer models obtain global image information through self-attention modules, but this introduces substantial computational complexity. In contrast, MLP-based models, instead of using attention-based mechanisms, employ deeper MLP layers to capture global image information from both channel and spatial dimensions, demonstrating significant application potential.
[0004] The MLP-Mixer model proposes two types of MLP network layers: Channel-Mixing MLPs and Token-Mixing MLPs. Channel-Mixing MLPs are mainly used to mix the feature information of each image block (token) along the channel dimension, while Token-Mixing MLPs are mainly used to fuse the feature information of tokens at different spatial locations. The main problem with this method is that the number of model parameters is too large, which makes the model prone to overfitting. The Sparse MLP method introduces Sparse MLP Blocks, meaning that the MLP only processes along the axis of the image spatial dimension. At the same time, by sharing parameters, it effectively reduces the number of model parameters, improves the accuracy of the model, and reduces the risk of overfitting. However, this method also brings some drawbacks. On the one hand, the model uses a parallel method for information interaction between tokens, which reduces the efficiency of token interaction. Specifically, for a token to interact with all other tokens in the two-dimensional space of the image, it must be achieved through two Sparse MLP Blocks. On the other hand, Sparse MLP Blocks are prone to losing information of adjacent tokens, resulting in poor aggregation of image information of locally adjacent tokens. More importantly, the expressive power of token-mixing MLPs is related to the spatial resolution of features. The smaller the spatial resolution of a feature, the weaker the expressive power of the corresponding token-mixing model will be, which greatly limits the performance of the model.
[0005] Therefore, existing technologies still need to be improved and enhanced. Summary of the Invention
[0006] The main objective of this invention is to provide a method and related equipment for image recognition of a cascaded, strip-like hybrid multilayer perceptron model, aiming to solve the problems of low interaction efficiency and poor aggregation effect between image blocks in the existing multilayer perceptron model.
[0007] To achieve the above objectives, the present invention adopts the following technical solution:
[0008] A method for image recognition of a cascaded strip-hybrid multilayer perceptron model, comprising:
[0009] The segmented original image is input into the first network layer of the cascaded banded hybrid multilayer perceptron model, and the first mapping operation, the first information interaction operation, and the first feature aggregation operation are performed in sequence to obtain the first output feature.
[0010] The first output feature is input into the second network layer of the multilayer perceptron model, and the first convolution operation, the second information interaction operation, and the second feature aggregation operation are performed sequentially to obtain the second output feature.
[0011] The second output feature is input into the third network layer of the multilayer perceptron model, and the second convolution operation, the third information interaction operation, and the third feature aggregation operation are performed sequentially to obtain the third output feature.
[0012] The third output feature is input into the fourth network layer of the multilayer perceptron model, and the third convolution operation, the fourth information interaction operation, and the fourth feature aggregation operation are performed in sequence to obtain the fourth output feature.
[0013] The fourth output feature is subjected to pooling and second mapping operations to obtain the image classification result.
[0014] In the image recognition method using the cascaded strip-mixed multilayer perceptron model, the step of inputting the segmented original image into the first network layer of the cascaded strip-mixed multilayer perceptron model, and sequentially performing a first mapping operation, a first information interaction operation, and a first feature aggregation operation to obtain a first output feature, specifically includes:
[0015] The original image in the first dimension is segmented to obtain multiple image blocks; the first linear module in the first network layer is used to perform the first mapping operation on all the image blocks to obtain the image features in the second dimension.
[0016] In the width direction, the first cascaded bandgap mixing module in the first network layer is used to sequentially perform the first first dimension permutation, stacking, the second first dimension permutation, restoration, cascade and fusion operations on the second dimension image features to obtain the first fused image features.
[0017] In the height dimension, the first cascaded strip mixing module is used to sequentially perform the first second-dimensional permutation, stacking, second second-dimensional permutation, restoration, cascade and fusion operations on the first fused image features to obtain the second fused image features;
[0018] The first feature is obtained by using the first channel mixing module in the first network layer to perform a first feature aggregation operation on the second fused image features.
[0019] In the cascaded bandgap hybrid multilayer perceptron model image recognition method, the step of inputting the first output feature into the second network layer of the multilayer perceptron model, and sequentially performing the first convolution operation, the second information interaction operation, and the second feature aggregation operation to obtain the second output feature specifically includes:
[0020] The first image patch fusion module in the second network layer is used to perform the first convolution operation on the first output feature to obtain the third dimension image feature;
[0021] In the width dimension, the third dimension image features are sequentially subjected to the first dimension permutation, stacking, the second dimension permutation, restoration, cascading and fusion operations using the second cascaded bandgap mixing module in the second network layer to obtain the third fused image features.
[0022] In the height dimension, the second cascaded banded mixing module is used to sequentially perform the first second-dimensional permutation, stacking, the second second-dimensional permutation, restoration, cascade and fusion operations on the third fused image features to obtain the fourth fused image features;
[0023] The second feature is obtained by using the second channel mixing module in the second network layer to perform a second feature aggregation operation on the fourth fused image features.
[0024] In the cascaded bandgap hybrid multilayer perceptron model image recognition method, the step of inputting the second output feature into the third network layer of the multilayer perceptron model, and sequentially performing the second convolution operation, the third information interaction operation, and the third feature aggregation operation to obtain the third output feature specifically includes:
[0025] The second image patch fusion module in the third network layer is used to perform the second convolution operation on the second output feature to obtain the fourth dimension image feature;
[0026] In the width direction, the third cascaded banded mixing module in the third network layer is used to sequentially perform the first first dimension permutation, stacking, the second first dimension permutation, restoration, cascade and fusion operations on the fourth dimension image features to obtain the fifth fused image features.
[0027] In the height dimension, the third cascaded banded mixing module is used to sequentially perform the first second-dimensional permutation, stacking, the second second-dimensional permutation, restoration, cascade and fusion operations on the fifth fused image feature to obtain the sixth fused image feature;
[0028] The third feature is obtained by using the third channel mixing module in the third network layer to perform a third feature aggregation operation on the sixth fused image features.
[0029] In the cascaded bandgap hybrid multilayer perceptron model image recognition method, the step of inputting the third output feature into the fourth network layer of the multilayer perceptron model, and sequentially performing the third convolution operation, the fourth information interaction operation, and the fourth feature aggregation operation to obtain the fourth output feature specifically includes:
[0030] The third image patch fusion module in the fourth network layer is used to perform the third convolution operation on the third output feature to obtain the fifth dimension image feature;
[0031] In the width direction, the fourth cascaded banded mixing module in the fourth network layer is used to sequentially perform the first first dimension permutation, stacking, the second first dimension permutation, restoration, cascade and fusion operations on the fifth dimension image features to obtain the seventh fused image features.
[0032] In the height dimension, the fourth cascaded banded mixing module is used to sequentially perform the first second-dimensional permutation, stacking, the second second-dimensional permutation, restoration, cascade and fusion operations on the seventh fused image feature to obtain the eighth fused image feature;
[0033] The fourth feature is obtained by using the fourth channel mixing module in the fourth network layer to perform a fourth feature aggregation operation on the eighth fused image features.
[0034] In the cascaded bandgap hybrid multilayer perceptron model image recognition method, the first dimension is H×W×3; the second dimension is... Where H represents the number of pixels in the vertical direction, W represents the number of pixels in the horizontal direction, C represents the number of channels, p represents the length of the image block, and q represents the width of the image block.
[0035] In the cascaded banded hybrid multilayer perceptron model image recognition method, the third dimension is... The fourth dimension is The fifth dimension is Where H represents the number of pixels in the vertical direction, W represents the number of pixels in the horizontal direction, C represents the number of channels, p represents the length of the image block, and q represents the width of the image block.
[0036] A cascaded strip hybrid multilayer perceptron model image recognition system, the cascaded strip hybrid multilayer perceptron model image recognition system comprising:
[0037] The first output feature acquisition module is used to input the segmented original image into the first network layer of the cascaded banded hybrid multilayer perceptron model, and sequentially perform the first mapping operation, the first information interaction operation, and the first feature aggregation operation to obtain the first output feature.
[0038] The second output feature acquisition module is used to input the first output feature into the second network layer in the multilayer perceptron model, and sequentially perform the first convolution operation, the second information interaction operation, and the second feature aggregation operation to obtain the second output feature.
[0039] The third output feature acquisition module is used to input the second output feature into the third network layer in the multilayer perceptron model, and sequentially perform the second convolution operation, the third information interaction operation and the third feature aggregation operation to obtain the third output feature.
[0040] The fourth output feature acquisition module is used to input the third output feature into the fourth network layer in the multilayer perceptron model, and sequentially perform the third convolution operation, the fourth information interaction operation, and the fourth feature aggregation operation to obtain the fourth output feature.
[0041] The image classification result acquisition module is used to perform pooling and second mapping operations on the fourth output feature to obtain the image classification result.
[0042] A smart terminal includes: a memory, a processor, a display, and a cascaded striped hybrid multilayer perceptron model image recognition program stored in the memory and executable on the processor. When executed by the processor, the cascaded striped hybrid multilayer perceptron model image recognition program implements the steps of the cascaded striped hybrid multilayer perceptron model image recognition method as described above.
[0043] A computer-readable storage medium stores a cascaded striped hybrid multilayer perceptron model image recognition program, which, when executed by a processor, implements the steps of the cascaded striped hybrid multilayer perceptron model image recognition method as described above.
[0044] Compared to existing technologies, this invention provides a cascaded, banded hybrid multilayer perceptron model image recognition method and related equipment. The method includes: inputting the segmented original image into a first network layer, performing a first mapping, a first information interaction, and a first feature aggregation operation sequentially to obtain a first output feature; inputting the first output feature into a second network layer, performing a first convolution, a second information interaction, and a second feature aggregation operation sequentially to obtain a second output feature; inputting the second output feature into a third network layer, performing a second convolution, a third information interaction, and a third feature aggregation operation sequentially to obtain a third output feature; inputting the third output feature into a fourth network layer, performing a third convolution, a fourth information interaction, and a fourth feature aggregation operation sequentially to obtain a fourth output feature; and performing pooling and a second mapping operation on the fourth output feature to obtain an image classification result, thereby improving the interaction efficiency and aggregation effect between image blocks in the multilayer perceptron model. Attached Figure Description
[0045] Figure 1 A flowchart illustrating a preferred embodiment of the cascaded strip hybrid multilayer perceptron model image recognition method provided by the present invention;
[0046] Figure 2 A schematic diagram of the framework of a multilayer perceptron model based on a cascaded strip hybrid model, which is a preferred embodiment of the image recognition method of the cascaded strip hybrid multilayer perceptron model provided by the present invention.
[0047] Figure 3 A flowchart of step S100 in a preferred embodiment of the cascaded strip hybrid multilayer perceptron model image recognition method provided by the present invention;
[0048] Figure 4 A structural diagram of the cascaded strip hybrid module of the Cast-MLP network in a preferred embodiment of the image recognition method of the cascaded strip hybrid multilayer perceptron model provided by the present invention;
[0049] Figure 5 A schematic diagram of the channel mixing structure in a preferred embodiment of the cascaded strip-mixed multilayer perceptron model image recognition method provided by the present invention;
[0050] Figure 6 A schematic diagram comparing the Strip MLP structure with the traditional MLP structure and the Sparse MLP structure in a preferred embodiment of the cascaded strip hybrid multilayer perceptron model image recognition method provided by the present invention.
[0051] Figure 7 A flowchart of step S200 in a preferred embodiment of the cascaded strip hybrid multilayer perceptron model image recognition method provided by the present invention;
[0052] Figure 8 The principle block diagram and flowchart of step S300 of the cascaded strip hybrid multilayer perceptron model image recognition system provided by the present invention are shown below.
[0053] Figure 9 A flowchart of step S400 in a preferred embodiment of the cascaded strip hybrid multilayer perceptron model image recognition method provided by the present invention;
[0054] Figure 10 The architecture diagram of the cascaded strip hybrid multilayer perceptron model image recognition system provided by the present invention;
[0055] Figure 11 This is a schematic diagram of the operating environment of a preferred embodiment of the smart terminal provided by the present invention. Detailed Implementation
[0056] To make the objectives, technical solutions, and effects of this invention clearer and more explicit, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
[0057] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the term “comprising” as used in this specification means the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. It should be understood that when we say an element is “connected” or “coupled” to another element, it can be directly connected or coupled to the other element, or there may be intermediate elements. Furthermore, “connected” or “coupled” as used herein can include wireless connections or wireless coupling. The term “and / or” as used herein includes all or any units and all combinations of one or more associated listed items.
[0058] It will be understood by those skilled in the art that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. It should also be understood that terms such as those defined in general dictionaries should be understood to have the same meaning as in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless specifically defined as herein.
[0059] This invention provides an image recognition method and related equipment using a Cascade Strip Mixing MLP (Cast-MLP) multilayer perceptron model. In the proposed Cascade Strip Mixing MLP network model, a novel Cascade Strip Mixing module (Strip MLP) is introduced. This module significantly improves the interaction efficiency between tokens through its cascaded structure, allowing adjacent tokens to interact at a strip scale, i.e., simultaneously mixing information over both short and long ranges. In this invention, the segmented original image is sequentially input into the first, second, third, and fourth network layers of the Cascade Strip Mixing MLP model, followed by pooling and second mapping operations to obtain the image classification result. This accelerates the interaction efficiency between image patches in the MLP model and improves the aggregation effect between image patches.
[0060] The following describes the design scheme of the image recognition method for the cascaded strip hybrid multilayer perceptron model through specific exemplary embodiments. It should be noted that the following embodiments are only used to explain the technical solution of the invention and are not intended to limit it specifically:
[0061] Please see Figure 1This invention provides a method for image recognition of a cascaded strip-hybrid multilayer perceptron model, the method comprising:
[0062] S100. The original image after segmentation is input into the first network layer of the cascaded banded hybrid multilayer perceptron model, and the first mapping operation, the first information interaction operation, and the first feature aggregation operation are performed in sequence to obtain the first output feature.
[0063] The image recognition method based on the CascadeStrip Mixing MLP model is implemented on the framework of the CascadeStrip Mixing MLP model. A schematic diagram of the framework of the CascadeStrip Mixing MLP model (Cast-MLP) is shown below. Figure 2 As shown, this is a multi-stage network structure, divided into four network layers: the first network layer (corresponding to...) Figure 2 In the first stage (stage 1), the second network layer (corresponding to...) Figure 2 In the second stage of the network, the third network layer (corresponding to...) Figure 2 Stage 3 and the fourth network layer (corresponding to) Figure 2 In stage 4, the network structures of the second, third, and fourth network layers are all the same, so the operations performed on their respective input features are all the same.
[0064] Specifically, the cascaded strip hybrid multilayer perceptron model image recognition method is implemented as follows:
[0065] First, given the original image X, the original image X is uniformly segmented to obtain the segmented original image. Then, proceed to stage one: input the segmented original image into the first network layer (corresponding to) of the cascaded bandgap hybrid multilayer perceptron model. Figure 2 In stage 1), the first network layer sequentially performs the first mapping operation, the first information interaction operation, and the first feature aggregation operation on the segmented original image. After repeating this process for T1 cycles in the first stage, the first output feature is obtained. In order to make the first output feature It is used as input to the second network layer for further processing.
[0066] Furthermore, please refer to Figure 3 In step S100, the segmented original image is input into the first network layer of the cascaded bandgap hybrid multilayer perceptron model, and the first mapping operation, the first information interaction operation, and the first feature aggregation operation are performed sequentially to obtain the first output feature, specifically including:
[0067] S110. The original image in the first dimension is segmented to obtain multiple image blocks;
[0068] S120. The first linear module in the first network layer is used to perform the first mapping operation on all the image blocks to obtain image features of the second dimension; wherein, the first dimension is H×W×3; the second dimension is... Where H represents the number of pixels in the vertical direction, W represents the number of pixels in the horizontal direction, C represents the number of channels, p represents the length of the image block, and q represents the width of the image block;
[0069] S130. In the width direction, the first cascaded bandgap mixing module in the first network layer is used to sequentially perform the first first dimension permutation, stacking, the second first dimension permutation, restoration, cascade and fusion operations on the image features of the second dimension to obtain the first fused image features.
[0070] S140. In the height dimension, the first cascaded strip mixing module is used to sequentially perform the first second-dimensional permutation, stacking, second second-dimensional permutation, restoration, cascade and fusion operations on the first fused image features to obtain the second fused image features.
[0071] S150. The first feature aggregation operation is performed on the second fused image features using the first channel mixing module in the first network layer to obtain the first output feature.
[0072] The first network layer includes: the first linear module (corresponding to Figure 2 In stage 1, the LinearEmbedding and the first cascaded bandgap hybrid module (corresponding to...) Figure 2 The Cascade Strip Mixing module in stage 1 and the first channel mixing module (corresponding to) Figure 2 Channel Mixing in Stage 1.
[0073] Specifically, after obtaining the original image X in the first dimension, the original image X is uniformly divided into multiple image patches; wherein, the dimension (first dimension) of the original image X is H×W×3 (H represents the number of pixels in the vertical direction, W represents the number of pixels in the horizontal direction, and C represents the number of channels (for example, the number of channels C=1 for a black and white image, while the number of channels C=3 for an RGB color image)). Therefore, the original image X is uniformly divided into... There are 10 image patches, each patch being p×q in size.
[0074] Then, each patch is mapped to a higher dimension (second dimension) through a linear layer (first linear module), i.e. The second-dimensional image features are obtained. When the second-dimensional image features are processed by the first cascaded bandgap mixing module (corresponding to...) Figure 2 During Cascade Strip Mixing in stage 1, the Strip MLP in the first cascaded strip mixing module applies second-dimensional image features in both the width and height directions:
[0075] like Figure 4 The diagram shows the structure of a cascaded stripe hybrid module (comprising a first, second, third, and fourth cascaded stripe hybrid module, all with the same structure) of a Cast-MLP network. This is in contrast to the structure of a Strip MLP (corresponding to...) Figure 4 The small squares in the grid are applied in the width direction: given input features This refers to the second-dimensional image features at this point. Then, the second-dimensional image features are sequentially subjected to a first-dimensional permutation, stacking, a second-dimensional permutation, restoration, concatenation, and fusion operations to obtain the first fused image features. This involves performing feature dimension permutation (the first-dimensional permutation), resulting in... The image features are divided into G groups along the channel dimension, i.e. The features of each group are stacked along the width direction, i.e.
[0076] However, due to this operation, the image features will have a large dimensionality in the width direction. To avoid excessive model computation and parameter count, this invention proposes a Group Strip MLP layer (corresponding to...). Figure 4 Input features From dimension permutation to restoration and all steps after dimension permutation, the data is divided into S segments along the width direction (S needs to be divisible by GW). Then, Strip MLP is applied to each segment of feature data, with weights not sharing parameters between segments. Finally, the image features are dimension permuted (a second first-dimensional permutation) to restore them to the same dimension as the input features, thus obtaining the image features. To further preserve image location information, the obtained image features are compared with the input features. Cascaded, and through a fully connected layer (corresponding to Figure 4 Image fusion is performed using Channel FC in the code to obtain the fused image features. (This is the first fused image feature).
[0077] Similarly, when Strip MLP is applied to the height direction, the first cascaded strip mixing module sequentially performs a first second-dimensional permutation, stacking, a second second-dimensional permutation, restoration, cascading, and fusion operation on the first fused image features to obtain the second fused image features. This step is based on... As input, this forms a cascaded model with the previous step. The feature processing method in this step is similar to that of Strip MLP applied to the width direction, except that the feature dimension permutation is performed along the height direction, ultimately yielding the output features. (Second fused image features).
[0078] Finally, utilizing the first channel mixing module (corresponding to) in the first network layer Figure 2 In stage 1, Channel-Mixing performs a first feature aggregation operation on the second fused image features to obtain the first output features.
[0079] Channel-Mixing is primarily used for aggregating information along the channel dimension, and its structure is as follows: Figure 5 As shown in the figure. In this invention, by designing a Strip MLP and Cascade Strip Mixing structure, it is possible to achieve information interaction between the current token and other tokens in two-dimensional space using only a single module, thereby improving the efficiency of token interaction.
[0080] However, the Cascade Strip Mixing structure in the Cast-MLP structure of this invention is based on a novel Strip MLP structure. A comparative diagram of the Strip MLP structure proposed for the first time in this invention with traditional MLP and Sparse MLP structures is shown below. Figure 6 As shown. In this invention, the Strip MLP structure applies MLP (MultiLayer Perceptron) to the current axis of the token and the two adjacent axes, which can effectively fuse the information of tokens on adjacent axes, simultaneously acquire short-distance and long-distance information, and improve the expressive power of the model, enabling tokens to interact with information at the "Strip Scale".
[0081] Traditional MLP methods involve all information exchange between the current token and other tokens, resulting in massive computational costs, large model parameters, and overfitting. Sparse MLP methods ignore token exchanges along adjacent axes, easily losing information from neighboring tokens and reducing the efficiency of token exchange. The Cascade StripMixing structure, however, enables more efficient information exchange between tokens and further enhances the model's expressive power.
[0082] For further information, please refer to [link / reference]. Figure 1 S200, The first output feature is input into the second network layer of the multilayer perceptron model, and the first convolution operation, the second information interaction operation, and the second feature aggregation operation are performed in sequence to obtain the second output feature.
[0083] Specifically, after obtaining the first output feature, the process proceeds to stage two:
[0084] The first output feature The input is fed into the second network layer (corresponding to) in the multilayer perceptron model. Figure 2 In stage 2), the first network layer is used to process the first output features. The first convolution operation, the second information interaction operation, and the second feature aggregation operation are performed sequentially. After repeating this process for T2 cycles in the second stage, the second output feature is obtained. In order to make the second output feature It is used as input to the third network layer for further processing.
[0085] Furthermore, please refer to Figure 7 In step S200, the first output feature is input into the second network layer of the multilayer perceptron model, and the first convolution operation, the second information interaction operation, and the second feature aggregation operation are performed sequentially to obtain the second output feature, specifically including:
[0086] S210. The first image patch fusion module in the second network layer is used to perform the first convolution operation on the first output feature to obtain the third-dimensional image feature; wherein, the third dimension is
[0087] S220. In the width direction, the third dimension image features are subjected to the first first dimension permutation, stacking, the second first dimension permutation, restoration, cascading and fusion operations in sequence using the second cascaded bandgap mixing module in the second network layer to obtain the third fused image features.
[0088] S230. In the height dimension, the second cascaded strip mixing module is used to sequentially perform the first second-dimensional permutation, stacking, the second second-dimensional permutation, restoration, cascade and fusion operations on the third fused image features to obtain the fourth fused image features.
[0089] S240. The second feature aggregation operation is performed on the fourth fused image features using the second channel mixing module in the second network layer to obtain the second output feature.
[0090] The second network layer includes: the first image patch fusion module (corresponding to...) Figure 2 In stage 2, Patch Merging), the second cascaded bandgap mixing module (corresponding to) Figure 2 The Cascade StripMixing module in stage 2 and the second channel mixing module (corresponding to) Figure 2 Channel Mixing in Stage 2.
[0091] Specifically, after obtaining the first output feature, the process proceeds to stage two:
[0092] After obtaining the first output feature, the first output feature is input into the second network layer, and the first image patch fusion module in the second network layer (corresponding to...) is used. Figure 2 In stage 2, Patch Merging) is applied to the first output feature. Perform the first convolution operation to obtain the third dimension. The image features, that is, the first image patch fusion module reduces the resolution of the image features by a factor of 2 through convolution operations, while increasing the number of image channels to twice that of the input channels, i.e., by Become
[0093] Similarly, when the third-dimensional image features pass through the second cascaded strip mixing module, the Strip MLP in the second cascaded strip mixing module applies the third-dimensional image features in both the width and height directions:
[0094] Applying Strip MLP in the width direction: Given input features That is, the third-dimensional image features at this point. Then, the third-dimensional image features are subjected to the following operations in sequence: first-dimensional permutation, stacking, second-dimensional permutation, restoration, concatenation, and fusion, to obtain the third fused image features. That is, feature dimension permutation (first-dimensional permutation) is performed, resulting in... The image features are divided into G groups along the channel dimension, i.e. The features of each group are stacked along the width direction, i.e.
[0095] However, this operation results in a large dimensionality of image features in the width direction. To avoid excessive model computation and parameter count, this invention proposes a grouped strip MLP layer. The data is divided into S segments along the width direction. Then, a Strip MLP is applied to each segment of feature data, with weights not sharing parameters between segments. Finally, the image features are dimensionally permuted (a second first-dimensional permutation) to restore them to the same dimension as the input features, thus obtaining the image features. To further preserve image location information, the obtained image features are compared with the input features. The images are cascaded and fused using fully connected layers to obtain the fused image features. (This is the third fused image feature).
[0096] Similarly, when Strip MLP is applied to the height direction, the second cascaded strip mixing module sequentially performs a first second-dimensional permutation, stacking, a second second-dimensional permutation, restoration, cascading, and fusion operation on the third fused image features to obtain the fourth fused image features. This step is based on... As input, this forms a cascaded model with the previous step. The feature processing method in this step is similar to that of Strip MLP applied to the width direction, except that the feature dimension permutation is performed along the height direction, ultimately yielding the output features. (Fourth fused image features).
[0097] Finally, the second channel mixing module in the second network layer (corresponding to) Figure 2 In stage 2, Channel-Mixing performs a second feature aggregation operation on the fourth fused image features to obtain the second output features.
[0098] For further information, please refer to [link / reference]. Figure 1 S300, The second output feature is input into the third network layer of the multilayer perceptron model, and the second convolution operation, the third information interaction operation and the third feature aggregation operation are performed in sequence to obtain the third output feature.
[0099] Specifically, after obtaining the second output feature, the process proceeds to stage three:
[0100] The second output feature The input is given to the third network layer (corresponding to) in the multilayer perceptron model. Figure 2In stage 3), the third network layer is used to process the second output feature. The second convolution operation, the third information interaction operation, and the third feature aggregation operation are performed sequentially. After repeating this process for T3 cycles in the third stage, the third output feature is obtained. In order to make the third output feature It is used as input to the fourth network layer for further processing.
[0101] Furthermore, please refer to Figure 8 In step S300, the second output feature is input into the third network layer of the multilayer perceptron model, and the second convolution operation, the third information interaction operation, and the third feature aggregation operation are performed sequentially to obtain the third output feature, specifically including:
[0102] S310. The second image patch fusion module in the third network layer is used to perform the second convolution operation on the second output features to obtain the fourth-dimensional image features; wherein, the fourth dimension is
[0103] S320. In the width direction, the third cascaded banded mixing module in the third network layer is used to sequentially perform the first first dimension permutation, stacking, the second first dimension permutation, restoration, cascade and fusion operations on the image features of the fourth dimension to obtain the fifth fused image features.
[0104] S330. In the height dimension, the third cascaded banded mixing module is used to sequentially perform the first second-dimensional permutation, stacking, the second second-dimensional permutation, restoration, cascade and fusion operations on the fifth fused image feature to obtain the sixth fused image feature.
[0105] S340. The third channel mixing module in the third network layer is used to perform a third feature aggregation operation on the sixth fused image features to obtain the third output feature.
[0106] The third network layer includes: the second image patch fusion module (corresponding to...) Figure 2 In stage 3, Patch Merging), the third cascaded bandgap mixing module (corresponding to Figure 2 The Cascade StripMixing in stage 3 and the third channel mixing module (corresponding to) Figure 2 Channel Mixing in stage 3 (middle stage).
[0107] Specifically, after obtaining the second output feature, the process proceeds to stage three:
[0108] First, the second image patch fusion module in the third network layer (corresponding to) is used. Figure 2 In stage 3, Patch Merging performs the second convolution operation on the second output feature to obtain the fourth dimension. Image features.
[0109] Then, in the width dimension, the third cascaded bandgap hybrid module (corresponding to) in the third network layer is utilized. Figure 2 In stage 3, Cascade Strip Mixing sequentially performs the first first-dimensional permutation, stacking, the second first-dimensional permutation, restoration, cascading, and fusion operations on the fourth-dimensional image features to obtain the fifth fused image features.
[0110] Furthermore, in the height dimension, the third cascaded banded mixing module is used to sequentially perform a first second-dimensional permutation, stacking, a second second-dimensional permutation, restoration, cascading, and fusion operation on the fifth fused image feature to obtain the sixth fused image feature; finally, the third channel mixing module in the third network layer (corresponding to...) is used... Figure 2 In stage 3, Channel Mixing performs the third feature aggregation operation on the sixth fused image features to obtain the third output feature.
[0111] For further information, please refer to [link / reference]. Figure 1 S400, The third output feature is input into the fourth network layer in the multilayer perceptron model, and the third convolution operation, the fourth information interaction operation and the fourth feature aggregation operation are performed in sequence to obtain the fourth output feature.
[0112] Specifically, after obtaining the third output feature, the process proceeds to stage four:
[0113] The third output feature The input is given to the fourth network layer (corresponding to) in the multilayer perceptron model. Figure 2 In stage 4), the third output feature is processed using the fourth network layer. The third convolution operation, the fourth information interaction operation, and the fourth feature aggregation operation are performed sequentially. After repeating the fourth stage for T4 cycles, the fourth output feature is obtained. In order to provide the fourth output feature The image classification result is obtained by performing pooling and second mapping operations.
[0114] Furthermore, please refer to Figure 9In step S400, the third output feature is input into the fourth network layer of the multilayer perceptron model, and the third convolution operation, the fourth information interaction operation, and the fourth feature aggregation operation are performed sequentially to obtain the fourth output feature, specifically including:
[0115] S410. The third image patch fusion module in the fourth network layer is used to perform the third convolution operation on the third output feature to obtain the fifth-dimensional image feature; wherein, the fifth dimension is...
[0116] S420. In the width direction, the fourth cascaded banded mixing module in the fourth network layer is used to sequentially perform the first first dimension permutation, stacking, the second first dimension permutation, restoration, cascade and fusion operations on the fifth dimension image features to obtain the seventh fused image features.
[0117] S430. In the height dimension, the fourth cascaded banded mixing module is used to sequentially perform the first second-dimensional permutation, stacking, the second second-dimensional permutation, restoration, cascade and fusion operations on the seventh fused image feature to obtain the eighth fused image feature.
[0118] S440. The fourth feature aggregation operation is performed on the eighth fused image features using the fourth channel mixing module in the fourth network layer to obtain the fourth output feature.
[0119] Specifically, after obtaining the third output feature, the process proceeds to stage four:
[0120] First, the third image patch fusion module in the fourth network layer (corresponding to) is used. Figure 2 In stage 4, Patch Merging) is used for the third output feature. Performing the third convolution operation yields the fifth dimension. Image features.
[0121] Then, in the width dimension, the third cascaded bandgap hybrid module (corresponding to) in the third network layer is utilized. Figure 2 In stage 3, Cascade Strip Mixing sequentially performs the first first-dimensional permutation, stacking, the second first-dimensional permutation, restoration, concatenation, and fusion operations on the fourth-dimensional image features to obtain the seventh fused image features.
[0122] Furthermore, in the height dimension, the third cascaded banded mixing module is used to sequentially perform a first second-dimensional permutation, stacking, a second second-dimensional permutation, restoration, cascading, and fusion operation on the seventh fused image features to obtain the eighth fused image features; finally, the third channel mixing module in the third network layer (corresponding to...) is used... Figure 2 In stage 3, Channel Mixing performs a third feature aggregation operation on the eighth fused image features to obtain the fourth output feature.
[0123] For further information, please refer to [link / reference]. Figure 1 S500, perform pooling and second mapping operations on the fourth output feature to obtain the image classification result.
[0124] Specifically, after obtaining the fourth output feature Then, for the fourth output feature Perform the pooling operation and the second mapping operation, i.e., the fourth output feature. After Global Average Pooling (GAP) and linear layer processing, the final classification result is obtained: the image classification result.
[0125] In this invention, the segmented original image is input into the first, second, third, and fourth network layers of a cascaded, strip-mixed multilayer perceptron model for mapping, convolution, information interaction, and feature aggregation operations, ultimately yielding the image classification result. This cascaded structure significantly improves the interaction efficiency between tokens, enabling adjacent tokens to interact at a strip scale, thus allowing for information mixing across both short and long ranges simultaneously.
[0126] Next, the process of demonstrating the difference between Cast-MLP in this invention and existing CNN-based, Transformer-based, and MLP-based models using invention examples and experimental results is as follows:
[0127] Model Configuration: To compare the method of this invention with existing methods, four model variants were designed: Cast-MLP-T*, Cast-MLP-T, Cast-MLP-S, and Cast-MLP-B. The number of parameters and computational complexity of each model gradually increase, and the model size is similar to or close to that of publicly available methods such as Swin Transformer. The parameters of each model structure are as follows:
[0128] Cast-MLP-T*: C=80, {T1~T4}={2, 2, 6, 2};
[0129] Cast-MLP-T: C=80, {T1~T4}={2, 2, 16, 2};
[0130] Cast-MLP-S: C=96, {T1~T4}={2, 2, 18, 2};
[0131] Cast-MLP-B: C=128, {T1~T4}={2, 2, 18, 2};
[0132] This invention compares Cast-MLP with CNN-based models, Transformer-based models, and MLP-based models. Three different publicly available datasets of varying sizes were used, and the datasets are described below:
[0133] Caltech-101: Contains 101 categories and a total of 9k images. For each category, 80% of the images are randomly selected as the training set, and the remaining images are used as the test set.
[0134] CIFAR-100: Contains 100 categories and a total of 60k images, of which 50k are used for model training and 10k are used for model testing;
[0135] ImageNet-1K: Contains 1000 categories, with 1.28M training images and 50k test images;
[0136] The experimental results of Cast-MLP on the Caltech-101 dataset are shown in Table 1. The method of this invention achieves superior experimental results: compared to CNN models, Cast-MLP-B's Top-1 Accuracy is 3.39% higher than the ResNet152 model, and on average, it is 3.09% higher than the ResNet model across three different model sizes. Compared to Transformer models, the method of this invention outperforms each Swin Transformer model by 11.13% / 12.69% / 14.01%, while the ViT-B / 16 model only achieves 53.96% accuracy. Compared to MLP models, the model of this invention outperforms Cast-MLP-T* by 3.03% and Wave-MLP-T by 3.89%, and Cast-MLP-S outperforms Hire-MLP-B by 3.89%. These experiments demonstrate that the method of this invention has significant advantages and competitiveness on small datasets.
[0137]
[0138] Table 1
[0139] The experimental results of Cast-MLP on the Cifar-100 dataset are shown in Table 2. Its Top-1 accuracy is on average 0.6%–1.39% higher than CNN models, and it outperforms the Swin-B model by 8.19% and MLP-based models by an average of 0.23%–1.59%. The experiments demonstrate that this invention has a significant advantage in model accuracy on medium-sized datasets.
[0140]
[0141] Table 2
[0142] The experimental results of Cast-MLP on the ImageNet-1k dataset are shown in Table 3. The method of this invention achieved an accuracy of 80.5%, with only 19M model parameters, and the accuracy was 1.6% higher than CycleMLP-B1. This shows that the performance of this invention on large datasets is comparable to or better than that of state-of-the-art models.
[0143] Table 4 shows the transfer learning experimental results of Cast-MLP on CIFAR-10 and CIFAR-100, representing the transfer learning results of Cast-MLP-T* on CIFAR-10 and CIFAR-100 based on the ImageNet-1 pre-trained model. It can be seen that the transfer learning accuracy of the method of this invention is 0.6% and 0.5% higher than other methods on CIFAR-10 and CIFAR-100, respectively, while the model has only 18M parameters, further demonstrating that the method of this invention is more efficient and has superior performance.
[0144]
[0145] Table 3
[0146]
[0147] Table 4
[0148] In summary, the Group Strip MLP network layer proposed in this invention can effectively enhance the expressive power of the model. Experiments on public datasets of various sizes demonstrate that the method of this invention has significant performance advantages. On the Caltech-101 dataset, the Cast-MLP method achieves a Top-1 accuracy of 92.51% (Top-1 accuracy refers to the accuracy of predicting the class with the highest probability ranking in line with the actual result), which is 14.09% higher than the Swin-B model; on the ImageNet-1K dataset, the small model of Cast-MLP (19M model parameters, computational complexity of 2.8G FLOPs) achieves an accuracy of 80.5%, which is 1.6% higher than the CycleMLP-B1 model.
[0149] Further, please refer to Figure 10 This invention provides a cascaded strip hybrid multilayer perceptron model image recognition system, the cascaded strip hybrid multilayer perceptron model image recognition system comprising:
[0150] The first output feature acquisition module 1 is used to input the segmented original image into the first network layer of the cascaded banded hybrid multilayer perceptron model, and sequentially perform the first mapping operation, the first information interaction operation and the first feature aggregation operation to obtain the first output feature;
[0151] The second output feature acquisition module 2 is used to input the first output feature into the second network layer in the multilayer perceptron model, and sequentially perform the first convolution operation, the second information interaction operation, and the second feature aggregation operation to obtain the second output feature.
[0152] The third output feature acquisition module 3 is used to input the second output feature into the third network layer in the multilayer perceptron model, and sequentially perform the second convolution operation, the third information interaction operation and the third feature aggregation operation to obtain the third output feature.
[0153] The fourth output feature acquisition module 4 is used to input the third output feature into the fourth network layer in the multilayer perceptron model, and sequentially perform the third convolution operation, the fourth information interaction operation and the fourth feature aggregation operation to obtain the fourth output feature;
[0154] The image classification result acquisition module 5 is used to perform pooling and second mapping operations on the fourth output feature to obtain the image classification result.
[0155] Specifically, the cascaded strip-mixed multilayer perceptron model image recognition system is used to implement the cascaded strip-mixed multilayer perceptron model image recognition method. The cascaded strip-mixed multilayer perceptron model image recognition method is implemented as follows:
[0156] First, given the original image X, the original image X is uniformly segmented to obtain the segmented original image. Entering Stage 1: The segmented original image is input into the first network layer (corresponding to) of the cascaded bandgap hybrid multilayer perceptron model. Figure 2 In stage 1), the first network layer sequentially performs the first mapping operation, the first information interaction operation, and the first feature aggregation operation on the segmented original image. After repeating this process for T1 cycles in the first stage, the first output feature is obtained.
[0157] Then, we move on to Phase Two:
[0158] The first output feature The input is fed into the second network layer (corresponding to) in the multilayer perceptron model. Figure 2 In stage 2), the first network layer is used to process the first output features. The first convolution operation, the second information interaction operation, and the second feature aggregation operation are performed sequentially. After repeating this process for T2 cycles in the second stage, the second output feature is obtained.
[0159] Secondly, we move on to stage three:
[0160] The second output feature The input is given to the third network layer (corresponding to) in the multilayer perceptron model. Figure 2 In stage 3), the third network layer is used to process the second output feature. The second convolution operation, the third information interaction operation, and the third feature aggregation operation are performed sequentially. After repeating this process for T3 cycles in the third stage, the third output feature is obtained.
[0161] Furthermore, we move on to stage four:
[0162] The third output feature The input is given to the fourth network layer (corresponding to) in the multilayer perceptron model. Figure 2 In stage 4), the third output feature is processed using the fourth network layer. The third convolution operation, the fourth information interaction operation, and the fourth feature aggregation operation are performed sequentially. After repeating the fourth stage for T4 cycles, the fourth output feature is obtained.
[0163] Finally, regarding the fourth output feature Perform the pooling operation and the second mapping operation, i.e., the fourth output feature. After Global Average Pooling (GAP) and linear layer processing, the final classification result is obtained: the image classification result.
[0164] like Figure 11 As shown, based on the above-mentioned cascaded strip hybrid multilayer perceptron model image recognition method, the present invention also provides a smart terminal, which includes a processor 10, a memory 20 and a display 30. Figure 11 Only some components of the smart terminal are shown, but it should be understood that it is not required to implement all of the components shown, and more or fewer components may be implemented instead.
[0165] In some embodiments, the memory 20 may be an internal storage unit of the smart terminal, such as a hard disk or memory of the smart terminal. In other embodiments, the memory 20 may be an external storage device of the smart terminal, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the smart terminal. Further, the memory 20 may include both internal and external storage units of the smart terminal. The memory 20 is used to store application software and various types of data installed on the smart terminal, such as the program code installed on the smart terminal. The memory 20 can also be used to temporarily store data that has been output or will be output. In one embodiment, the memory 20 stores a cascaded striped hybrid multilayer perceptron model image recognition program 40, which can be executed by the processor 10 to implement the cascaded striped hybrid multilayer perceptron model image recognition method of this application.
[0166] In some embodiments, the processor 10 may be a central processing unit (CPU), a microprocessor, or other data processing chip, used to run program code stored in the memory 20 or process data, such as executing the cascaded strip hybrid multilayer perceptron model image recognition method.
[0167] In some embodiments, the display 30 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen. The display 30 is used to display information on the smart terminal and to display a visual user interface. The components 10-30 of the smart terminal communicate with each other via a system bus.
[0168] In one embodiment, the steps implemented by the processor 10 when executing the cascaded strip hybrid multilayer perceptron model image recognition program 40 in the memory 20 can refer to the corresponding description in the above-described cascaded strip hybrid multilayer perceptron model image recognition method, and will not be repeated here.
[0169] Specifically, in this embodiment, the specific functions of each module of the above-mentioned cascaded strip hybrid multilayer perceptron model image recognition system based on polyhedral model modeling can be referred to the corresponding description in the above-mentioned cascaded strip hybrid multilayer perceptron model image recognition method, and will not be repeated here.
[0170] Furthermore, the present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores an automatic parallel strategy search program 40, which, when executed by a processor, implements the steps of the cascaded strip hybrid multilayer perceptron model image recognition method as described above; since the steps of the cascaded strip hybrid multilayer perceptron model image recognition method have been described in detail above, they will not be repeated here.
[0171] In summary, the present invention provides an image recognition method and related equipment using a cascaded, banded hybrid multilayer perceptron model. The method includes: inputting the segmented original image into a first network layer, performing a first mapping, a first information interaction, and a first feature aggregation operation sequentially to obtain a first output feature; inputting the first output feature into a second network layer, performing a first convolution, a second information interaction, and a second feature aggregation operation sequentially to obtain a second output feature; inputting the second output feature into a third network layer, performing a second convolution, a third information interaction, and a third feature aggregation operation sequentially to obtain a third output feature; inputting the third output feature into a fourth network layer, performing a third convolution, a fourth information interaction, and a fourth feature aggregation operation sequentially to obtain a fourth output feature; and performing pooling and a second mapping operation on the fourth output feature to obtain an image classification result, thereby improving the interaction efficiency and aggregation effect between image blocks in the multilayer perceptron model.
[0172] It is understood that those skilled in the art can make equivalent substitutions or modifications to the technical solution and inventive concept of the present invention, and all such substitutions or modifications should fall within the protection scope of the appended claims.
Claims
1. A method for image recognition using a cascaded, strip-hybrid multilayer perceptron model, characterized in that, The cascaded strip hybrid multilayer perceptron model image recognition method includes: The segmented original image is input into the first network layer of the cascaded banded hybrid multilayer perceptron model, and the first mapping operation, the first information interaction operation, and the first feature aggregation operation are performed in sequence to obtain the first output feature. The first output feature is input into the second network layer of the multilayer perceptron model, and the first convolution operation, the second information interaction operation, and the second feature aggregation operation are performed sequentially to obtain the second output feature. The second output feature is input into the third network layer of the multilayer perceptron model, and the second convolution operation, the third information interaction operation, and the third feature aggregation operation are performed in sequence to obtain the third output feature. The third output feature is input into the fourth network layer of the multilayer perceptron model, and the third convolution operation, the fourth information interaction operation, and the fourth feature aggregation operation are performed in sequence to obtain the fourth output feature. The fourth output feature is subjected to pooling and second mapping operations to obtain the image classification result; The cascaded strip hybrid multilayer perceptron model is a multi-stage cascaded structure containing four network layers. The first network layer is the initial feature extraction layer, and the second to fourth network layers have the same structure, forming a repeated stack of layers. The first network layer includes a linear embedding module, a first cascaded bandgap mixing module, and a first channel mixing module; The second network layer includes a first convolutional module, a second cascaded bandgap mixing module, and a second channel mixing module; The third network layer includes a second convolutional module, a third cascaded bandgap hybrid module, and a third channel hybrid module; The fourth network layer includes a third convolutional module, a fourth cascaded bandgap hybrid module, and a fourth channel hybrid module; The first cascaded strip hybrid module includes a Strip MLP layer and a multi-step dimensional operation layer.
2. The image recognition method for a cascaded strip hybrid multilayer perceptron model according to claim 1, characterized in that, The process of inputting the segmented original image into the first network layer of a cascaded bandgap hybrid multilayer perceptron model, and sequentially performing a first mapping operation, a first information interaction operation, and a first feature aggregation operation to obtain a first output feature, specifically includes: The original image in the first dimension is segmented to obtain multiple image blocks; The first linear module in the first network layer is used to perform the first mapping operation on all the image blocks to obtain the second-dimensional image features. In the width direction, the first cascaded bandgap mixing module in the first network layer is used to sequentially perform the first first dimension permutation, stacking, the second first dimension permutation, restoration, cascade and fusion operations on the second dimension image features to obtain the first fused image features. In the height dimension, the first cascaded strip mixing module is used to sequentially perform the first second-dimensional permutation, stacking, second second-dimensional permutation, restoration, cascade and fusion operations on the first fused image features to obtain the second fused image features; The first feature is obtained by using the first channel mixing module in the first network layer to perform a first feature aggregation operation on the second fused image features.
3. The image recognition method for a cascaded strip hybrid multilayer perceptron model according to claim 1, characterized in that, The step of inputting the first output feature into the second network layer of the multilayer perceptron model, and sequentially performing the first convolution operation, the second information interaction operation, and the second feature aggregation operation to obtain the second output feature specifically includes: The first image patch fusion module in the second network layer is used to perform the first convolution operation on the first output feature to obtain the third dimension image feature; In the width direction, the third dimension image features are sequentially subjected to the first dimension permutation, stacking, the second dimension permutation, restoration, cascading and fusion operations using the second cascaded bandgap mixing module in the second network layer to obtain the third fused image features. In the height dimension, the second cascaded banded mixing module is used to sequentially perform the first second-dimensional permutation, stacking, the second second-dimensional permutation, restoration, cascade and fusion operations on the third fused image features to obtain the fourth fused image features; The second feature is obtained by using the second channel mixing module in the second network layer to perform a second feature aggregation operation on the fourth fused image features.
4. The image recognition method for a cascaded strip hybrid multilayer perceptron model according to claim 3, characterized in that, The step of inputting the second output feature into the third network layer of the multilayer perceptron model, and sequentially performing the second convolution operation, the third information interaction operation, and the third feature aggregation operation to obtain the third output feature, specifically includes: The second image patch fusion module in the third network layer is used to perform the second convolution operation on the second output feature to obtain the fourth dimension image feature; In the width direction, the third cascaded bandgap mixing module in the third network layer is used to sequentially perform the first first dimension permutation, stacking, the second first dimension permutation, restoration, cascade and fusion operations on the image features in the fourth dimension to obtain the fifth fused image features. In the height dimension, the third cascaded banded mixing module is used to sequentially perform the first second-dimensional permutation, stacking, the second second-dimensional permutation, restoration, cascade and fusion operations on the fifth fused image feature to obtain the sixth fused image feature; The third feature is obtained by using the third channel mixing module in the third network layer to perform a third feature aggregation operation on the sixth fused image features.
5. The image recognition method for a cascaded strip hybrid multilayer perceptron model according to claim 4, characterized in that, The step of inputting the third output feature into the fourth network layer of the multilayer perceptron model, and sequentially performing the third convolution operation, the fourth information interaction operation, and the fourth feature aggregation operation to obtain the fourth output feature, specifically includes: The third image patch fusion module in the fourth network layer is used to perform the third convolution operation on the third output feature to obtain the fifth dimension image feature; In the width direction, the fourth cascaded banded mixing module in the fourth network layer is used to sequentially perform the first first dimension permutation, stacking, the second first dimension permutation, restoration, cascade and fusion operations on the fifth dimension image features to obtain the seventh fused image features. In the height dimension, the fourth cascaded banded mixing module is used to sequentially perform the first second-dimensional permutation, stacking, the second second-dimensional permutation, restoration, cascade and fusion operations on the seventh fused image feature to obtain the eighth fused image feature; The fourth feature is obtained by using the fourth channel mixing module in the fourth network layer to perform a fourth feature aggregation operation on the eighth fused image features.
6. The image recognition method for a cascaded strip hybrid multilayer perceptron model according to claim 2, characterized in that, The first dimension is The second dimension is Where H represents the number of pixels in the vertical direction, W represents the number of pixels in the horizontal direction, C represents the number of channels, p represents the length of the image block, and q represents the width of the image block.
7. The image recognition method for a cascaded strip hybrid multilayer perceptron model according to claim 5, characterized in that, The third dimension is The fourth dimension is The fifth dimension is Where H represents the number of pixels in the vertical direction, W represents the number of pixels in the horizontal direction, C represents the number of channels, p represents the length of the image block, and q represents the width of the image block.
8. A cascaded, strip-like hybrid multilayer perceptron model image recognition system, characterized in that, The cascaded strip hybrid multilayer perceptron model image recognition system is applied to the cascaded strip hybrid multilayer perceptron model image recognition method according to any one of claims 1-7, wherein the cascaded strip hybrid multilayer perceptron model image recognition system comprises: The first output feature acquisition module is used to input the segmented original image into the first network layer of the cascaded banded hybrid multilayer perceptron model, and sequentially perform the first mapping operation, the first information interaction operation, and the first feature aggregation operation to obtain the first output feature. The second output feature acquisition module is used to input the first output feature into the second network layer in the multilayer perceptron model, and sequentially perform the first convolution operation, the second information interaction operation, and the second feature aggregation operation to obtain the second output feature. The third output feature acquisition module is used to input the second output feature into the third network layer in the multilayer perceptron model, and sequentially perform the second convolution operation, the third information interaction operation and the third feature aggregation operation to obtain the third output feature. The fourth output feature acquisition module is used to input the third output feature into the fourth network layer in the multilayer perceptron model, and sequentially perform the third convolution operation, the fourth information interaction operation, and the fourth feature aggregation operation to obtain the fourth output feature. The image classification result acquisition module is used to perform pooling and second mapping operations on the fourth output feature to obtain the image classification result.
9. A smart terminal, characterized in that, The intelligent terminal includes: a memory, a processor, a display, and a cascaded striped hybrid multilayer perceptron model image recognition program stored in the memory and executable on the processor. When the cascaded striped hybrid multilayer perceptron model image recognition program is executed by the processor, it implements the steps of the cascaded striped hybrid multilayer perceptron model image recognition method as described in any one of claims 1-7.
10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a cascaded strip hybrid multilayer perceptron model image recognition program, which, when executed by a processor, implements the steps of the cascaded strip hybrid multilayer perceptron model image recognition method as described in any one of claims 1-7.
Citation Information
Patent Citations
Image recognition method based on stream convolution
CN112288028A
Image recognition method and system based on convolutional neural network, terminal and medium
CN112396123A