Multi-view image three-dimensional target identification method and system based on MLP attention mechanism

Abstract

The invention relates to a multi-view image three-dimensional target identification method and system based on an MLP attention mechanism. The method comprises the following steps: extracting multi-scale high-dimensional features of each input image; performing feature enhancement on the features of each scale of each image by adopting a local and global combined multi-layer perceptron attention mechanism to obtain enhanced multi-scale features of each image; fusing the multi-scale features of the single image to obtain a feature descriptor of each image, splicing the feature descriptors of all the single images to obtain a global feature map, and then carrying out cross-view feature association modeling on the global feature map to obtain a global feature descriptor; and based on the global feature descriptor, obtaining a discrimination score of the input data belonging to each category, and selecting the category with the highest score as a final recognition result. According to the method, on the premise that the multi-view image three-dimensional target recognition precision is guaranteed, the parameter quantity, the calculation quantity and the reasoning time of the neural network are reduced.

Description

Technical Field

[0001] The present invention belongs to the field of computer vision technology, and in particular relates to a multi-view image three-dimensional target recognition method and system based on an MLP attention mechanism. Background Art

[0002] LiDAR sensors can directly obtain three-dimensional (3D) information of a scene, that is, obtain three-dimensional point cloud data of the scene. Since LiDAR is insensitive to external environments such as lighting, it can collect scene data around the clock and has been widely used in fields such as autonomous driving and robotics. Object recognition and classification based on three-dimensional point cloud data is one of the key technologies for scene intelligent perception and one of the important research tasks in the field of computer vision.

[0003] With the development of artificial intelligence methods represented by deep learning, 3D object recognition based on deep learning has become a mainstream method. According to the expression form of input data, existing deep learning 3D object recognition methods can be divided into: 1) point-based methods; 2) voxel-based methods; 3) multi-view-based methods. In point-based methods, point cloud data is directly input into the network for feature extraction and recognition and classification. However, point clouds have the characteristics of disorder and unstructuredness, and it is more complicated to design a network to directly process point cloud data, and the amount of calculation is also larger. The voxel-based method first voxelizes the point cloud data, and then uses a 3D convolutional neural network (CNN) to extract features from the voxels. However, voxelized expression will cause some loss of input information, and the 3D neural network consumes a greater computational cost. The multi-view-based method first projects the input 3D data to different perspectives to obtain multi-view images, and then extracts and aggregates the features of the multi-view images to achieve high-precision 3D object recognition. Compared with point- and voxel-based methods, multi-view-based methods can make full use of high-performance neural networks in the field of 2D (Two-dimensional) image processing to achieve higher recognition accuracy.

[0004] There are two core issues in view-based methods: 1) how to improve the expressive power of each view feature; 2) how to effectively associate the features of images from different perspectives. Early methods usually extract high-dimensional features of each view through 2D CNN, and then use the maximum pooling layer to aggregate the features of multi-view images to obtain descriptors for object recognition. However, this type of method ignores the feature correlation between multi-view images and the long-range feature correlation within each view image. With the application of the Transformer network structure in the field of natural language processing, this network structure has also been continuously applied to computer vision tasks, including multi-view 3D object recognition tasks. Compared with CNN, the self-attention mechanism used by Transformer can better model and associate long-range information within and between images, and the feature extraction effect is better than CNN, and the recognition accuracy is higher. However, the attention mechanism in Transformer needs to calculate the dot product between each feature, and the amount of calculation is exponentially related to the number of features. The amount of calculation is large and there are many parameters, which is not conducive to practical application and deployment. In addition, Transformer usually pays more attention to global correlation and ignores the local correlation of features. Therefore, in view-based methods, how to achieve high-precision three-dimensional object recognition while reducing network parameters and computational costs is an urgent problem to be solved. Summary of the invention

[0005] In the multi-view 3D target recognition task, it is necessary to ensure the expression ability of the neural network to extract features and the recognition accuracy of the target, and to reduce the computational complexity of the network to facilitate practical application and deployment. To achieve the above purpose, the present invention proposes a multi-view image 3D target recognition method based on MLP attention mechanism, which specifically includes the following steps:

[0006] Step 1, input multi-view images of a three-dimensional object projected from multiple perspectives, and extract multi-scale high-dimensional features of each input image;

[0007] Step 2: For the features of each scale of each image, a multi-layer perceptron attention mechanism combining local and global is used to enhance the features, and the enhanced multi-scale feature set of each image is obtained;

[0008] Step 3: fuse the multi-scale features of a single image to obtain the feature descriptor of each image, concatenate the feature descriptors of all single images to obtain a global feature map, and then perform cross-view feature association modeling on the global feature map to obtain a global feature descriptor;

[0009] Step 4: Based on the global feature descriptor in step 3, obtain the discrimination score of the input data belonging to each category, and select the category with the highest score as the final recognition result.

[0010] Furthermore, in step 1, a convolutional neural network is used to extract features of M scales of each input image to obtain a single view feature set; the convolutional neural network includes GoogLeNet, AlexNet, ResNet and VGG.

[0011] Furthermore, the specific implementation of step 2 includes the following sub-steps:

[0012] Step 2.1, feature downsampling: For all single-view feature sets F obtained in step 1 i After downsampling, we get Where N is the number of images, Indicates that F i All feature maps in Downsampling is performed to obtain a new single-view feature set;

[0013] Step 2.2, single-scale feature local correlation: All single view feature sets in Each feature map in Perform local information modeling and obtain Where j = 1, 2, ... M represents the scale, Express Each feature map in After performing local correlation on single-scale features, a feature set is obtained;

[0014] Step 2.3, single-scale feature global association: for the feature set Each feature map in Supplement the global information of the view and get in Express Each feature map in After globally correlating single-scale features, the feature set is obtained;

[0015] Step 2.4, multi-scale feature fusion: For Each feature set in Will All feature maps in Connect in the channel dimension to get a feature map of size H×W×P Where P = C × M, H and W are the length and width of the feature map, respectively, and C is the number of channels; then, the feature set F i The last feature in Downsample to H×W×P size, and After performing residual connection, we get view-level features For all feature sets After multi-scale feature fusion processing, the feature set is obtained

[0016] Furthermore, in step 2.2, based on the MLP attention mechanism and introducing window attention, a single-scale feature local association module is designed to All single view feature sets in Each feature map in For local information modeling, the output of the single-scale feature local association module The calculation is as follows:

[0017]

[0018] Among them, MLP represents the MLP network layer, W1 is the weight matrix output by the MLP network layer, Norm represents the normalization layer, FFN represents the feedforward network, and softmax is the normalized exponential function; LMA is the local window attention, that is, the feature vector at any position on the feature map is only associated with the feature vector in its 3×3 neighborhood to explore the local information between adjacent features; Each feature map in After local association of single-scale features, the feature set is obtained For all After single-scale feature local correlation processing, we get

[0019] Furthermore, in step 2.3, a single-scale feature global association module is designed to Each feature map in Supplementary view global information, the output of the single-scale feature global association module The calculation is as follows:

[0020]

[0021]

[0022] Among them, MLP represents the MLP network layer, W2 is the weight matrix output by the MLP network layer, Norm and FFN represent the normalization layer and the feedforward network respectively, and softmax represents the normalized exponential function; Each feature map in After global association of single-scale features, the feature set is obtained For all feature sets After global correlation processing of single-scale features, we get

[0023] Furthermore, the specific implementation of step 3 is as follows:

[0024] For the enhanced multi-scale feature set F view Each view feature in Through the pooling layer Aggregate into a feature vector Z of size 1×P i , which is the feature descriptor of a single image, and all feature vectors are concatenated to obtain a global feature map F of size N×P global ;

[0025] Based on MLP attention, a cross-view feature association module based on Transformer structure is constructed. The cross-view feature association module consists of T cascaded cross-view feature association network layers. The output x of the tth cross-view feature association network layer is t The calculation is as follows:

[0026] W t =MLP(x t-1 )

[0027] x′ t =x t-1 +Norm(softmax(W t )x t-1 )

[0028] x t =x t-1 +Norm(FFN(x t-1 ))

[0029] Among them, x t-1 is the output of the t-1th cross-view feature association network layer, MLP represents the MLP network layer, and W t Represents the weight matrix of the output of the MLP network layer, Norm represents the normalization layer, softmax and FFN represent the normalized exponential function and the feedforward network respectively; the global feature map F global Input to the cross-view feature association module, and after T cross-view feature association network layers, the feature map x is obtained T , whose size is N×P; then, x is pooled through the pooling layer T Aggregate into a global feature descriptor of size 1×P.

[0030] Furthermore, in step 4, a fully connected layer is used to process the global feature descriptor to obtain the discriminant score of each category of the input data.

[0031] The present invention also provides a multi-view image three-dimensional object recognition system based on the MLP attention mechanism, comprising the following units:

[0032] A multi-scale high-dimensional feature extraction unit, used to input multi-view images of a three-dimensional object projected at multiple viewing angles, and extract multi-scale high-dimensional features of each input image;

[0033] The multi-scale feature set acquisition unit is used to enhance the features of each scale of each image by using a multi-layer perceptron attention mechanism combining local and global features to obtain an enhanced multi-scale feature set for each image;

[0034] A global feature descriptor acquisition unit is used to fuse the multi-scale features of a single image to obtain the feature descriptor of each image, concatenate the feature descriptors of all single images to obtain a global feature map, and then perform cross-view feature association modeling on the global feature map to obtain a global feature descriptor;

[0035] The recognition unit is used to obtain the discrimination score of each category of the input data based on the global feature descriptor, and select the category with the highest score as the final recognition result.

[0036] The present invention also provides a multi-view image three-dimensional target recognition device based on an MLP attention mechanism, comprising a processor and a memory, the memory being used to store program instructions, and the processor being used to call the stored instructions in the memory to execute a multi-view image three-dimensional target recognition method based on an MLP attention mechanism as described in the above technical solution.

[0037] The present invention also provides a storage medium on which program data is stored. When the program data is executed by a processor, a multi-view image three-dimensional target recognition method based on an MLP attention mechanism as described in the above technical solution is implemented.

[0038] The present invention aims to solve the problem of high computational complexity in the Transformer-based multi-view 3D target recognition method and proposes a multi-view image 3D target recognition method based on MLP (Multi-layer Perceptron) attention mechanism, which can associate the local and global features of each view image and realize effective aggregation of cross-view features. It can also greatly reduce the computational complexity of the network while ensuring high-precision 3D target recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 This is a flowchart of a multi-view image three-dimensional target recognition method based on the MLP attention mechanism in the present invention.

[0040] Figure 2 This is a structural comparison diagram of the MLP attention and Transformer attention proposed in this invention.

[0041] Figure 3 This is an architecture diagram of the single-scale feature local association and single-scale feature global association modules proposed in the present invention.

[0042] Figure 4 This is an architectural diagram of the cross-view feature association module proposed in the present invention.

[0043] Figure 5 This is an input and output example of an embodiment of the present invention. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings. It should be understood that the implementation examples described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0045] The embodiment of the present invention provides a multi-view image 3D object recognition method based on MLP attention mechanism, the overall process is as follows Figure 1 As shown, the specific steps are as follows:

[0046] Step 1: Single-view CNN multi-scale feature extraction.

[0047] The input of the present invention is a multi-view image of a three-dimensional object projected from multiple viewing angles, denoted as V set = {V 1 ,V 2 ,…V N}. For V set Any view image V in i (i=1, 2, ... N), the present invention uses a convolutional neural network (including but not limited to GoogLeNet, AlexNet, ResNet and VGG, etc.) to extract features of M scales to obtain a single view feature set V set After extracting the single view feature set from all views, we get

[0048] Step 2: Single-view feature association based on MLP attention.

[0049] In order to improve the view feature representation capability, the present invention will set Each view feature set F in i To associate information. Figure 2 As shown on the left, in the Transformer, the attention mechanism first projects the feature map to obtain three feature matrices: Query, Key, and Value. Then, the weight matrix is ​​obtained by the dot product of the Query matrix and the Key. Finally, a weighted mapping is performed between different feature vectors in the feature map according to the weight matrix. The attention mechanism enables the Transformer to effectively capture the correlation between different feature vectors. However, the attention mechanism has high computational cost and a large number of parameters, which leads to slow network reasoning and training speed, and is not conducive to practical application and deployment. The present invention proposes an MLP attention mechanism, whose structure is as follows: Figure 2As shown on the right. The attention mechanism uses the MLP network layer to generate a weight matrix and directly performs weighted mapping on the feature vector of the original feature map. In order to improve the view feature representation capability, the present invention models the different scale features of each view from the local and global levels. The modeling process is as follows: Figure 3 As shown. Then, the present invention fuses the multi-scale features into the feature expression of the view. The specific steps of the single view feature association of the present invention are as follows:

[0050] Step 2.1, feature downsampling.

[0051] In order to enable the subsequent steps to share network parameters when processing view features, thereby further reducing the amount of network computation, the present invention will downsample the feature map. set Any single-view feature set F in i The present invention will F i All feature maps in Downsample and unify the feature map size to H×W×C, where H and W are the length and width of the feature map, respectively, and C is the number of channels, to obtain a new single-view feature set For all single-view feature sets F i After downsampling, we can get

[0052] Step 2.2: Single-scale feature local correlation.

[0053] In order to make full use of the local information in the view, this paper designs a single-scale feature local association module based on the MLP attention mechanism and introduces window attention. All single view feature sets in Each feature map in Perform local information modeling, the structure of which is as follows Figure 3 The output of this module is shown on the left. The calculation is as follows:

[0054]

[0055] Among them, MLP represents the MLP network layer, W1 is the weight matrix output by the MLP network layer, Norm represents the normalization layer, FFN represents the feedforward network, and softmax is the normalized exponential function. LWA is the local window attention, that is, the feature vector at any position on the feature map is only associated with the feature vector in its 3×3 neighborhood to explore the local information between adjacent features. Each feature map in After local association of single-scale features, the feature set is obtained For all After single-scale feature local correlation processing, we get

[0056] Step 2.3: Global correlation of single-scale features.

[0057] In order to further improve the feature representation capability, the present invention designs a single-scale feature global association module to Each feature map in Supplementary view global information, its structure is as follows Figure 3 The output of this module is shown on the right. The calculation is as follows:

[0058]

[0059] Among them, MLP represents the MLP network layer, and W2 is the weight matrix output by the MLP network layer. Norm and FFN represent the normalization layer and the feedforward network respectively, and softmax represents the normalized exponential function. In the single-scale feature global association module, each feature vector is modeled with all feature vectors on the feature map to extract the global feature association. Each feature map in After global association of single-scale features, the feature set is obtained For all feature sets After global correlation processing of single-scale features, we get

[0060] Step 2.4, multi-scale feature fusion.

[0061] for Each feature set in The present invention will All feature maps in Connect in the channel dimension to get a feature map of size H×W×P (P=C×M) Then, the feature set F i The last feature in Downsample to H×W×P size, and After performing residual connection, we get view-level features For all feature sets After multi-scale feature fusion processing, the feature set is obtained

[0062] Step 3: Cross-view feature fusion based on MLP attention.

[0063] For F view Each view feature in The present invention uses a pooling layer to Aggregate into a feature vector Z of size 1×P i, which is the feature descriptor of a single image, and all feature vectors are concatenated to obtain a global feature map F of size N×P global .like Figure 4 As shown, in order to realize the cross-view feature information interaction modeling, the present invention proposes a cross-view feature association module similar to the Transformer structure based on MLP attention. The module is composed of T cross-view feature association network layers in cascade. The output x of the tth cross-view feature association network layer is t The calculation is as follows:

[0064] W t =MLP(x t-1 )

[0065] x′ t =x t-1 +Norm(softmax(W t )x t-1 )

[0066] x t =x t-1 +Norm(FFN(x t-1 ))

[0067] Among them, x t-1 is the output of the t-1th cross-view feature association network layer, MLP represents the MLP network layer, and W t represents the weight matrix of the output of the MLP network layer, Norm represents the normalization layer, softmax and FFN represent the normalized exponential function and the feedforward network respectively. global Input to the cross-view feature association module, and after T cross-view feature association network layers, the feature map x is obtained T , whose size is N×P. Then, x is pooled through the pooling layer T Aggregate into a global feature descriptor of size 1×P.

[0068] Step 4: 3D target recognition and classification.

[0069] Based on the feature descriptors obtained after multi-view fusion in step 3, the descriptors are processed using a fully connected layer to obtain the discrimination scores of the input data belonging to each category, and the category with the highest score is selected as the final recognition result.

[0070] The effect of the present invention is described below in conjunction with experimental data and results:

[0071] Dataset:

[0072] The present invention uses the 3D object recognition public benchmark dataset ModelNet40 and its subset ModelNet10 to train and test the network. The specific introduction of the dataset is shown in Table 1. The present invention uses the training set as training data and uses the test set to verify the overall recognition accuracy.

[0073] Table 1 Dataset

[0074]

[0075] Experimental results:

[0076] In the data set used in the present invention, each object is projected into 12 images. Figure 5 As shown in (a) in , when performing recognition, the network inputs 12 projection views. Figure 5 As shown in (b), the output of the present invention is a discriminant score vector, each element in the vector represents the probability of belonging to the category, and the present invention selects the category to which the element with the highest score belongs as the recognition result. Table 2 shows the comparison of the recognition accuracy of the method proposed by the present invention and other state-of-the-art methods.

[0077] Table 2. Comparison with state-of-the-art methods

[0078]

[0079] Table 3 shows the comparison of the proposed method with other state-of-the-art methods in terms of computational overhead and network parameter count, where the computational overhead is measured using floating point number operations (FLOPs).

[0080]

[0081] In a second aspect, the present invention further provides a multi-view image three-dimensional object recognition system based on an MLP attention mechanism, comprising the following units:

[0082] A multi-scale high-dimensional feature extraction unit, used to input multi-view images of a three-dimensional object projected at multiple viewing angles, and extract multi-scale high-dimensional features of each input image;

[0083] The multi-scale feature set acquisition unit is used to enhance the features of each scale of each image by using a multi-layer perceptron attention mechanism combining local and global features to obtain an enhanced multi-scale feature set for each image;

[0084] A global feature descriptor acquisition unit is used to fuse the multi-scale features of a single image to obtain the feature descriptor of each image, concatenate the feature descriptors of all single images to obtain a global feature map, and then perform cross-view feature association modeling on the global feature map to obtain a global feature descriptor;

[0085] The recognition unit is used to obtain the discrimination score of each category of the input data based on the global feature descriptor, and select the category with the highest score as the final recognition result.

[0086] In the third aspect, an embodiment of the present invention also provides a multi-view image three-dimensional target recognition device based on an MLP attention mechanism, comprising a processor and a memory, the memory being used to store program instructions, and the processor being used to call the stored instructions in the memory to execute a multi-view image three-dimensional target recognition method based on an MLP attention mechanism as described in the above technical solution.

[0087] In a fourth aspect, an embodiment of the present invention further provides a storage medium having program data stored thereon, and when the program data is executed by a processor, a multi-view image three-dimensional target recognition method based on an MLP attention mechanism as described in the above technical solution is implemented.

[0088] It should be understood that the above description of the preferred embodiment is relatively detailed and cannot be regarded as limiting the scope of patent protection of the present invention. Under the enlightenment of the present invention, ordinary technicians in this field can also make substitutions or modifications without departing from the scope of protection of the claims of the present invention, which all fall within the scope of protection of the present invention. The scope of protection requested for the present invention shall be based on the attached claims.

Claims

1. A method for multi-view 3D object recognition based on MLP attention mechanism, characterized in that: The steps include: Step 1, input multi-view images of a three-dimensional object projected from multiple perspectives, and extract multi-scale high-dimensional features of each input image; Step 2: For the features of each scale of each image, a multi-layer perceptron attention mechanism combining local and global is used to enhance the features, and a multi-scale feature set of each image is obtained after enhancement. Step 3: fuse the multi-scale features of a single image to obtain the feature descriptor of each image, concatenate the feature descriptors of all single images to obtain a global feature map, and then perform cross-view feature association modeling on the global feature map to obtain a global feature descriptor; Step 4: Based on the global feature descriptor in step 3, obtain the discrimination score of the input data belonging to each category, and select the category with the highest score as the final recognition result.

2. The method for multi-view image 3D object recognition based on MLP attention mechanism as claimed in claim 1, characterized in that: In step 1, a convolutional neural network is used to extract features of M scales of each input image to obtain a single view feature set; the convolutional neural network includes GoogLeNet, AlexNet, ResNet and VGG.

3. The method for multi-view image 3D object recognition based on MLP attention mechanism as claimed in claim 1, characterized in that: The specific implementation of step 2 includes the following sub-steps: Step 2.1, feature downsampling: For all single-view feature sets F obtained in step 1 i After downsampling, we get Where N is the number of images, Indicates that F i All feature maps in Downsampling is performed to obtain a new single-view feature set; Step 2.2, single-scale feature local correlation: All single view feature sets in Each feature map in Perform local information modeling and obtain Where j = 1, 2, ... M represents the scale, Express Each feature map in After performing local correlation on single-scale features, a feature set is obtained; Step 2.3, single-scale feature global association: for the feature set Each feature map in Supplement the global information of the view and get in Express Each feature map in After globally correlating single-scale features, the feature set is obtained; Step 2.4, multi-scale feature fusion: For Each feature set in Will All feature maps in Connect in the channel dimension to get a feature map of size H×W×P Where P = C × M, H and W are the length and width of the feature map, respectively, and C is the number of channels; then, the feature set F i The last feature in Downsample to H×W×P size, and After performing residual connection, we get view-level features For all feature sets After multi-scale feature fusion processing, the feature set is obtained 4. The method for multi-view image 3D object recognition based on MLP attention mechanism as claimed in claim 1, characterized in that: In step 2.2, based on the MLP attention mechanism and introducing window attention, a single-scale feature local association module is designed to All single view feature sets in Each feature map in For local information modeling, the output of the single-scale feature local association module The calculation is as follows: Among them, MLP represents the MLP network layer, W1 is the weight matrix output by the MLP network layer, Norm represents the normalization layer, FFN represents the feedforward network, and softmax is the normalized exponential function; LMA is the local window attention, that is, the feature vector at any position on the feature map is only associated with the feature vector in its 3×3 neighborhood to explore the local information between adjacent features; Each feature map in After local association of single-scale features, the feature set is obtained For all After single-scale feature local correlation processing, we get 5. The method for multi-view image 3D object recognition based on MLP attention mechanism as claimed in claim 1, characterized in that: In step 2.3, a single-scale feature global association module is designed to Each feature map in Supplementary view global information, the output of the single-scale feature global association module The calculation is as follows: Among them, MLP represents the MLP network layer, W2 is the weight matrix output by the MLP network layer, Norm and FFN represent the normalization layer and the feedforward network respectively, and softmax represents the normalized exponential function; Each feature map in After global association of single-scale features, the feature set is obtained For all feature sets After global correlation processing of single-scale features, we get 6. The method for multi-view image 3D object recognition based on MLP attention mechanism as claimed in claim 1, characterized in that: The specific implementation of step 3 is as follows: For the enhanced multi-scale feature set F view Each view feature in Through the pooling layer Aggregate into a feature vector Z of size 1×P i , which is the feature descriptor of a single image, and all feature vectors are concatenated to obtain a global feature map F of size N×P global ; Based on MLP attention, a cross-view feature association module based on Transformer structure is constructed. The cross-view feature association module consists of T cascaded cross-view feature association network layers. The output x of the tth cross-view feature association network layer is t The calculation is as follows: W t =MLP(x t-1 ) x′ t =x t-1 +Norm(softmax(W t )x t-1 ) x t =x t-1 +Norm(FFN(x t-1 )) Among them, x t-1 is the output of the t-1th cross-view feature association network layer, MLP represents the MLP network layer, and W t Represents the weight matrix of the output of the MLP network layer, Norm represents the normalization layer, softmax and FFN represent the normalized exponential function and the feedforward network respectively; the global feature map F global Input to the cross-view feature association module, and after T cross-view feature association network layers, the feature map x is obtained T , whose size is N×P; then, x is pooled through the pooling layer T Aggregate into a global feature descriptor of size 1×P.

7. The method for multi-view image 3D object recognition based on MLP attention mechanism as claimed in claim 1, characterized in that: In step 4, the global feature descriptor is processed using a fully connected layer to obtain the discriminant score of each category of the input data.

8. A multi-view image 3D object recognition system based on MLP attention mechanism, characterized in that: The following units are included: A multi-scale high-dimensional feature extraction unit, used to input multi-view images of a three-dimensional object projected at multiple viewing angles, and extract multi-scale high-dimensional features of each input image; The multi-scale feature set acquisition unit is used to enhance the features of each scale of each image by using a multi-layer perceptron attention mechanism combining local and global features to obtain an enhanced multi-scale feature set for each image. A global feature descriptor acquisition unit is used to fuse the multi-scale features of a single image to obtain the feature descriptor of each image, concatenate the feature descriptors of all single images to obtain a global feature map, and then perform cross-view feature association modeling on the global feature map to obtain a global feature descriptor; The recognition unit is used to obtain the discrimination score of each category of the input data based on the global feature descriptor, and select the category with the highest score as the final recognition result.

9. A multi-view image 3D object recognition device based on MLP attention mechanism, characterized by: It includes a processor and a memory, the memory is used to store program instructions, and the processor is used to call the stored instructions in the memory to execute a multi-view image three-dimensional target recognition method based on the MLP attention mechanism as described in any one of claims 1 to 7.

10. A storage medium having program data stored thereon, characterized in that: When the program data is executed by the processor, a multi-view image three-dimensional target recognition method based on the MLP attention mechanism as described in any one of claim 17 is implemented.

Images