A method and system for multi-view 3D object recognition based on MLP attention mechanism

Through the multi-view image 3D target recognition method based on the MLP attention mechanism, the problem of high computational complexity of Transformer is solved, and high-precision and low-computation 3D target recognition is achieved, which is suitable for the fields of autonomous driving and robotics.

CN119919629BActive Publication Date: 2025-09-26XIANGYANG ZHIMOU CLOUD TECH CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202411891823.8
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-12-20
Publication Date
2025-09-26
Estimated Expiration
2044-12-20

AI Technical Summary

Technical Problem

Existing Transformer-based multi-view 3D object recognition methods are computationally intensive and require many parameters, which is not conducive to practical application and deployment. At the same time, they ignore the local correlation of features, affecting recognition accuracy.

Method used

A multi-view 3D target recognition method based on the MLP attention mechanism is adopted. Feature enhancement is performed through a multi-layer perceptron attention mechanism that combines local and global features. Combined with a cross-view feature association module, the computational complexity is reduced and the recognition accuracy is improved.

Benefits of technology

On the basis of ensuring high-precision three-dimensional target recognition, the network's computational complexity and parameter quantity are significantly reduced, the feature expression capability and recognition accuracy are improved, and it is suitable for practical application and deployment.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119919629B_ABST
    Figure CN119919629B_ABST
Patent Text Reader

Abstract

The present invention relates to a multi-view image three-dimensional object recognition method and system based on an MLP attention mechanism, comprising: extracting multi-scale high-dimensional features from each input image; enhancing the features of each scale of each image using a combined local and global multi-layer perceptron attention mechanism to obtain the enhanced multi-scale features of each image; fusing the multi-scale features of a single image to obtain a feature descriptor for each image, concatenating the feature descriptors of all the single images to obtain a global feature map, and then performing cross-view feature association modeling on the global feature map to obtain a global feature descriptor; based on the global feature descriptor, obtaining a discrimination score for each category of the input data, and selecting the category with the highest score as the final recognition result. While ensuring the accuracy of multi-view image three-dimensional object recognition, the present invention reduces the number of parameters, computational complexity, and inference time of the neural network.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the field of computer vision technology, and in particular relates to a multi-view image three-dimensional target recognition method and system based on an MLP attention mechanism. Background Art

[0002] LiDAR sensors can directly capture three-dimensional (3D) information about a scene, essentially creating a 3D point cloud. Because LiDAR sensors are insensitive to external factors like lighting, they can collect scene data around the clock and are widely used in fields like autonomous driving and robotics. Object recognition and classification based on 3D point cloud data is a key technology for intelligent scene perception and a key research task in computer vision.

[0003] With the development of artificial intelligence methods, represented by deep learning, deep learning-based 3D object recognition has become a mainstream approach. Based on the representation of the input data, existing deep learning 3D object recognition methods can be further categorized as: 1) point-based methods; 2) voxel-based methods; and 3) multi-view-based methods. Point-based methods directly input point cloud data into the network for feature extraction and recognition and classification. However, the disordered and unstructured nature of point clouds makes designing networks to directly process point cloud data more complex and computationally intensive. Voxel-based methods first voxelize the point cloud data and then use a 3D convolutional neural network (CNN) to extract features from the voxels. However, voxel-based representation results in some loss of input information, and the 3D neural network consumes a greater computational cost. Multi-view-based methods first project the input 3D data onto different viewpoints to generate multi-view images. Feature extraction and aggregation of these multi-view images are then performed to achieve high-precision 3D object recognition. Compared with point- and voxel-based methods, multi-view-based methods can make full use of high-performance neural networks in the field of 2D (Two-dimensional) image processing to achieve higher recognition accuracy.

[0004] View-based methods face two key challenges: 1) how to improve the expressive power of features within each view; and 2) how to effectively correlate features between images from different viewpoints. Early approaches typically used 2D CNNs to extract high-dimensional features from each view and then aggregated the features from multiple views using a max-pooling layer to generate descriptors for object recognition. However, these approaches neglected the correlation between features across multiple views and the long-range feature correlations within each view. With the application of the Transformer network architecture in natural language processing, it has also been increasingly applied to computer vision tasks, including multi-view 3D object recognition. Compared to CNNs, the self-attention mechanism employed by the Transformer can better model and correlate long-range information within and between images, resulting in superior feature extraction and higher recognition accuracy. However, the Transformer's attention mechanism requires computing dot products between each feature, which is computationally expensive and requires many parameters, making it difficult to implement in practical applications and deployments. Furthermore, the Transformer typically focuses on global correlations while ignoring local feature correlations. Therefore, in view-based methods, how to achieve high-precision three-dimensional object recognition while reducing network parameters and computational costs is an urgent problem to be solved. Summary of the Invention

[0005] In the multi-view 3D object recognition task, it is necessary to ensure the expressiveness of the neural network in extracting features and the recognition accuracy of the target, while also reducing the network's computational complexity to facilitate practical application and deployment. To achieve the above objectives, the present invention proposes a multi-view 3D object recognition method based on the MLP attention mechanism, which specifically includes the following steps:

[0006] Step 1: Input multi-view images of a 3D object projected from multiple perspectives and extract multi-scale high-dimensional features from each input image;

[0007] Step 2: For each scale feature of each image, a multi-layer perceptron attention mechanism combining local and global attention is used to enhance the features, and the enhanced multi-scale feature set of each image is obtained;

[0008] Step 3: Fuse the multi-scale features of a single image to obtain the feature descriptor of each image, concatenate the feature descriptors of all single images to obtain a global feature map, and then perform cross-view feature correlation modeling on the global feature map to obtain a global feature descriptor.

[0009] In step 4, based on the global feature descriptor in step 3, the discrimination score of the input data belonging to each category is obtained, and the category with the highest score is selected as the final recognition result.

[0010] Furthermore, in step 1, a convolutional neural network is used to extract M-scale features of each input image to obtain a single-view feature set; the convolutional neural network includes GoogLeNet, AlexNet, ResNet and VGG.

[0011] Furthermore, the specific implementation of step 2 includes the following sub-steps:

[0012] Step 2.1, feature downsampling: For all single-view feature sets F obtained in step 1 i After downsampling, we get Where N is the number of images, Indicates that F i All feature maps in Downsampling is performed to obtain a new single-view feature set;

[0013] Step 2.2, single-scale feature local correlation: All single-view feature sets in Each feature map in Perform local information modeling and obtain Where j = 1, 2, ... M represents the scale, Express Each feature map in After performing local correlation on single-scale features, a feature set is obtained;

[0014] Step 2.3, single-scale feature global association: for feature set Each feature map in Supplement the global information of the view and get in Express Each feature map in After performing global correlation on single-scale features, the feature set is obtained;

[0015] Step 2.4, multi-scale feature fusion: For Each feature set in Will All feature maps within Connect in the channel dimension to obtain a feature map of size H×W×P Where P = C × M, H and W are the length and width of the feature map respectively, and C is the number of channels; then, the feature set F i The last feature in Downsample to H×W×P size, After performing residual connection, view-level features are obtained For all feature sets After multi-scale feature fusion processing, the feature set is obtained

[0016] Furthermore, in step 2.2, based on the MLP attention mechanism and introducing window attention, a single-scale feature local association module is designed to All single-view feature sets in Each feature map in Perform local information modeling, the output of the single-scale feature local association module The calculation is as follows:

[0017]

[0018] Among them, MLP represents the MLP network layer, W1 is the weight matrix output by the MLP network layer, Norm represents the normalization layer, FFN represents the feedforward network, and softmax is the normalized exponential function; LMA is the local window attention, that is, the feature vector at any position on the feature map is only associated with the feature vectors within its 3×3 neighborhood to explore the local information between adjacent features; Each feature map in After performing local correlation of single-scale features, the feature set is obtained For all After performing local correlation processing on single-scale features, we get

[0019] Furthermore, in step 2.3, a single-scale feature global association module is designed to Each feature map in Supplementary view global information, the output of the single-scale feature global association module The calculation is as follows:

[0020]

[0021]

[0022] Among them, MLP represents the MLP network layer, W2 is the weight matrix output by the MLP network layer, Norm and FFN represent the normalization layer and feedforward network respectively, and softmax represents the normalized exponential function; Each feature map in After global correlation of single-scale features, the feature set is obtained For all feature sets After performing global correlation processing on single-scale features, we can obtain

[0023] Furthermore, the specific implementation of step 3 is as follows:

[0024] For the enhanced multi-scale feature set F view Each view feature in Through the pooling layer Aggregate into a feature vector Z of size 1×P i , that is, the feature descriptor of a single image, and all feature vectors are spliced ​​to obtain a global feature map F of size N×P global ;

[0025] Based on MLP attention, a cross-view feature association module based on Transformer structure is constructed. The cross-view feature association module consists of T cross-view feature association network layers in cascade. The output x of the t-th cross-view feature association network layer is t The calculation is as follows:

[0026] W t =MLP(x t-1 )

[0027] x′ t =x t-1 +Norm(softmax(W t )x t-1 )

[0028] x t =x t-1 +Norm(FFN(x t-1 ))

[0029] Among them, x t-1 is the output of the t-1th cross-view feature association network layer, MLP represents the MLP network layer, and W t Represents the weight matrix of the output of the MLP network layer, Norm represents the normalization layer, softmax and FFN represent the normalized exponential function and feedforward network respectively; the global feature map F global Input to the cross-view feature association module, and after T cross-view feature association network layers, the feature map x is obtained T , whose size is N×P; then, x is pooled through the pooling layer T Aggregate into a global feature descriptor of size 1×P.

[0030] Furthermore, in step 4, a fully connected layer is used to process the global feature descriptor to obtain the discriminant score of each category of the input data.

[0031] The present invention also provides a multi-view image 3D object recognition system based on the MLP attention mechanism, comprising the following units:

[0032] A multi-scale high-dimensional feature extraction unit is used to input multi-view images of a three-dimensional object projected from multiple perspectives and extract multi-scale high-dimensional features from each input image;

[0033] The multi-scale feature set acquisition unit is used to enhance the features of each scale of each image using a multi-layer perceptron attention mechanism that combines local and global attention to obtain the enhanced multi-scale feature set of each image;

[0034] The global feature descriptor acquisition unit is used to fuse the multi-scale features of a single image to obtain the feature descriptor of each image, and then to concatenate the feature descriptors of all single images to obtain a global feature map. The global feature map is then modeled for cross-view feature association to obtain a global feature descriptor.

[0035] The recognition unit is used to obtain the discrimination score of each category of the input data based on the global feature descriptor, and select the category with the highest score as the final recognition result.

[0036] The present invention also provides a multi-view image three-dimensional target recognition device based on the MLP attention mechanism, including a processor and a memory, the memory is used to store program instructions, and the processor is used to call the stored instructions in the memory to execute a multi-view image three-dimensional target recognition method based on the MLP attention mechanism as described in the above technical solution.

[0037] The present invention also provides a storage medium on which program data is stored. When the program data is executed by a processor, a multi-view image three-dimensional target recognition method based on the MLP attention mechanism as described in the above technical solution is implemented.

[0038] This paper addresses the problem of high computational complexity in Transformer-based multi-view 3D object recognition methods and proposes a multi-view image 3D object recognition method based on the MLP (Multi-layer Perceptron) attention mechanism. This method can associate the local and global features of each view image and achieve effective aggregation of cross-view features. While ensuring high-precision 3D object recognition, it can also significantly reduce the computational complexity of the network. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 This is a flowchart of a multi-view image three-dimensional target recognition method based on the MLP attention mechanism in the present invention.

[0040] Figure 2 This is a structural comparison diagram of the MLP attention and Transformer attention proposed in this invention.

[0041] Figure 3 This is the architecture diagram of the single-scale feature local association and single-scale feature global association modules proposed in the present invention.

[0042] Figure 4 This is an architectural diagram of the cross-view feature association module proposed in the present invention.

[0043] Figure 5 This is an input and output example of an embodiment of the present invention. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the invention is further described in detail below with reference to the accompanying drawings. It should be understood that the embodiments described herein are only used to illustrate and explain the present invention and are not intended to limit the present invention.

[0045] The embodiment of the present invention provides a multi-view image 3D object recognition method based on MLP attention mechanism. The overall process is as follows: Figure 1 The specific steps are as follows:

[0046] Step 1: Single-view CNN multi-scale feature extraction.

[0047] The input of the present invention is a multi-view image of a three-dimensional object projected from multiple perspectives, denoted as V set ={V 1 ,V 2 ,…V N}. For V set Any view image V in i (i=1, 2, ... N), the present invention uses convolutional neural networks (including but not limited to GoogLeNet, AlexNet, ResNet and VGG, etc.) to extract features of M scales and obtain a single view feature set V set After extracting the single view feature set from all views, we get

[0048] Step 2: Single-view feature association based on MLP attention.

[0049] In order to improve the view feature representation capability, the present invention will set Each view feature set F in i Perform information association. Figure 2 As shown on the left, in the Transformer, the attention mechanism first projects the feature map to obtain three feature matrices: Query, Key, and Value. Then, the weight matrix is ​​obtained by the dot product of the Query matrix and the Key. Finally, a weighted mapping is performed between different feature vectors in the feature map according to the weight matrix. The attention mechanism enables the Transformer to effectively capture the correlation between different feature vectors. However, the attention mechanism has high computational cost and a large number of parameters, which leads to slow network inference and training speed, which is not conducive to practical application and deployment. The present invention proposes an MLP attention mechanism, whose structure is as follows: Figure 2As shown on the right. The attention mechanism uses the MLP network layer to generate a weight matrix and directly performs weighted mapping on the feature vector of the original feature map. In order to improve the view feature representation capability, the present invention models the different scale features of each view from the local and global levels. The modeling process is as follows Figure 3 As shown. Subsequently, the present invention fuses multi-scale features into the feature expression of the view. The specific steps of the present invention for single view feature association are as follows:

[0050] Step 2.1, feature downsampling.

[0051] In order to enable the subsequent steps to share network parameters when processing view features, thereby further reducing the amount of network computation, the present invention will perform downsampling on the feature map. set Any single-view feature set F in i , the present invention will F i All feature maps in Downsample and unify the size of the feature map to H×W×C, where H and W are the length and width of the feature map respectively, and C is the number of channels, to obtain a new single-view feature set For all single-view feature sets F i After downsampling, we can get

[0052] Step 2.2: Single-scale feature local correlation.

[0053] In order to make full use of the local information in the view, this paper designs a single-scale feature local association module based on the MLP attention mechanism and introduces window attention. All single-view feature sets in Each feature map in Perform local information modeling, the structure of which is as follows Figure 3 As shown on the left. The output of this module The calculation is as follows:

[0054]

[0055] Among them, MLP represents the MLP network layer, W1 is the weight matrix output by the MLP network layer, Norm represents the normalization layer, FFN represents the feedforward network, and softmax is the normalized exponential function. LWA is the local window attention, that is, the feature vector at any position on the feature map is only associated with the feature vectors within its 3×3 neighborhood to explore the local information between adjacent features. Each feature map in After performing local correlation of single-scale features, the feature set is obtained For all After performing local correlation processing on single-scale features, we get

[0056] Step 2.3: Global correlation of single-scale features.

[0057] In order to further improve the feature representation capability, the present invention designs a single-scale feature global association module to Each feature map in Supplementary view global information, its structure is as follows Figure 3 As shown on the right. The output of this module The calculation is as follows:

[0058]

[0059] Among them, MLP represents the MLP network layer, and W2 is the weight matrix output by the MLP network layer. Norm and FFN represent the normalization layer and the feedforward network respectively, and softmax represents the normalized exponential function. In the single-scale feature global association module, each feature vector is modeled with all feature vectors on the feature map to extract global feature correlation. Each feature map in After global correlation of single-scale features, the feature set is obtained For all feature sets After performing global correlation processing on single-scale features, we can obtain

[0060] Step 2.4, multi-scale feature fusion.

[0061] for Each feature set in The present invention will All feature maps within Connect in the channel dimension to obtain a feature map of size H×W×P (P=C×M) Then, the feature set F i The last feature in Downsample to H×W×P size, After performing residual connection, view-level features are obtained For all feature sets After multi-scale feature fusion processing, the feature set is obtained

[0062] Step 3: Cross-view feature fusion based on MLP attention.

[0063] For F view Each view feature in The present invention uses the pooling layer to Aggregate into a feature vector Z of size 1×P i, that is, the feature descriptor of a single image, and all feature vectors are spliced ​​to obtain a global feature map F of size N×P global .like Figure 4 As shown, in order to realize the cross-view feature information interaction modeling, this paper proposes a cross-view feature association module similar to the Transformer structure based on MLP attention. The module consists of T cross-view feature association network layers in cascade. The output x of the tth cross-view feature association network layer is t The calculation is as follows:

[0064] W t =MLP(x t-1 )

[0065] x′ t =x t-1 +Norm(softmax(W t )x t-1 )

[0066] x t =x t-1 +Norm(FFN(x t-1 ))

[0067] Among them, x t-1 is the output of the t-1th cross-view feature association network layer, MLP represents the MLP network layer, and W t Represents the weight matrix of the MLP network layer output, Norm represents the normalization layer, softmax and FFN represent the normalized exponential function and feedforward network respectively. global Input to the cross-view feature association module, and after T cross-view feature association network layers, the feature map x is obtained T , whose size is N×P. Then, x is pooled through the pooling layer. T Aggregate into a global feature descriptor of size 1×P.

[0068] Step 4: 3D target recognition and classification.

[0069] Based on the feature descriptors obtained after multi-view fusion in step 3, the descriptors are processed using a fully connected layer to obtain the discrimination scores of each category for the input data, and the category with the highest score is selected as the final recognition result.

[0070] The effect of the present invention is described below in conjunction with experimental data and results:

[0071] Dataset:

[0072] The present invention uses the 3D object recognition public benchmark dataset ModelNet40 and its subset ModelNet10 to train and test the network. The specific introduction of the dataset is shown in Table 1. The present invention uses the training set as training data and the test set to verify the overall recognition accuracy.

[0073] Table 1 Dataset

[0074]

[0075] Experimental results:

[0076] In the dataset used in this invention, each object is projected into 12 images. Figure 5 As shown in (a) in , when performing recognition, the network inputs 12 projection views. Figure 5 As shown in (b) of the figure, the output of the present invention is a discriminant score vector, where each element represents the probability of belonging to that category. The present invention selects the category of the element with the highest score as the recognition result. Table 2 shows a comparison of the recognition accuracy of the proposed method with other state-of-the-art methods.

[0077] Table 2 Comparison with state-of-the-art methods

[0078]

[0079] Table 3 shows the comparison of the proposed method with other state-of-the-art methods in terms of computational overhead and network parameter count. The computational overhead is measured by the number of floating point operations (FLOPs).

[0080]

[0081] In a second aspect, the present invention further provides a multi-view image 3D object recognition system based on an MLP attention mechanism, comprising the following units:

[0082] A multi-scale high-dimensional feature extraction unit is used to input multi-view images of a three-dimensional object projected from multiple perspectives and extract multi-scale high-dimensional features from each input image;

[0083] The multi-scale feature set acquisition unit is used to enhance the features of each scale of each image using a multi-layer perceptron attention mechanism that combines local and global attention to obtain the enhanced multi-scale feature set of each image;

[0084] The global feature descriptor acquisition unit is used to fuse the multi-scale features of a single image to obtain the feature descriptor of each image, and then to concatenate the feature descriptors of all single images to obtain a global feature map. The global feature map is then modeled for cross-view feature association to obtain a global feature descriptor.

[0085] The recognition unit is used to obtain the discrimination score of each category of the input data based on the global feature descriptor, and select the category with the highest score as the final recognition result.

[0086] In the third aspect, an embodiment of the present invention also provides a multi-view image three-dimensional target recognition device based on the MLP attention mechanism, including a processor and a memory, the memory is used to store program instructions, and the processor is used to call the stored instructions in the memory to execute a multi-view image three-dimensional target recognition method based on the MLP attention mechanism as described in the above technical solution.

[0087] In a fourth aspect, an embodiment of the present invention further provides a storage medium on which program data is stored. When the program data is executed by a processor, a multi-view image three-dimensional target recognition method based on the MLP attention mechanism as described in the above technical solution is implemented.

[0088] It should be understood that the above description of the preferred embodiment is relatively detailed and cannot be regarded as limiting the scope of protection of the patent of the present invention. Under the guidance of the present invention, ordinary technicians in this field can also make substitutions or modifications without departing from the scope of protection of the claims of the present invention, which all fall within the scope of protection of the present invention. The scope of protection requested by the present invention shall be based on the attached claims.

Claims

1. A multi-view 3D object recognition method based on MLP attention mechanism, characterized by: The steps include: Step 1: Input multi-view images of a 3D object projected from multiple perspectives and extract multi-scale high-dimensional features from each input image; Step 2: For each scale feature of each image, a multi-layer perceptron attention mechanism combining local and global attention is used to enhance the features, and the enhanced multi-scale feature set of each image is obtained; The specific implementation of step 2 includes the following sub-steps: Step 2.1, feature downsampling: For all single-view feature sets F obtained in step 1 i After downsampling, we get Where N is the number of images, Indicates that F i All feature maps in Downsampling is performed to obtain a new single-view feature set; Step 2.2, single-scale feature local association: Based on the MLP attention mechanism and introducing window attention, a single-scale feature local association module is designed to All single-view feature sets in Each feature map in Perform local information modeling, the output of the single-scale feature local association module The calculation is as follows: Among them, MLP represents the MLP network layer, W1 is the weight matrix output by the MLP network layer, Norm represents the normalization layer, FFN represents the feedforward network, and softmax is the normalized exponential function; LWA is the local window attention, that is, the feature vector at any position on the feature map is only associated with the feature vectors in its 3×3 neighborhood to explore the local information between adjacent features; Each feature map in After performing local correlation of single-scale features, the feature set is obtained For all After performing local correlation processing on single-scale features, we get Step 2.3, single-scale feature global association: Design a single-scale feature global association module to Each feature map in Supplementary view global information, the output of the single-scale feature global association module The calculation is as follows: Among them, Each feature map in After global correlation of single-scale features, the feature set is obtained For all feature sets After performing global correlation processing on single-scale features, we can obtain Step 3: Cross-view feature fusion based on MLP attention: The multi-scale features of a single image are fused to obtain the feature descriptor of each image. The feature descriptors of all single images are concatenated to obtain a global feature map. The cross-view feature association modeling is then performed on the global feature map to obtain the global feature descriptor. In step 4, based on the global feature descriptor in step 3, the discrimination score of the input data belonging to each category is obtained, and the category with the highest score is selected as the final recognition result.

2. The method for multi-view 3D object recognition based on the MLP attention mechanism according to claim 1, characterized in that: In step 1, a convolutional neural network is used to extract M-scale features of each input image to obtain a single-view feature set; the convolutional neural network includes GoogLeNet, AlexNet, ResNet and VGG.

3. The method for multi-view 3D object recognition based on the MLP attention mechanism according to claim 1, characterized in that: Step 2 also includes: Step 2.4, multi-scale feature fusion: For Each feature set in Will All feature maps within Connect in the channel dimension to obtain a feature map of size H×W×P Where P = C × M, H and W are the length and width of the feature map respectively, and C is the number of channels; then, the feature set F i The last feature in Downsample to H×W×P size, After performing residual connection, view-level features are obtained For all feature sets After multi-scale feature fusion processing, the feature set is obtained 4. The method for multi-view 3D object recognition based on the MLP attention mechanism according to claim 1, characterized in that: The specific implementation of step 3 is as follows: For the enhanced multi-scale feature set F view Each view feature in Through the pooling layer Aggregate into a feature vector Z of size 1×P i , that is, the feature descriptor of a single image, and all feature vectors are spliced ​​to obtain a global feature map F of size N×P global ; Based on MLP attention, a cross-view feature association module based on Transformer structure is constructed. The cross-view feature association module consists of T cross-view feature association network layers in cascade. The output x of the t-th cross-view feature association network layer is t The calculation is as follows: W t =MLP(x t-1 ) x′ t =x t-1 +Norm(softmax(W t )·x t-1 ) x t =x t-1 +Norm(FFN(x t-1 )) Among them, x t-1 is the output of the t-1th cross-view feature association network layer, MLP represents the MLP network layer, and W t Represents the weight matrix of the output of the MLP network layer, Norm represents the normalization layer, softmax and FFN represent the normalized exponential function and feedforward network respectively; the global feature map F global Input to the cross-view feature association module, and after T cross-view feature association network layers, the feature map x is obtained T , whose size is N×P; then, x is pooled through the pooling layer T Aggregate into a global feature descriptor of size 1×P.

5. The method for multi-view 3D object recognition based on the MLP attention mechanism according to claim 1, characterized in that: In step 4, the global feature descriptor is processed using a fully connected layer to obtain the discriminant score of each category of the input data.

6. A multi-view image 3D object recognition system based on an MLP attention mechanism, used to implement a multi-view image 3D object recognition method based on an MLP attention mechanism as described in any one of claims 1 to 5, characterized in that: Includes the following units: A multi-scale high-dimensional feature extraction unit is used to input multi-view images of a three-dimensional object projected from multiple perspectives and extract multi-scale high-dimensional features from each input image; The multi-scale feature set acquisition unit is used to enhance the features of each scale of each image using a multi-layer perceptron attention mechanism that combines local and global attention to obtain the enhanced multi-scale feature set of each image; The global feature descriptor acquisition unit is used to fuse the multi-scale features of a single image to obtain the feature descriptor of each image, and then to concatenate the feature descriptors of all single images to obtain a global feature map. The global feature map is then modeled for cross-view feature association to obtain a global feature descriptor. The recognition unit is used to obtain the discrimination score of each category of the input data based on the global feature descriptor, and select the category with the highest score as the final recognition result.

7. A multi-view 3D object recognition device based on an MLP attention mechanism, characterized by: It includes a processor and a memory, the memory is used to store program instructions, and the processor is used to call the stored instructions in the memory to execute a multi-view image three-dimensional target recognition method based on the MLP attention mechanism as described in any one of claims 1 to 5.

8. A storage medium having program data stored thereon, characterized in that: When the program data is executed by the processor, a multi-view image three-dimensional target recognition method based on the MLP attention mechanism is implemented as described in any one of claims 1 to 5.

Citation Information

Patent Citations

  • Multi-view 3D object identification method and system

    CN116403028A

  • Dam defect time-sequence image description method based on local self-attention mechanism

    WO2023217163A1