A method and device for vehicle logo recognition based on a transformer

Abstract

The application provides a vehicle logo recognition method based on a Transformer, which comprises the following steps: receiving a vehicle image; performing high-width fixed ratio edge filling, scaling and high-width 32 multiple edge filling processing on the vehicle image to obtain a processed vehicle image; constructing a vehicle logo recognition network based on a MobileNetV2 network and a Transformer; inputting a vehicle sample image to repeatedly train the vehicle logo recognition network until the network converges, so as to obtain a trained vehicle logo recognition network; and inputting the processed vehicle image into the trained vehicle logo recognition network to obtain a recognition result of the vehicle logo and output the recognition result. Compared with the prior art, the vehicle logo recognition method has higher recognition accuracy, and can recognize vehicle logo information well even in the case of vehicle logo reflection and vehicle logo loss.

Description

Technical Field

[0001] This invention relates to image processing, smart parking, and license plate recognition, and particularly to a vehicle logo recognition method and apparatus based on Transformer. Background Technology

[0002] With the improvement of living standards and industrial development, the number of motor vehicles has grown rapidly, and vehicle information recognition through image recognition has become a hot topic in recent years. Common vehicle information recognition methods mainly rely on license plate numbers; however, for vehicles without license plates, with damaged or obscured license plates, accurate vehicle information cannot be obtained. To address these issues, identifying vehicle brands has become a research hotspot this year.

[0003] Existing vehicle logo recognition methods primarily identify the logos on vehicles to determine their identity. However, these methods suffer from several problems: vehicle logos come in a wide variety of styles, including icons and text, and vary in size and aspect ratio, making it difficult for recognition algorithms to identify small targets with extremely large aspect ratios; furthermore, issues such as glare, missing logos, and damaged logos can prevent the correct identification of logo types.

[0004] In conclusion, there is an urgent need to propose a Transformer-based vehicle logo recognition method to solve the above problems. Summary of the Invention

[0005] In view of this, the main objective of the present invention is to provide a vehicle logo recognition method based on Transformer, which can achieve both accuracy and speed.

[0006] To achieve the above objectives, according to a first aspect of the present invention, a vehicle logo recognition method based on Transformer is provided, the method comprising:

[0007] The first step is to receive vehicle images;

[0008] The second step is to perform edge padding, scaling, and height and width multiplied by 32 on the vehicle image to obtain the processed vehicle image.

[0009] The third step involves constructing a vehicle logo recognition network based on the MobileNetV2 network and Transformer; inputting vehicle sample images and repeatedly training the vehicle logo recognition network until the network converges, thus obtaining a trained vehicle logo recognition network.

[0010] The fourth step is to input the processed vehicle image into the trained vehicle logo recognition network to obtain and output the vehicle logo recognition result.

[0011] In the first step, the vehicle image refers to the sub-image of the vehicle's bounding rectangle.

[0012] The second step includes:

[0013] The fixed aspect ratio edge-padding process involves calculating the fixed aspect ratio height and width of the vehicle image, and then expanding the vehicle image outwards based on the fixed aspect ratio height and width, filling the pixels in the expanded area with 0 pixels to obtain the first vehicle image.

[0014] The fixed-size scaling process involves reducing or enlarging the first vehicle image to obtain the second vehicle image by a fixed size.

[0015] The process of padding the edges to multiples of 32 involves expanding the height and width of the second vehicle image to multiples of 32, and padding the pixels in the expanded area to 0, thus obtaining the processed vehicle image.

[0016] The height-to-width ratio is set according to the resolution of the images captured by the camera.

[0017] The vehicle logo recognition network in the third step S3 includes: Input, CU, MU*1, MU*2, TU*2, MU*2, TU*3, MU*2, TU*2, MU*1, global pooling, and Softmax.

[0018] Wherein, Input represents the input layer; CU represents a convolutional unit, including convolutional layers, batch normalization (BN) layers, and activation functions (ReLU); MU represents a unit module of MobileNetV2, which includes CU, DU, and CU, where DU represents a depthwise convolutional unit, which includes depthwise convolutional layers, batch normalization (BN) layers, and activation functions (ReLU); TU represents a unit module of Transformer; global pooling represents a global pooling layer; and Softmax represents the Softmax loss function.

[0019] Wherein, "*1" represents repeating once, "*2" represents repeating twice, and "*3" represents repeating three times.

[0020] Furthermore, the connection method between the two unit modules, MU and CU, includes: outputting a feature map through MU. H, W, and C represent the height, width, and number of channels of feature map F, respectively. By passing feature map F through a 1x1xCxd convolutional template, the number of channels is reduced to d, resulting in the feature map. Feature map F L Expand into N non-overlapping patches, resulting in Where P = w × h, Feature map F U After L Transformer modules, we get Feature map F G Restored to its original shape Then, using a 1x1xdxC convolution template, the number of channels is restored to C.

[0021] Wherein, patch represents a small block, h and w represent the height and width of patch respectively, P represents the size of patch, and N represents the number of patches that the feature map is expanded into.

[0022] Furthermore, the value of w is 2, 4, or 6; the value of h is 2, 4, or 6; and the value of L is 2, 3, or 4.

[0023] The vehicle sample image is a fixed-size image of a rectangular frame containing the vehicle logo label.

[0024] According to another aspect of the present invention, a Transformer-based vehicle logo recognition device is provided, the device comprising:

[0025] Vehicle image input module, used to receive vehicle images;

[0026] The edge-padding and scaling image processing module is used to perform edge-padding, scaling, and edge-padding with a height and width multiple of 32 on vehicle images to obtain the processed vehicle images.

[0027] The vehicle logo recognition network construction and training module is used to build a vehicle logo recognition network based on MobileNetV2 and Transformer; input vehicle sample images, and repeatedly train the vehicle logo recognition network until the network converges to obtain the trained vehicle logo recognition network;

[0028] The vehicle logo recognition module is used to input the processed vehicle image into the trained vehicle logo recognition network, obtain the vehicle logo recognition result, and output it.

[0029] The vehicle image in the vehicle image input module refers to the sub-image of the vehicle's outer rectangular frame.

[0030] The edge-padding and scaling image processing module includes:

[0031] The fixed aspect ratio edge-padding module is used to calculate the height and width of the vehicle image according to the fixed aspect ratio and the height and width of the vehicle image; and expand the vehicle image outward according to the fixed aspect ratio height and width, and fill the pixels of the expanded area with 0 to obtain the first vehicle image.

[0032] A fixed-size scaling processing module is used to reduce or enlarge the first vehicle image according to a fixed size to obtain a second vehicle image;

[0033] The height and width are multiples of 32 for the edge-padding processing module, which is used to expand the height and width of the second vehicle image to multiples of 32 respectively, and pad the pixels in the expanded area to 0, so as to obtain the processed vehicle image.

[0034] The height-to-width ratio is set according to the resolution of the images captured by the camera.

[0035] The vehicle logo recognition network in the vehicle logo recognition network construction and training module includes: Input, CU, MU*1, MU*2, TU*2, MU*2, TU*3, MU*2, TU*2, MU*1, global pooling, and Softmax.

[0036] Wherein, Input represents the input layer; CU represents a convolutional unit, including convolutional layers, batch normalization (BN) layers, and activation functions (ReLU); MU represents a unit module of MobileNetV2, which includes CU, DU, and CU, where DU represents a depthwise convolutional unit, which includes depthwise convolutional layers, batch normalization (BN) layers, and activation functions (ReLU); TU represents a unit module of Transformer; global pooling represents a global pooling layer; and Softmax represents the Softmax loss function.

[0037] Wherein, "*1" represents repeating once, "*2" represents repeating twice, and "*3" represents repeating three times.

[0038] Furthermore, the connection method between the two unit modules, MU and CU, includes: outputting a feature map through MU. H, W, and C represent the height, width, and number of channels of feature map F, respectively. By passing feature map F through a 1x1xCxd convolutional template, the number of channels is reduced to d, resulting in the feature map. Feature map F L Expand into N non-overlapping patches, resulting in Where P = w × h, Feature map F U After L Transformer modules, we get Feature map F G Restored to its original shape Then, using a 1x1xdxC convolution template, the number of channels is restored to C.

[0039] Wherein, patch represents a small block, h and w represent the height and width of patch respectively, P represents the size of patch, and N represents the number of patches that the feature map is expanded into.

[0040] Furthermore, the value of w is 2, 4, or 6, the value of h is 2, 4, or 6, and the value of L is 2, 3, or 4.

[0041] Compared with existing vehicle logo recognition technologies, the advantages of the vehicle logo recognition method based on Transformer in this invention are: by applying Transformer to convolutional neural networks, while prioritizing the features of the vehicle logo, it combines other vehicle features to effectively identify vehicle logo information even in cases of logo glare or missing logos. Attached Figure Description

[0042] Figure 1 A flowchart of a Transformer-based vehicle logo recognition method according to the present invention is shown.

[0043] Figure 2 A framework diagram of a Transformer-based vehicle logo recognition device according to the present invention is shown. Detailed Implementation

[0044] To enable those skilled in the art to further understand the structure, features and other objectives of the present invention, the following detailed description is provided in conjunction with the accompanying preferred embodiments. The preferred embodiments described are only for illustrating the technical solutions of the present invention and are not intended to limit the present invention.

[0045] Figure 1 A flowchart of a vehicle logo recognition method based on Transformer according to the present invention is provided. Figure 1 As shown, a vehicle logo recognition method based on Transformer according to the present invention includes:

[0046] The first step, S1, is to receive vehicle images;

[0047] The second step, S2, involves processing the vehicle image by adding edges at a fixed height and width ratio, scaling, and adding edges at a multiple of 32 for both height and width, to obtain the processed vehicle image.

[0048] The third step, S3, involves constructing a vehicle logo recognition network based on the MobileNetV2 network and Transformer; inputting vehicle sample images and repeatedly training the vehicle logo recognition network until the network converges, thus obtaining a trained vehicle logo recognition network.

[0049] In the fourth step S4, the processed vehicle image is input into the trained vehicle logo recognition network to obtain and output the vehicle logo recognition result.

[0050] In this process, the vehicle image in the first step S1 refers to the sub-image of the vehicle's bounding rectangle. The vehicle detection method detects the vehicle's bounding rectangle from the image to be detected, and extracts the sub-image of the bounding rectangle from the image based on its position. The vehicle detection method can employ a deep learning network-based approach.

[0051] The second step S2 includes:

[0052] In step S21, the height and width of the vehicle image are calculated according to the fixed height and width ratio. The vehicle image is then expanded outward according to the fixed height and width ratio, and the pixels in the expanded area are padded to 0 to obtain the first vehicle image.

[0053] Fixed-size scaling process S22: The first vehicle image is reduced or enlarged according to a fixed size to obtain the second vehicle image;

[0054] Step S23, which involves padding the height and width of the second vehicle image to multiples of 32, involves expanding the height and width of the second vehicle image to multiples of 32, and padding the pixels in the expanded area to 0, thus obtaining the processed vehicle image.

[0055] The fixed height-to-width ratio edge-padding process S21 is to ensure that the vehicle image is not distorted; the fixed size scaling process S22 is to minimize the time required for vehicle logo recognition without losing vehicle logo information; and the edge-padding process S23, where the height and width are multiples of 32, is to meet the image input requirements of the vehicle logo recognition network.

[0056] The aspect ratio is set based on the resolution of the image captured by the camera. For example, if the image resolution is 16:9, then the aspect ratio is set to 16:9.

[0057] For example, the fixed size is one of the following: a height of 192 pixels and a width of 108 pixels; or a height of 224 pixels and a width of 126 pixels.

[0058] The standard MobileNetV2 network includes: Input, CU, MU*1, MU*2, MU*3, MU*4, MU*3, MU*3, MU*1, global pooling, and Softmax.

[0059] The logo recognition network in the third step S3 is based on the standard MobileNetV2 network, but replaces some MU modules with TU modules. After the replacement, the learned features are more concentrated on the logo and its surroundings.

[0060] The vehicle logo recognition network includes: Input, CU, MU*1, MU*2, TU*2, MU*2, TU*3, MU*2, TU*2, MU*1, global pooling, and Softmax.

[0061] In this context, Input represents the input layer; CU (Convolution Unit) represents a convolutional unit, including convolutional layers, batch normalization (BN) layers, and ReLU activation functions; MU (MobileNetV2 Unit) represents a MobileNetV2 unit module, which includes CU, DU, and CU, where DU (DepthWise Convolution Unit) represents a depthwise convolutional unit, which includes depthwise convolutional layers, batch normalization (BN) layers, and ReLU activation functions; TU (Transformer Unit) represents a Transformer unit module; global pooling represents a global pooling layer; and Softmax represents the Softmax loss function.

[0062] Here, "*1" represents repeating once, "*2" represents repeating twice, and "*3" represents repeating three times. For example, "MU*2" means that the MobilenetV2 unit module repeats the operation twice, and "TU*3" means that the Transformer unit module repeats the operation three times.

[0063] For example, the unit module of the MobilenetV2 is the Mobilenet V2 structure in Figure 4(d) of the document “MobileNetV2: Inverted Residuals and Linear Bottlenecks. M Sandler, A Howard, M Zhu, A Zhmoginov, LCChen.CVPR.2018”.

[0064] Furthermore, the connection method between the two unit modules, MU and CU, includes: outputting a feature map through MU. H, W, and C represent the height, width, and number of channels of feature map F, respectively. By passing feature map F through a 1x1xCxd convolutional template, the number of channels is reduced to d, resulting in the feature map. Feature map F L Expand into N non-overlapping patches, resulting in Where P = w × h, Feature map F UAfter L Transformer modules, we get Feature map F G Restored to its original shape Then, using a 1x1xdxC convolution template, the number of channels is restored to C.

[0065] Wherein, patch represents a small block, h and w represent the height and width of patch respectively, P represents the size of patch, and N represents the number of patches that the feature map is expanded into.

[0066] Furthermore, the value of w can be 2, 4, or 6, and the value of h can be 2, 4, or 6. For example, if w = h = 2, the patch is 2*2, and the feature map with height H and width W can be expanded as follows: One patch.

[0067] The value of L is 2, 3, or 4. For example, L = 2.

[0068] For example, the unit module of the Transformer is the structure of the Vision Transformer in Figure 1 of the literature “An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. A Dosovitskiy, L Beyer, A Kolesnikov, D Weissenborn, N Houlsby”.

[0069] The vehicle sample image is a fixed-size image of a rectangular frame containing the vehicle logo label.

[0070] Figure 2 A framework diagram of a Transformer-based vehicle logo recognition device according to the present invention is shown. Figure 2 As shown, a vehicle logo recognition device based on Transformer according to the present invention includes:

[0071] Vehicle image input module 1, used to receive vehicle images;

[0072] The edge-padding and scaling image processing module 2 is used to perform edge-padding, scaling, and edge-padding with a height and width multiple of 32 on the vehicle image to obtain the processed vehicle image.

[0073] The vehicle logo recognition network construction and training module 3 is used to construct a vehicle logo recognition network based on the MobileNetV2 network and Transformer; input vehicle sample images, repeatedly train the vehicle logo recognition network until the network converges, and obtain the trained vehicle logo recognition network;

[0074] The vehicle logo recognition module 4 is used to input the processed vehicle image into the trained vehicle logo recognition network, obtain the vehicle logo recognition result, and output it.

[0075] The vehicle image in the vehicle image input module 1 refers to the sub-image of the vehicle's outer rectangular frame.

[0076] The edge-padding and scaling image processing module 2 includes:

[0077] The height and width fixed ratio edge-padding processing module 21 is used to calculate the height and width after the fixed ratio according to the height and width of the vehicle image; expand the vehicle image outward according to the height and width after the fixed ratio, and fill the pixels of the expanded area with 0 to obtain the first vehicle image.

[0078] The fixed-size scaling processing module 22 is used to reduce or enlarge the first vehicle image according to a fixed size to obtain the second vehicle image;

[0079] The height and width are multiples of 32 for the edge-padding processing module 23, which is used to expand the height and width of the second vehicle image to multiples of 32 respectively, and pad the pixels in the expanded area to 0, so as to obtain the processed vehicle image.

[0080] The fixed aspect ratio edge-padding module 21 is used to ensure that the vehicle image is not distorted; the fixed size scaling module 22 is used to minimize the time required for vehicle logo recognition without losing vehicle logo information; and the edge-padding module 23, which is a multiple of 32 for aspect ratio, is used to meet the image input requirements of the vehicle logo recognition network.

[0081] The height-to-width ratio is set according to the resolution of the images captured by the camera.

[0082] The vehicle logo recognition network in the vehicle logo recognition network construction and training module 3 includes: Input, CU, MU*1, MU*2, TU*2, MU*2, TU*3, MU*2, TU*2, MU*1, global pooling, and Softmax.

[0083] In this context, Input represents the input layer; CU (Convolution Unit) represents a convolutional unit, including convolutional layers, batch normalization (BN) layers, and ReLU activation functions; MU (MobileNetV2 Unit) represents a MobileNetV2 unit module, which includes CU, DU, and CU, where DU (DepthWise Convolution Unit) represents a depthwise convolutional unit, which includes depthwise convolutional layers, batch normalization (BN) layers, and ReLU activation functions; TU (Transformer Unit) represents a Transformer unit module; global pooling represents a global pooling layer; and Softmax represents the Softmax loss function.

[0084] Here, "*1" represents repeating once, "*2" represents repeating twice, and "*3" represents repeating three times. For example, "MU*2" means that the MobilenetV2 unit module repeats the operation twice, and "TU*3" means that the Transformer unit module repeats the operation three times.

[0085] Furthermore, the connection method between the two unit modules, MU and CU, includes: outputting a feature map through MU. H, W, and C represent the height, width, and number of channels of feature map F, respectively. By passing feature map F through a 1x1xCxd convolutional template, the number of channels is reduced to d, resulting in the feature map. Feature map F L Expand into N non-overlapping patches, resulting in Where P = w × h, Feature map F U After L Transformer modules, we get Feature map F G Restored to its original shape Then, using a 1x1xdxC convolution template, the number of channels is restored to C.

[0086] Wherein, patch represents a small block, h and w represent the height and width of patch respectively, P represents the size of patch, and N represents the number of patches that the feature map is expanded into.

[0087] Furthermore, the value of w is 2, 4, or 6, the value of h is 2, 4, or 6, and the value of L is 2, 3, or 4.

[0088] The vehicle sample image is a fixed-size image of a rectangular frame containing the vehicle logo label.

[0089] Compared with existing vehicle logo recognition technologies, the advantages of the vehicle logo recognition method based on Transformer in this invention are: by applying Transformer to convolutional neural networks, while prioritizing the features of the vehicle logo, it combines other vehicle features to effectively identify vehicle logo information even in cases of logo glare or missing logos.

[0090] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. It should be understood that the present invention is not limited to the implementations described herein. These implementations are described to help those skilled in the art practice the present invention. Any person skilled in the art can easily make further improvements and modifications without departing from the spirit and scope of the present invention. Therefore, the present invention is only limited by the content and scope of the claims, and is intended to cover all alternatives and equivalents included within the spirit and scope of the present invention as defined by the appended claims.

Claims

1. A method for vehicle logo recognition based on a Transformer, characterized in that, The method comprises: A first step of receiving a vehicle image; A second step of performing high-width fixed-ratio edge padding, scaling, and high-width 32-multiple edge padding processing on the vehicle image to obtain a processed vehicle image; A third step of constructing a vehicle logo recognition network based on a MobileNetV2 network and a Transformer, inputting a vehicle sample image, repeatedly training the vehicle logo recognition network until the network converges, and obtaining a trained vehicle logo recognition network; A fourth step of inputting the processed vehicle image into the trained vehicle logo recognition network to obtain a recognition result of a vehicle logo and outputting the recognition result; Further, the second step comprises: A high-width fixed-ratio edge padding processing step of calculating a fixed-ratio height and a fixed-ratio width according to a high-width fixed ratio, a height, and a width of the vehicle image, performing outer expansion on the vehicle image according to the fixed-ratio height and the fixed-ratio width, and padding pixels in an outer expansion area with 0 to obtain a first vehicle image; A fixed-size scaling processing step of performing scaling on the first vehicle image according to a fixed size to obtain a second vehicle image; A high-width 32-multiple edge padding processing step of performing outer expansion on a height and a width of the second vehicle image to multiples of 32, padding pixels in an outer expansion area with 0, and obtaining a processed vehicle image; The high-width fixed ratio is set according to an image resolution collected by a camera; Further, the vehicle logo recognition network in the third step comprises: Input, CU, MU*1, MU*2, TU*2, MU*2, TU*3, MU*2, TU*2, MU*1, global pooling, and Softmax; The Input represents an input layer; the CU represents a convolution unit comprising a convolution layer, a BN layer, and an activation function relu; the MU represents a MobileNetV2 unit module comprising: CU, DU, and CU, wherein the DU represents a depth convolution unit comprising a depth convolution layer, a BN layer, and an activation function (relu); the TU represents a Transformer unit module; the global pooling represents a global pooling layer; and the Softmax represents a Softmax loss function; wherein the "*1" represents repeating 1 time, the "*2" represents repeating 2 times, and the "*3" represents repeating 3 times.

2. The method of claim 1, wherein, The fixed size is one of the following: a height of 192 pixels and a width of 108 pixels, or a height of 224 pixels and a width of 126 pixels.

3. The method of claim 1, wherein, The connection method of the two unit modules of the MU and the CU comprises: passing the MU output feature map H, W, and C are height, width, and channel number of the feature map F respectively; the feature map F is passed through a 1x1xCxd convolution template, the channel number is changed to d, and a feature map The feature map F L is unfolded into N non-overlapping patches, and a feature map wherein P = w x h, The feature map F U is passed through L times of Transformer modules, and a feature map The feature map F G is restored to the previous shape A 1x1xdxC convolution template is used again, the channel number is restored to C, and a feature map wherein the patch represents a small block, h and w represent height and width of the patch respectively, P represents a size of the patch, and N represents a number of patches into which the feature map is unfolded.

4. The method of claim 3, wherein, The value of w is 2, 4, or 6, and the value of h is 2, 4, or 6; and the value of L is 2, 3, or 4. 5.A Transformer-based vehicle logo recognition device, characterized in that, The device comprises: A vehicle image input module configured to receive a vehicle image; An edge padding and scaling image processing module configured to perform high-width fixed-ratio edge padding, scaling, and high-width 32-multiple edge padding processing on the vehicle image to obtain a processed vehicle image; and The vehicle logo recognition network construction and training module is configured to construct a vehicle logo recognition network based on a MobileNetV2 network and a Transformer, input vehicle sample images, repeatedly train the vehicle logo recognition network until the network converges, and obtain a trained vehicle logo recognition network. The vehicle logo recognition module is configured to input the processed vehicle image into the trained vehicle logo recognition network, obtain a recognition result of a vehicle logo, and output the recognition result. Further, the image processing module includes: The high-width fixed-ratio edge filling processing module is configured to calculate a fixed-ratio height and a fixed-ratio width according to a high-width fixed ratio, a height, and a width of the vehicle image, and perform outer expansion on the vehicle image according to the fixed-ratio height and the fixed-ratio width, with pixels in the outer expansion area being filled with 0, to obtain a first vehicle image. The fixed-size scaling processing module is configured to scale down or up the first vehicle image according to a fixed size, to obtain a second vehicle image. The high-width 32-multiple edge filling processing module is configured to perform outer expansion on the height and the width of the second vehicle image to multiples of 32, with pixels in the outer expansion area being filled with 0, to obtain the processed vehicle image. The high-width fixed ratio is set according to an image resolution collected by a camera. Further, the vehicle logo recognition network in the vehicle logo recognition network construction and training module includes: Input, CU, MU*1, MU*2, TU*2, MU*2, TU*3, MU*2, TU*2, MU*1, global pooling, and Softmax. The Input represents an input layer; the CU represents a convolution unit including a convolution layer, a BN layer, and an activation function relu; the MU represents a MobileNetV2 unit module including: CU, DU, and CU, where the DU represents a depth convolution unit including a depth convolution layer, a BN layer, and an activation function (relu); the TU represents a Transformer unit module; the global pooling represents a global pooling layer; and the Softmax represents a Softmax loss function.

6. The apparatus of claim 5, wherein, The fixed size is one of the following: a height of 192 pixels and a width of 108 pixels; and a height of 224 pixels and a width of 126 pixels.

7. The apparatus of claim 5, wherein, The connection method of the two unit modules of the MU and the CU includes: passing the MU output feature map H, W, and C are height, width, and channel number of the feature map F respectively; the feature map F is passed through a 1x1xCxd convolution template, the channel number is changed to d, and a feature map The feature map F is passed through a 1x1xCxd convolution template, the channel number is changed to d, and a feature map L is obtained. Wherein P = w x h, The feature map F is passed through a 1x1xCxd convolution template, the channel number is changed to d, and a feature map U is obtained. The feature map F is passed through a 1x1xCxd convolution template, the channel number is changed to d, and a feature map G is obtained. The feature map F is passed through a 1x1xCxd convolution template, the channel number is changed to d, and a feature map Wherein, the patch represents a small block, h and w represent the height and width of the patch respectively, P represents the size of the patch, and N represents the number of patches obtained by unfolding the feature map.

8. The apparatus of claim 7, wherein, The value of w is 2, 4, or 6, and the value of h is 2, 4, or 6; and the value of L is 2, 3, or 4.

Images