Traffic scene target detection method and device based on improved RT-DETR and storage medium

Abstract

The application provides a traffic scene target detection method and device based on improved RT-DETR and a storage medium, and belongs to the field of target detection; the method comprises the following steps: data set preparation; constructing an improved RT-DETR model, including a feature extraction network Backbone module, a feature fusion network Neck module and a feature recognition network Head module, the feature extraction network Backbone module integrates a WM-Dual Block module which fuses wavelet transform and linear selective scanning; an SQ-ACA module which fuses output feature maps of different stages and performs attention interaction along the axial direction is added in the feature fusion network Neck module on the basis of feature fusion; the WM-Dual Block module replaces the SPPF module in the Backbone; model training; output result; the application can output accurate positioning of a target region in a traffic scene and the type of the target.

Description

Technical Field

[0001] This application relates to the field of target detection technology, and in particular to a target detection method, device and storage medium for traffic scenes based on an improved RT-DETR. Background Technology

[0002] Object detection algorithms play a crucial role in autonomous driving and intelligent transportation systems, providing reliable and real-time environmental perception inputs for core tasks such as assisted driving decision-making, traffic signal perception, and pedestrian behavior prediction, and have broad application value.

[0003] In recent years, with the rapid development of computer vision technology, deep learning-based object detection methods have been widely applied in intelligent transportation scenarios. Compared with traditional machine learning methods (such as using the SIFT operator for manual feature extraction and combining it with a classifier for recognition), modern detection models represented by the YOLO series and RT-DETR have achieved a better balance between accuracy and inference efficiency.

[0004] However, in practical applications, existing algorithms still face many challenges: (1) Target occlusion problem in dense scenes: In urban environments with dense traffic or complex road structures, targets such as pedestrians and non-motorized vehicles are often partially or completely occluded by adjacent vehicles, resulting in a significant increase in the missed detection rate. (2) Perception degradation under low light and adverse weather conditions: In scenarios such as night, cloudy days, or rain and fog, image quality is limited by insufficient light, reflection interference, or sensor noise, which can easily lead to false detection or missed detection. The reflection of rainwater on the road surface may be misjudged as traffic lights, while traffic signs or pedestrians in dim environments may not be effectively identified due to blurred features. Summary of the Invention

[0005] To address the aforementioned technical issues, this application proposes a traffic scene target detection method, device, and storage medium based on an improved RT-DETR architecture. Based on the RT-DETR architecture, it aims to further enhance robustness and adaptability in complex traffic scenes, enabling accurate localization of target areas in traffic scenes while outputting the type of traffic targets in the image.

[0006] The technical solution adopted in this application is: a traffic scene target detection method based on improved RT-DETR, including the following steps: Step 1: Construct and partition the traffic scene object detection dataset to obtain the test set and training set; Step 2: Construct a traffic scene target detection model. This model is an improvement on the RT-DETR model and includes a Backbone feature extraction network module, a Neck feature fusion network module, and a Head feature recognition network module. The Backbone feature extraction network module integrates the WM-DualBlock module, which fuses wavelet transform and linear selective scanning. The Neck feature fusion network module adds the SQ-ACA module, which fuses output feature maps from different stages and performs attention interaction along the axis, on the basis of feature fusion. The WM-DualBlock module replaces the SPPF module in the Backbone module. The SQ-ACA module is added before the stitching operation of feature maps at different scales. Step 3: Train the model using the training set images and save the parameters; Step 4: Input the test set images into the trained model to obtain the final traffic target detection results and label their categories.

[0007] Furthermore, the WM-Dual Block module includes a first Split module. After the first Split module, the feature map is divided into two information streams according to the number of channels based on the splitting factor 'a'. The first information stream is directly passed to the first Concat module without any additional processing. The second information stream is processed by wavelet transform to obtain four feature maps of different frequencies, namely LL, LH, HL, and HH. Each feature map is processed by the selective scan module. The processing result is then processed by inverse wavelet transform to obtain the feature map of the second information stream, which is then passed to the first Concat module. The first Concat module concatenates the two information streams along the channel direction to obtain the output feature map.

[0008] Furthermore, the selective scan module is used to duplicate the four feature maps of different frequencies twice. One copy is processed by the first Conv module and the SiLU activation function to obtain a weighted feature map; the other copy is processed by the second Conv module, the first SS2D module and the LayerNorm module to obtain a spatial scan feature map. Finally, the weighted feature map and the spatial scan feature map are multiplied to obtain the output feature map.

[0009] Furthermore, the SQ-ACA module includes a Positional Embedding module, a Feature Alignment module, and a CrossPath Bridge module. The Positional Embedding module processes and fuses the two feature maps input to the SQ-ACA module to output an information stream. Subsequently, the Feature Alignment module aligns the features of the information stream. The aligned information stream is then processed by the CrossPath Bridge module to extract local spatial information based on Conv modules of varying degrees. The information stream output by the CrossPath Bridge module is then divided into five information streams by the third Conv module: Vx, Kx, Q, Ky, and Vy. Q interacts with Vx and Kx to obtain a horizontal attention feature map; Q interacts with Ky and Vy to obtain a vertical attention interaction map. Simultaneously, Q is added to both the horizontal and vertical attention interaction maps by the DWConv module. The two attention interaction maps are then concatenated and integrated by the second Concat module. The concatenated fusion is further integrated and fused by the MLP module to obtain the output features.

[0010] Furthermore, the Feature Alignment module first performs average and max pooling on the first input feature map, then performs convolution with a kernel size of 7, and finally adds it to the second input feature map to achieve feature alignment, outputting an aligned feature map.

[0011] Furthermore, the CrossPath Bridge module uses multi-scale convolutions to operate on the alignment feature map. First, the second Split module splits the alignment feature map into four equal parts along the channel direction. The first feature map is processed by the fourth Conv module; the second feature map is processed by the fifth Conv module; the third feature map is processed by the sixth Conv module; and the fourth feature map is processed by the seventh Conv module. Then, the third Concat module merges the four feature maps into one along the channel direction.

[0012] Furthermore, the fourth Conv module has a convolution size of 5 and a dilation of 1; the fifth Conv module has a convolution kernel size of 3 and a dilation of 1; the sixth Conv module has a convolution kernel size of 3 and a dilation of 2; and the seventh Conv module has a convolution kernel size of 3 and a dilation of 3.

[0013] Furthermore, the Positional Embedding module performs convolution operations on the two input feature maps respectively.

[0014] A traffic scene target detection device based on improved RT-DETR, comprising: The image acquisition module is used to acquire images of different types of traffic scenes; The image detection module is used to input traffic scene images into the traffic scene target model to obtain the accurate localization of traffic targets and the category to which the traffic targets in the image belong. The traffic scene target model is an improvement on the RT-DETR model, including a feature extraction network Backbone module, a feature fusion network Neck module, and a feature recognition network Head module. The feature extraction network Backbone module integrates the WM-Dual Block module, which fuses wavelet transform and linear selective scanning. The feature fusion network Neck module adds the SQ-ACA module, which fuses output feature maps from different stages and performs attention interaction along the axis, on the basis of feature fusion. The WM-Dual Block module replaces the SPPF module in the Backbone. The SQ-ACA module is added before the stitching operation of feature maps at different scales.

[0015] A computer-readable storage medium having a computer program / instructions stored thereon, which, when executed by a processor, implement the steps of the method.

[0016] The advantages of this application over the prior art are as follows: This application proposes a traffic scene target detection model based on improved RT-DETR, which can improve the accuracy of traffic target detection, efficiently and accurately detect traffic target objects in different scenarios, and label their types, thus significantly improving efficiency. Attached Figure Description

[0017] The following description, in conjunction with the accompanying drawings, further illustrates this application: Figure 1 This is a flowchart of a traffic scene target detection method based on improved RT-DETR in one or more embodiments of this application; Figure 2 This is a schematic diagram of the structure of a traffic scene target detection model based on improved RT-DETR in one or more embodiments of this application; Figure 3 This is a schematic diagram of the WM-Dual Block module structure proposed in one or more embodiments of this application, which can effectively improve the feature extraction capability of the model. Figure 4 This is a schematic diagram of the SQ-ACA module structure proposed in one or more embodiments of this application, which can effectively improve the model's ability to select feature targets at different scales. Figure 5This is a schematic diagram of the training process of a traffic scene target detection model in one or more embodiments of this application; Figure 6 This is a schematic diagram of the target detection effect in a traffic scene in one embodiment of this application; Figure 7 This is a schematic diagram of the target detection results in a traffic scene using the traditional RT-DETR model; Figure 8 This is a schematic diagram of the target detection results in a traffic scene using the improved RT-DETR model according to the embodiments of this application. Detailed Implementation

[0018] like Figures 1 to 8 As shown, this application provides a traffic scene target detection method based on improved RT-DETR, including the following steps: Step 1: Use the BDD100K dataset as the traffic scene object detection dataset, and divide the test set and training set according to a 9:1 ratio. Different types of traffic scene objects include buses, cars, pedestrians, bicycles, trucks, motorcycles, trains, cyclists, traffic signs and traffic lights. Step 2: Construct a target detection model for traffic scenes; Step 3: Train the model using the training set images and save the parameters; Step 4: Input the test set images into the trained model to obtain the final traffic target detection results and label their categories.

[0019] Traffic scene images are RGB images. Different types of traffic scene images are captured by visual sensors mounted on vehicles on different types of roads. It should be noted that in order to ensure the computational speed and real-time performance of the traffic target detection and classification model, this embodiment needs to adjust the size of different types of traffic scene images according to the samples.

[0020] The traffic scene target detection model in this application is an improvement on the RT-DETR model, such as... Figure 2As shown, the network includes a Backbone module for feature extraction, a Neck module for feature fusion, and a Head module for feature recognition. The Backbone module integrates the WM-Dual Block (Wavelet-MambaDualPath Block) module, which fuses wavelet transform and linear selective scanning. The Neck module adds the SQ-ACA (Shared Query Axial CrossAttention) module, which fuses output feature maps from different stages and performs attention interaction along the axis, based on feature fusion. The SPPF module in the Backbone is replaced by the WM-Dual Block module. The SQ-ACA module is added before the stitching operation of feature maps at different scales.

[0021] like Figure 3 As shown, the WM-Dual Block module integrates wavelet transform and linear selective scanning to improve the model's feature extraction capabilities. It includes a first Split module, which divides the feature map into two information streams based on the number of channels, using a splitting factor 'a'. The first information stream is directly fed into the first Concat module without additional processing. The second information stream undergoes wavelet transform to obtain four feature maps of different frequencies: LL, LH, HL, and HH. Each feature map is processed by the selective scan module, and the processing result is fed into an inverse wavelet transform to obtain the feature map processed in the second information stream, which is then fed into the first Concat module. The first Concat module concatenates the two information streams along the channel direction to obtain the output feature map.

[0022] In one specific embodiment, the segmentation factor a in the WM-Dual Block module is 0.5.

[0023] In the WM-Dual Block module, the second information stream, after wavelet transform, yields four feature maps of different frequencies. Their size is reduced to half of the original size, while the number of channels remains unchanged.

[0024] The selective scan module in the WM-Dual Block module is used to duplicate the feature maps of four different frequencies twice. One copy is processed by the first Conv module and the SiLU activation function to obtain a weighted feature map; the other copy is processed by the second Conv module, the first SS2D module and the LayerNorm module to obtain a spatial scan feature map. Finally, the weighted feature map and the spatial scan feature map are multiplied to obtain the output feature map.

[0025] like Figure 4As shown, the SQ-ACA module can fuse feature maps of different scales output by the model to improve cross-scale feature fusion capabilities. The SQ-ACA module includes a Positional Embedding module, a Feature Alignment module, and a CrossPath Bridge module. The Positional Embedding module processes and fuses the two feature maps input to the SQ-ACA module to output an information stream. The Feature Alignment module then aligns the information stream. The aligned information stream is then processed by the CrossPath Bridge module, which extracts local spatial information based on Conv modules of varying degrees. Finally, the CrossPath Bridge module... The information stream output from the Bridge module is divided into five information streams by the third Conv module: Vx, Kx, Q, Ky, and Vy. Q interacts with Vx and Kx to obtain a horizontal attention feature map; Q interacts with Ky and Vy to obtain a vertical attention interaction map. At the same time, Q is added to both the horizontal and vertical attention interaction maps by the DWConv module. After addition, the two attention interaction maps are concatenated and integrated by the second Concat module. The concatenated fusion is further integrated and fused by the MLP module to obtain the output features.

[0026] The Positional Embedding module in the SQ-ACA module performs convolution operations on the two input feature maps respectively. The kernel size is 5.

[0027] In the Feature Alignment module of the SQ-ACA module, the first input feature map is first subjected to mean and max pooling, then a convolution with a kernel size of 7 is performed, and finally it is added to the second input feature map to achieve feature alignment and output the aligned feature map.

[0028] The CrossPath Bridge module in the SQ-ACA module uses multi-scale convolutions to operate on the alignment feature map. First, the second Split module splits the alignment feature map into four equal parts along the channel direction. The first feature map is processed by the fourth Conv module; the second feature map is processed by the fifth Conv module; the third feature map is processed by the sixth Conv module; and the fourth feature map is processed by the seventh Conv module. Then, the third Concat module merges the four feature maps into one along the channel direction. The fourth Conv module has a convolution size of 5 and a dilation of 1; the fifth Conv module has a convolution kernel size of 3 and a dilation of 1; the sixth Conv module has a convolution kernel size of 3 and a dilation of 2; and the seventh Conv module has a convolution kernel size of 3 and a dilation of 3.

[0029] The training process of the traffic target detection model in step three is as follows: Several different types of traffic scene images were acquired and labeled according to their categories; in the traffic scene images, all traffic targets were labeled with corresponding borders and corresponding labels were set to indicate different traffic targets; The traffic scene images in the training set are input into the traffic scene target detection model, which outputs the predicted bounding boxes of traffic targets in the corresponding traffic scene images and the category to which the traffic targets belong. The bounding boxes and category labels of traffic scene images are input into the traffic scene object detection model for training.

[0030] The boundary loss function used during training is the Complete Intersection over Union (CIoU) loss, which is expressed as follows:

[0031] Where: IoU represents the intersection-union ratio (IoU) between the predicted bounding box and the ground truth bounding box; ρ represents the Euclidean distance between the centers of the predicted and ground truth bounding boxes; b represents the coordinates of the center point of the predicted bounding box; b gt represents the coordinates of the center point of the actual bounding box; 'c' represents the diagonal distance between the closed regions of the two bounding boxes; 'v' represents the consistency of the relative proportions of the two rectangles; 'a' represents the weight coefficient. CIoU This represents the CIoU boundary loss function.

[0032] This embodiment provides a specific model training process, such as... Figure 5 As shown, the training process of the traffic scene target detection model includes: S1: Construct a target detection dataset for traffic scenes based on the BDD100K dataset; in which all traffic targets in each image have been labeled with corresponding bounding boxes and given semantic labels to distinguish different traffic target categories; S2: Input images of multiple traffic targets, including buses, cars, pedestrians, bicycles, trucks, motorcycles, trains, and cyclists, into the traffic scene target detection model. The model outputs the predicted bounding boxes of each target in the corresponding image and its category. S3: The traffic scene target detection model is trained under supervision using the labeled bounding boxes and category labels of the images. The bounding box regression adopts the Complete Intersection over Union (CIoU) loss function to improve the localization accuracy and convergence stability.

[0033] The parameters during training were set as follows: 100 iterations, initial learning rate of 0.001, minimum learning rate of 0.001, batch size of 16, Adam optimizer, and weight decay of 0.0005.

[0034] The traffic target detection results obtained in step four include the precise location coordinates of the traffic target area and the traffic target category.

[0035] This application also proposes a traffic scene target detection device based on an improved RT-DETR, including an image acquisition module and an image detection module; The image acquisition module is used to acquire images of different types of traffic scenes; The image detection module is used to input traffic scene images into the traffic scene target model to obtain the accurate localization of traffic targets and the category to which the traffic targets in the image belong.

[0036] The traffic scene target detection model is built on an improved RT-DETR model. The traffic scene target detection model includes a feature extraction network Backbone module, a feature fusion network Neck module, and a feature recognition network Head module. The Backbone module incorporates the self-developed WM-Dual Block module, which can effectively improve the model's feature extraction capability. The Neck module adds the SQ-ACA module, which improves the model's ability to detect falsely detected objects, based on feature fusion.

[0037] In a preferred embodiment of this application, this application also provides a storage medium, the storage medium including a stored computer program, wherein, when the computer program is running, it controls the device where the computer-readable storage medium is located to execute the traffic scene target detection method based on the improved RT-DETR proposed in this application.

[0038] It should be noted that the execution subject of the traffic scene target detection method based on the improved RT-DETR provided in this application can be an electronic device, a component in an electronic device, an integrated circuit, or a chip. The electronic device can be a mobile electronic device or a non-mobile electronic device. For example, mobile electronic devices can be mobile phones, tablets, laptops, PDAs, in-vehicle electronic devices, wearable devices, ultra-mobile personal computers (UMPCs), etc., while non-mobile electronic devices can be servers, network-attached storage (NAS), personal computers (PCs), etc., and this application does not impose specific limitations.

[0039] Figure 6 This is a schematic diagram illustrating the traffic scene object detection results of the improved RT-DETR model and the traditional RT-DETR model, as shown below. Figure 6 As shown, the image includes multiple JPG images, which contain various types of traffic targets. The traffic scene target detection model based on the improved RT-DETR has stronger recognition capabilities than the traditional RT-DETR model, and can capture small targets in dense scenes and targets in dark scenes.

[0040] Figure 7 This diagram illustrates the target detection results in a traffic scene using the traditional RT-DETR model. The average accuracy (AP) for cars is 83.5%, for buses 56.2%, for pedestrians 67.9%, for bicycles 50.6%, for trucks 59.6%, for motorcycles 42.7%, for cyclists 46.2%, for traffic signs 74.1%, and for traffic lights 71.1%. The mean average accuracy (mAP) for all categories is 55.2%.

[0041] Figure 8 This diagram illustrates the target detection results in a traffic scene using the improved RT-DETR model proposed in this application. The accuracy (AP) values ​​are as follows: cars 83.5%, buses 57.2%, pedestrians 68.1%, bicycles 51.9%, trucks 60.5%, motorcycles 46.0%, cyclists 46.7%, traffic signs 74.7%, and traffic lights 71.5%. The average mAP across all categories is 56.0%. It is evident that this application significantly improves the accuracy of target detection in traffic scenes across all categories, effectively validating its improvement effect on the traffic scene target detection task.

[0042] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. A traffic scene target detection method based on improved RT-DETR, characterized in that: Includes the following steps: Step 1: Construct and partition the traffic scene object detection dataset to obtain the test set and training set; Step 2: Construct a traffic scene target detection model. The traffic scene target detection model is based on the improved RT-DETR model and includes a feature extraction network Backbone module, a feature fusion network Neck module, and a feature recognition network Head module. The feature extraction network Backbone module integrates the WM-Dual Block module, which fuses wavelet transform and linear selective scanning. The Neck module of the feature fusion network adds the SQ-ACA module, which fuses the output feature maps from different stages and performs attention interactions along the axis, on the basis of feature fusion; the WM-Dual Block module replaces the SPPF module in the Backbone; the SQ-ACA module is added before the stitching operation of feature maps at different scales. The WM-Dual Block module includes a first Split module. After the first Split module, the feature map is divided into two information streams based on the number of channels according to the splitting factor 'a'. The first information stream is directly passed to the first Concat module without any additional processing. The second information stream is processed by wavelet transform to obtain four feature maps of different frequencies: LL, LH, HL, and HH. Each feature map is processed by the selective scan module. The processing result is then subjected to inverse wavelet transform to obtain the feature map of the second information stream, which is then passed to the first Concat module. The first Concat module concatenates the two information streams along the channel direction to obtain the output feature map. The SQ-ACA module comprises a Positional Embedding module, a Feature Alignment module, and a CrossPath Bridge module. The Positional Embedding module processes and fuses the two feature maps input to the SQ-ACA module to output an information stream. The Feature Alignment module then aligns the features of the information stream. The aligned information stream is then processed by the CrossPath Bridge module, which extracts local spatial information based on Conv modules of varying degrees. The information stream output from the CrossPath Bridge module is then divided into five information streams by the third Conv module: Vx, Kx, Q, Ky, and Vy. Q interacts with Vx and Kx to obtain a horizontal attention feature map; Q interacts with Ky and Vy to obtain a vertical attention interaction map. Simultaneously, Q is added to both the horizontal and vertical attention interaction maps via the DWConv module. The two attention interaction maps are then concatenated and integrated by the second Concat module. The spliced ​​and fused components are further integrated and fused through an MLP module to obtain the output features; Step 3: Train the model using the training set images and save the parameters; Step 4: Input the test set images into the trained model to obtain the final traffic target detection results and label their respective categories.

2. The traffic scene target detection method based on improved RT-DETR according to claim 1, characterized in that: The selective scan module is used to duplicate the four feature maps of different frequencies twice. One copy is processed by the first Conv module and the SiLU activation function to obtain a weighted feature map. The other copy is processed by the second Conv module, the first SS2D module and the LayerNorm module to obtain a spatial scan feature map. Finally, the weighted feature map and the spatial scan feature map are multiplied to obtain the output feature map.

3. A traffic scene target detection method based on improved RT-DETR according to claim 1, characterized in that: The Feature Alignment module first performs average and max pooling on the first input feature map, then performs convolution with a kernel size of 7, and finally adds it to the second input feature map to achieve feature alignment, outputting an aligned feature map.

4. A traffic scene target detection method based on improved RT-DETR according to claim 3, characterized in that: The CrossPath Bridge module uses multi-scale convolutions to operate on the alignment feature maps. First, the second Split module splits the alignment feature map into four equal parts along the channel direction. The first feature map is processed by the fourth Conv module; the second feature map is processed by the fifth Conv module; the third feature map is processed by the sixth Conv module; and the fourth feature map is processed by the seventh Conv module. Then, the third Concat module merges the four feature maps into one along the channel direction.

5. A traffic scene target detection method based on improved RT-DETR according to claim 4, characterized in that: The fourth Conv module has a convolution size of 5 and a dilation of 1. The fifth Conv module has a kernel size of 3 and a dilation of 1; the sixth Conv module has a kernel size of 3 and a dilation of 2; and the seventh Conv module has a kernel size of 3 and a dilation of 3.

6. A traffic scene target detection method based on improved RT-DETR according to claim 1, characterized in that: The Positional Embedding module performs convolution operations on the two input feature maps respectively.

7. A traffic scene target detection device based on improved RT-DETR, characterized in that: include: The image acquisition module is used to acquire images of different types of traffic scenes; The image detection module is used to input traffic scene images into the traffic scene target model to obtain the accurate localization of traffic targets and the category to which the traffic targets in the image belong. The traffic scene target model is an improvement on the RT-DETR model, including a feature extraction network Backbone module, a feature fusion network Neck module, and a feature recognition network Head module. The feature extraction network Backbone module integrates the WM-Dual Block module that fuses wavelet transform and linear selective scanning. The Neck module of the feature fusion network adds the SQ-ACA module, which fuses the output feature maps from different stages and performs attention interactions along the axis, on the basis of feature fusion; the WM-Dual Block module replaces the SPPF module in the Backbone; the SQ-ACA module is added before the stitching operation of feature maps at different scales. The WM-Dual Block module includes a first Split module. After the first Split module, the feature map is divided into two information streams based on the number of channels according to the splitting factor 'a'. The first information stream is directly passed to the first Concat module without any additional processing. The second information stream is processed by wavelet transform to obtain four feature maps of different frequencies: LL, LH, HL, and HH. Each feature map is processed by the selective scan module. The processing result is then subjected to inverse wavelet transform to obtain the feature map of the second information stream, which is then passed to the first Concat module. The first Concat module concatenates the two information streams along the channel direction to obtain the output feature map. The SQ-ACA module comprises a Positional Embedding module, a Feature Alignment module, and a CrossPath Bridge module. The Positional Embedding module processes and fuses the two feature maps input to the SQ-ACA module to output an information stream. The Feature Alignment module then aligns the features of the information stream. The aligned information stream is then processed by the CrossPath Bridge module, which extracts local spatial information based on Conv modules of varying degrees. The information stream output from the CrossPath Bridge module is then divided into five information streams by the third Conv module: Vx, Kx, Q, Ky, and Vy. Q interacts with Vx and Kx to obtain a horizontal attention feature map; Q interacts with Ky and Vy to obtain a vertical attention interaction map. Simultaneously, Q is added to both the horizontal and vertical attention interaction maps via the DWConv module. The two attention interaction maps are then concatenated and integrated by the second Concat module. The spliced ​​and fused components are further integrated and fused through the MLP module to obtain the output features.

8. A computer-readable storage medium having a computer program / instructions stored thereon, characterized in that: When the computer program / instructions are executed by the processor, they implement the steps of the method according to any one of claims 1-6.

Images