A target detection method in dark environment

Abstract

This invention provides a method for object detection in low-light environments, comprising the following steps: 1) training a decomposition model to decompose illumination maps and reflectance maps; 2) introducing an attention mechanism to fuse depth information provided by millimeter-wave radar with the reflectance component to obtain object features; 3) introducing an attention fusion mechanism to fuse image features with object features; and 4) using the YOLO framework detection head to calculate and predict the target's position and category. This method not only provides a new perspective and technical path for object detection in low-light environments, but also demonstrates the tremendous potential of interdisciplinary technology integration in solving complex problems.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer vision target detection, and in particular relates to a target detection method in a dark environment. Background Art

[0002] Object detection technology is an extremely important research area in the field of computer vision, with a wide range of applications and enormous potential. It currently plays a significant role in everyday production and life, such as in autonomous driving, text processing, and facial recognition. With the advancement of scientific research, numerous object detection algorithms have been developed. The YOLO series of algorithms is currently one of the most widely used object detection algorithms. Upon its introduction, it garnered widespread attention for its rapid detection capabilities. After multiple iterations, the YOLO algorithm not only retains its advantage of fast detection speed, but also overcomes its weakness of low detection accuracy.

[0003] However, most existing object detection algorithms are studied only under the assumption of normal lighting conditions. Their performance is often significantly compromised in low-light scenarios, such as at night, in tunnels, and inclement weather. These scenarios require even more accurate object detection, yet current research on object detection in low-light scenarios is very limited. Compared to scenes with normal lighting, images in low-light scenarios exhibit many characteristics that negatively impact detection, such as low brightness, high noise, and significant color deviation. This leads to significant differences between low-light and normal-light scenes, meaning that algorithms designed for normal-light scenes often fail to effectively apply to low-light scenarios. However, the common task of object detection in low-light scenarios holds broad application prospects, such as in intelligent driving, surveillance, and drones in dark environments. However, current research on low-light images primarily focuses on designing efficient low-light enhancement algorithms, which employ artificial intelligence algorithms to visually enhance the brightness of images, resulting in clearer outlines and details of objects. While the aforementioned strategy can reduce the brightness difference between low-light and normal-light images, existing low-light enhancement algorithms inevitably lead to problems such as noise amplification and color deviation amplification. This prevents the subsequent feature extraction network from extracting high-quality features, ultimately resulting in suboptimal detection results. Furthermore, the cascaded implementation of independent enhancement and detection models is a non-end-to-end model architecture, which increases the computational complexity and reduces detection speed. Summary of the Invention

[0004] The goal of this invention is to overcome the shortcomings of traditional target detection methods in low-light environments by improving existing target detection algorithms. This improved method can effectively compensate for the limitations of pure images in low-light environments, effectively extracting the characteristic information of objects in the scene, while maintaining the efficiency of the end-to-end model. This significantly improves target detection performance in low-light environments while maintaining the algorithm's efficiency. This method has wide application in fields such as autonomous driving, security monitoring, and drones.

[0005] To achieve the above objectives, the present invention proposes a method for detecting a target in a dark environment, comprising the following steps:

[0006] Step 1) Train the decomposition model to decompose the illumination map and the reflectance map; introduce the decomposition model, analyze the input image, and decompose the illumination map and the reflectance map into illumination map components and reflectance map components;

[0007] Step 2) Introduce the attention mechanism to fuse the 2D point cloud data provided by the millimeter-wave radar with the reflection map component to obtain object features;

[0008] Step 3) Introduce the attention fusion mechanism, fuse the image features obtained by splicing the input image and the illumination map with the object features, and use the YOLO backbone network to process the image to extract the final image features;

[0009] Step 4) The YOLO framework detection head calculates the position and category of the predicted target; in the final target detection stage, the final image features are used to accurately calculate the position and category of the predicted target through the detection head.

[0010] The present invention has the following beneficial technical effects:

[0011] The present invention fully utilizes the advantage of the Retinex theory that the reflectance component of an image can reflect the characteristics of the object itself. By utilizing the depth information provided by the millimeter-wave radar to enhance the reflectance component of the image, the reflectance features in the image can be extracted more effectively and accurately. Such features are referred to as "object features" in the present invention. Finally, a pyramid feature generation mechanism and a self-attention feature fusion mechanism are used to effectively integrate these object features into the image feature extraction process of the YOLO target detection framework at multiple scale dimensions, thereby enhancing the image features in the original YOLO algorithm and improving the algorithm's detection accuracy for targets in low-light environments. This method not only provides a new perspective and technical path for target detection in low-light environments, but also demonstrates the great potential of interdisciplinary technology integration in solving complex problems. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 Schematic diagram of the process of the target detection method of the present invention;

[0013] Figure 2 This is a framework diagram of the target detection algorithm in the present invention;

[0014] Figure 3 Decompose the model diagram into the illumination map and reflection map;

[0015] Figure 4 This is the structure diagram of the object feature extraction module;

[0016] Figure 5 Schematic diagram of CSP structure;

[0017] Figure 6 Schematic diagram of the process of fusing body features and image features;

[0018] Figure 7 Schematic diagram of the self-attention fusion mechanism structure;

[0019] Figure 8 This is the architecture diagram of the dark light target detection system. DETAILED DESCRIPTION

[0020] The technical solution of the present invention will be further described below in conjunction with the accompanying drawings and embodiments. The following embodiments are only used to more clearly illustrate the technical solution of the present invention and are not intended to limit the scope of protection of this application.

[0021] Reference Figure 1 , Figure 2 , as one embodiment of the present invention, provides a method for detecting an object in a dark environment, comprising the following steps:

[0022] Step 1) First, we need to build an object detection dataset with millimeter-wave data. This dataset not only includes image data but also millimeter-wave radar data, providing a rich multimodal data resource. The images and millimeter-wave radar data in the dataset are captured synchronously and appear in pairs.

[0023] This dataset contains approximately 6,270 frames of data, each consisting of an image and corresponding millimeter-wave radar point cloud data. All data was collected in low-light environments, covering common indoor and outdoor scenes, ensuring the dataset's diversity and practicality. To support accurate object detection algorithm evaluation, the bounding box information of each human target in the dataset is carefully annotated.

[0024] The millimeter wave dataset is carefully divided into training and validation sets in a 4:1 ratio. 5016 frames of data are allocated as the training set for model learning and optimization, and the remaining 1254 frames of data are used as the validation set to evaluate the model's generalization ability and actual performance.

[0025] Step 2) Train the decomposition model to decompose the illumination map and the reflection map. Figure 3 As shown in the figure, a decomposition model of illumination map and reflectance map based on convolutional neural network is proposed. The decomposition model is composed of 5 layers of 3×3 convolutions containing ReLU activation functions, and a multi-head self-attention module MSA is introduced after the third convolution layer, so that the decomposition model can utilize the global information of the image. The attention module realizes attention encoding by evenly dividing the input vector into 64 blocks. Finally, a decoupling head module is introduced after the fifth convolution layer to obtain the final illumination map and reflectance map decomposition result. The decoupling head module is specifically composed of two parallel 3×3 convolutions with ReLU activation functions and a Sigmoid function. The structure of this module is shown in the figure. Figure 3 shown.

[0026] In order to train the decomposition model, it is necessary to create constraints on the illumination map and the reflection map. When training the decomposition model, it is necessary to input a bright light image at the same time. With a dark light picture , and the illumination component of the bright light image obtained by decomposing the model and the reflected component , and get the illumination component of the dark light picture at the same time and the reflected component Then the bright light image can be passed through or and Reconstruction. The reconstruction process can be expressed as:

[0027] ;

[0028] There are also corresponding:

[0029] ;

[0030] in, Representing the binary operation of the reconstruction operation, the following reconstruction loss function is introduced:

[0031] ;

[0032] At the same time, since the reflection component R is independent of illumination, the reflection component R of a paired bright light image and a dark light image should be as consistent as possible. Based on this conclusion, the following reflection consistency loss function is introduced:

[0033] ;

[0034] The final decomposition model loss function can be expressed as:

[0035] ;

[0036] in, is the loss function, which controls the proportion of reconstruction loss and reflection consistency loss in the overall loss. In this embodiment, The value of is 0.4, and the image reconstruction operation Defined as element-wise multiplication.

[0037] Step 3) introduces an attention mechanism to fuse the point cloud data provided by the millimeter-wave radar with the reflection component to obtain object features. To improve the accuracy and robustness of the target detection algorithm in low-light environments, the target detection algorithm framework proposed in this paper incorporates data provided by the millimeter-wave radar. Millimeter-wave radar, with its high resolution and penetrating power, can provide rich distance and velocity information for target detection. However, the raw data output by millimeter-wave radar is typically a three-dimensional point cloud, making direct processing of this data complex and computationally intensive. To effectively utilize this information and simplify the processing flow, this paper employs a method that maps 3D point cloud data to 2D data. The specific data mapping process consists of two parts: coordinate mapping and distance mapping. Coordinate mapping involves mapping the horizontal and vertical coordinates of the 3D point cloud data output by the radar to the horizontal and vertical coordinates of a 2D image. This step converts complex 3D spatial information into an easily processable 2D image format, significantly simplifying subsequent processing. Distance mapping involves converting the distance information in the point cloud data into grayscale values ​​in the image, preserving target distance information.

[0038] After coordinate mapping and distance mapping, a 2D image reflecting the millimeter wave radar point cloud data can be initially obtained. In order to further improve the quality of the image and reduce the influence of noise, the present invention also uses a Gaussian filter for image smoothing.

[0039] Through the aforementioned mapping method, the present invention effectively processes millimeter-wave radar data. This processing approach reduces data processing complexity while preserving key features provided by millimeter-wave radar data, such as distance and location information. This provides a new data source for image-based object detection technology. By fusing millimeter-wave radar data, this object detection method can improve detection accuracy and robustness in complex environments, particularly in low light or weather conditions.

[0040] After successfully acquiring the 2D mapping results of the millimeter-wave radar data and decomposing the illumination map and reflectance map components from the input image, the present invention innovatively incorporates the point cloud data acquired by the millimeter-wave device. This critical step enables the deep fusion of millimeter-wave detection information with the reflectance map, thereby generating comprehensive object features rich in essential attributes such as object distance and outline. This fusion process involves three computational stages: reflectance map and millimeter-wave feature extraction, feature fusion, and pyramid feature generation.

[0041] Specifically including: millimeter wave point cloud data mapping. The detection distance limit of the millimeter wave radar can be expressed as L, then the detection distance limits in the left and right directions, vertical directions, and front and back directions on the millimeter wave radar detection plane are expressed as 、 and Correspondingly, the point cloud coordinates in the three directions can be expressed as 、 and ,at the same time, satisfy , satisfy , satisfy Then, the resolution of the mapped target 2D image is defined as , then the process of left-right and vertical coordinate mapping can be expressed as:

[0042] ;

[0043] ;

[0044] The distance mapping process is expressed as:

[0045] ;

[0046] in, Function represents the rounding function. Is a constant, representing the maximum value of the image grayscale.

[0047] The deep fusion of millimeter-wave point cloud data and reflectance maps generates comprehensive object features rich in essential attributes such as object distance and outline. This fusion process consists of three computational stages: reflectance map and millimeter-wave feature extraction, feature fusion, and pyramid feature generation.

[0048] The first stage involves feature extraction from the reflectance map and millimeter-wave 2D radar data. Two parallel feature extraction networks are designed to mine the rich feature information contained in the reflectance map and millimeter-wave 2D radar data, respectively. The goal of this stage is to extract feature information that reflects the essential properties of the object from two different data sources: the image reflectance map and the millimeter-wave 2D radar data.

[0049] Next comes the feature fusion stage. This invention employs an advanced multi-head attention mechanism (MSA) that effectively integrates the reflectance map features with the millimeter-wave 2D radar data features. This approach not only preserves the unique information of each feature but also enables complementarity and enhancement between features, resulting in richer and more detailed object features.

[0050] Finally, a pyramid feature extraction structure is used to generate features of objects of different scales. This design enables the model of the present invention to flexibly adapt to the detection of objects of different sizes and distances, greatly improving the applicability and detection performance of the model.

[0051] The structural design of the entire feature extraction and fusion module is as follows Figure 4 Through this module, the present invention successfully achieves the goal of extracting and fusing key object features from the reflectance map component of the input image and the millimeter-wave 2D radar data, providing strong feature support for subsequent target detection and ensuring the performance of the model in low-light environments.

[0052] First, the point cloud data obtained by millimeter wave is mapped into a vector in 2D space using a plane mapping method, while retaining its original distance information through element values. In this process, a certain threshold filtering method is also used to remove noise. After obtaining the millimeter wave 2D radar data, it is fed into the feature extraction model in parallel with the reflection map. The reflection map and the millimeter wave 2D radar data use the same feature extraction model, and in order to extract features more efficiently, the present invention introduces a CSP structure into the feature extraction model. The schematic diagram of the CSP structure is shown in FIG. Figure 5 As shown in Figure 1, this architecture divides the input feature map into two parts: the foreground and the background. The foreground typically contains more important features, while the background contains less important features. These two feature maps are processed by different network stages. Afterwards, the two feature maps are fused together. This cross-cutting design enables the model to achieve better feature extraction capabilities. The feature extraction model includes a six-layer CSP architecture.

[0053] The reflection map features obtained by the feature extraction model are expressed as , the millimeter wave characteristics are expressed as , introduce a multi-head attention (MSA) to fuse it. First, we need to and According to The structure of the output vector group is obtained , and then set up 3 weight matrices that can be learned , such that:

[0054] ;

[0055] Among them, (Q, K, V) is a set of query vectors obtained by calculation, and then the attention result is obtained by multiplying Q and K. The attention result is the fused feature :

[0056] ;

[0057] in, is the square root of the first dimension of the weight matrix. , and Expand to multiple, that is, form a multi-head attention mechanism, and then splice the output results of each attention head with a learnable weight matrix Multiply and fuse the result into the input vector Output vectors of the same shape Finally, the output vector Reorganize the input into and , and then add the two together to get the final object feature .

[0058] After obtaining the object feature T, it is passed through Figure 4 The pyramid structure shown is mapped into features of different sizes in order to be integrated with image features at different stages. The structure contains several convolution layers and upsampling operations. First, the feature map convolution operation is performed from top to bottom to integrate stronger position information into the lower feature map, and then the feature map is upsampled from bottom to top. At the same time, the feature map of the previous stage is spliced ​​and integrated to integrate stronger semantic information into the high-level feature map, and finally three features of different sizes from large to small are obtained. .

[0059] Step 4) Introduce the self-attention fusion mechanism to fuse object features with image features.

[0060] This paper adopts the target detection enrichment based on the YOLO architecture as the basic target detection algorithm to achieve fast and accurate recognition of targets in images. The core of the YOLO architecture lies in its powerful backbone network, which is responsible for deep image feature extraction of the input image. The structure of this backbone network includes four key calculation stages, which can effectively capture image features from low-level to high-level to support complex target detection tasks. For the sake of simplicity, this paper marks the output results of these four calculation stages as ,in Represents the feature output of the i-th calculation stage.

[0061] In addition, in the object feature extraction process, object features at three scales were successfully extracted, which are represented as These object features are consistent in shape with the image features extracted by the YOLO backbone network except for the number of channels. This design makes the feature fusion process more direct and effective.

[0062] In order to achieve effective feature fusion, this paper introduces a fusion function , which is responsible for fusing object features with image features. With this function, the feature fusion results of each stage can be concisely expressed by the following formula:

[0063] ;

[0064] in, Represents the calculation result of the i-th stage. After the introduction of feature fusion, the input of the i-th calculation stage of the original YOLO backbone network is changed from the original Change to At the same time, the previously obtained illumination map is spliced ​​into the input image to obtain the image feature P to enhance the illumination semantics of the input image. First, the image feature P is input into the YOLO backbone network and a series of convolution operations are performed to obtain , and then compare it with the largest size object feature After the self-attention fusion function, we get ,and After convolution calculation, we get And in the next step with the object features of the second size Perform self-attention fusion to obtain Calculated Object features with minimum size Fusion to obtain fusion features , after calculation, we finally get the feature . The calculation steps of the four features are as follows Figure 6 As shown in the figure. In this way, not only the original feature information of the image is retained, but also additional object features provided by sensors such as millimeter wave radar are introduced, which greatly enriches the model's ability to perceive the target and improves the accuracy and robustness of target detection. This feature fusion strategy is particularly important in low-light conditions because it can significantly improve the model's detection performance for targets in low-light environments. The structural diagram of the fusion module is shown in the figure. Figure 7 shown.

[0065] Since object features focus more on the characteristics of the object itself, and image features focus more on the high-level semantics of the image, there will be structural differences between object features and image features. In order to more effectively fuse these two features, the present invention combines the designs of CA and EMA and designs a self-attention fusion mechanism that can fully consider the structural differences of the two features while fusing the features. It inputs the input vector into three parallel calculation paths, namely the vertical global path, the horizontal global path and the local path, which are used to capture the vertical, horizontal and local feature distribution information of the feature vector respectively, and then use this information to achieve the effect of self-attention fusion. On this basis, the present invention also introduces a cross-spatial attention mechanism, which allows the results of local attention and the results of global attention to be cross-calculated, and allows attention features of different scales to be further fused, so that the spatial semantic features are better evenly distributed in the feature vector.

[0066] The cross-spatial attention module accepts two input feature vectors and extracts attention weights for grouped feature maps through three parallel paths. The first two parallel paths are in the global attention branch, and the third path is in the local attention branch. In the global attention branch, a 1D global average pooling operation is used to encode channels along the vertical and horizontal spatial directions of the features. In the local attention branch, a single convolution with a 3×3 kernel is stacked to capture local inter-channel attention encoding. This attention not only encodes information between channels to adjust the importance of different channels, but also preserves precise spatial structure information within the channels.

[0067] This mechanism produces two attention results, one of which is the output of the global attention branch, and the other is the output of the local attention branch. In order to cross-fuse the results of the two branches, it is necessary to use 2D global average pooling and a 1×1 convolution to encode the global spatial information in the output of each branch. This global spatial information will be used to cross-fuse the calculation results of the two branches. At the same time, for efficient calculation, the natural nonlinear function Softmax of the 2D Gaussian map is used to fit the linear transformation at the output of the two branches. The fused attention result is obtained by cross-multiplying the global spatial information encoding result obtained above with the attention results of the two branches. Finally, the output attention fusion result of this module is the result of adding the two generated spatial attention weight values.

[0068] Step 5) The YOLO framework detection head accurately calculates and predicts the target's location and category. Using the YOLOv5 detection head, which employs a multi-scale prediction method, the model detects objects at different scales within the input image, adapting to objects of varying sizes. Prediction is performed using three detection heads of different scales, and the combination of a Feature Pyramid Network (FPN) and a Path Aggregation Network (PAN) effectively integrates feature information from all levels. This architecture enhances the model's detection performance for objects of varying sizes, particularly small and large ones.

[0069] Step 6) Train the algorithm model. The optimizer uses stochastic gradient descent (SGD), a commonly used optimization algorithm used to minimize the model's loss function during training. It improves model performance by iteratively updating the model's weights. The learning rate scheduler uses a cosine annealing strategy. This method gradually decreases the learning rate according to a cosine function, helping to smoothly adjust the learning rate in the later stages of training, avoiding oscillations during training and ultimately finding a better local minimum. The initial learning rate is set to 0.0032. This is the step size for each weight update at the beginning of training. The initial learning rate setting has a significant impact on the model's training speed and final performance. The final learning rate is set to 0.12. This is the learning rate the learning rate scheduler aims to achieve at the end of training. Combined with the initial learning rate, it forms the path of the learning rate over time. The momentum is set to 0.843. Momentum helps accelerate SGD optimization in relevant directions and suppress oscillations. It adjusts the update direction by weighted accumulation of past gradients, which can speed up training and improve efficiency. Weight decay is set to 0.00036. Weight decay is a regularization technique used to prevent model overfitting by slightly reducing the weight values ​​to make the model more robust. The warm-up period is set to 2.0. This is the initial stage of training, during which the learning rate is gradually increased from a small value to the initial setting, which helps stabilize the model in the early stages of training. The warm-up momentum is set to 0.5. During the warm-up period, the momentum value is gradually adjusted from this low starting value to the initial setting to help the model adapt to the learning process more smoothly in the early stages of training. The warm-up bias learning rate is set to 0.05. This is the learning rate used for the bias parameter during the warm-up period. It is usually set to a lower value than the weight learning rate to avoid over-adjusting the bias parameter at the beginning of training. The model was trained on the training set for 60 epochs. This process not only improved the model's detection capabilities in low-light environments, but also enhanced the model's ability to process millimeter-wave radar data.

[0070] Step 7) Deploy the trained target detection model to the terminal device to perform target detection and recognition in a realistic dark light environment. The architecture diagram of the dark light target detection system is shown in the figure below. Figure 8The prototype system for low-light object detection adopts a classic client / server architecture to achieve efficient data processing and user interaction. In this highly specialized system, the server handles the core task of model inference, while the client focuses on data collection, storage, and display.

Claims

1. A target detection method in a dark environment, characterized in that: The method comprises the following steps: Step 1) Train the decomposition model to decompose the illumination map and the reflectance map; introduce the decomposition model, analyze the input image, and decompose the illumination map and the reflectance map into illumination map components and reflectance map components; Step 2) Introduce the attention mechanism to fuse the 2D point cloud data provided by the millimeter-wave radar with the reflection map component to obtain object features; Step 3) Introduce the attention fusion mechanism, fuse the image features obtained by splicing the input image and the illumination map with the object features, and use the YOLO backbone network to process the image to extract the final image features; Step 4) The YOLO framework detection head calculates the position and category of the predicted target; in the final target detection stage, the final image features are used to accurately calculate the position and category of the predicted target through the detection head.

2. The method according to claim 1, characterized in that The decomposition model in step 1 is a light map reflectance map decomposition model based on a convolutional neural network, specifically: The decomposition model consists of five layers of 3×3 convolutions with ReLU activation functions. A multi-head self-attention module MSA is introduced after the third convolution layer, enabling the decomposition model to utilize the global information of the image. The attention module MSA realizes attention encoding by evenly dividing the input vector into 64 blocks. Finally, a decoupling head module is introduced after the fifth convolution layer to obtain the final illumination map and reflectance map decomposition result. The decoupling head module is specifically composed of two parallel 3×3 convolutions with ReLU activation functions and a Sigmoid function.

3. The method according to claim 2, characterized in that In step 1, the decomposition model is trained; at the same time, a bright light image is input With a dark light picture , and the illumination component of the bright light image obtained by decomposing the model and the reflected component , and get the illumination component of the dark light picture at the same time and the reflected component , then the bright light image passes through or and Reconstruction, the reconstruction process is expressed as: ； There are also corresponding: ； in, Representing the binary operation of the reconstruction operation, the following reconstruction loss function is introduced: ； The following reflection consistency loss function is introduced: ； The final decomposition model loss function is expressed as: ； in, is the loss function, which controls the proportion of reconstruction loss and reflection consistency loss in the overall loss respectively.

4. The method according to claim 1, wherein Step 2 includes three calculation stages, namely the reflection map and millimeter wave feature extraction stage, the feature fusion stage and the pyramid feature generation stage. In the reflection map and millimeter wave feature extraction stage, the 3D point cloud data output by the millimeter wave radar is mapped to 2D data and the feature extraction model is used to extract millimeter wave features; and another parallel feature extraction model is used to extract reflection map features from the reflection map component. In the feature fusion stage, a multi-head attention mechanism is introduced to fuse the emission map feature extraction results and the millimeter wave feature extraction results obtained in the previous stage. In the pyramid feature generation stage, the feature results obtained in the previous stage are mapped through the pyramid structure to generate multi-scale feature representations to obtain object features. .

5. The method according to claim 4, characterized in that Mapping 3D point cloud data to 2D data consists of two parts: coordinate mapping and distance mapping. Coordinate mapping refers to mapping the horizontal and vertical coordinates of the 3D point cloud data output by the radar to the horizontal and vertical coordinates of the 2D image. Distance mapping refers to converting the distance information in the point cloud data into the grayscale value of the image. It includes: The detection distance limit of the millimeter-wave radar is expressed as L, and the detection distance limits in the left and right directions, vertical directions, and front and back directions on the millimeter-wave radar detection plane are expressed as 、 and , correspondingly, the point cloud coordinates in the three directions are expressed as 、 and ,at the same time, satisfy , satisfy , satisfy , then, the resolution of the mapped target 2D image is defined as , the process of left-right and vertical coordinate mapping is expressed as: ； ； The distance mapping process is expressed as: ； in, Function represents the rounding function. Is a constant, representing the maximum value of the image grayscale.

6. The method according to claim 4, characterized in that The reflection map features obtained by the feature extraction model are expressed as , the millimeter wave characteristics are expressed as , introduce a multi-head attention MSA to fuse it; first and Perform structural division respectively to obtain the output vector group , then set the weight matrix , such that: ； Among them, (Q, K, V) is a set of query vectors obtained by calculation, and then the attention result is obtained by multiplying Q and K. The attention result is the fused feature : ； in, is the square root of the first dimension of the weight matrix, , and Expand to multiple, that is, form a multi-head attention mechanism, and then splice the output results of each attention head with a learnable weight matrix Multiply and fuse the result into the input vector Output vectors of the same shape , finally, the output vector Reorganize the input into and , and then add the two together to get the final object feature .

7. The method according to claim 4, characterized in that The self-attention fusion mechanism in step 3 is as follows: the input vector is fed into three parallel computation paths: a vertical global path, a horizontal global path, and a local path. These paths are used to capture the vertical, horizontal, and local feature distribution information of the feature vector, respectively. This information is then used to implement self-attention fusion. We further introduce a cross-spatial attention module, which accepts two given input feature vectors and extracts attention weights for grouped feature maps through three parallel paths. The first two parallel paths are on the global attention branch, and the third path is on the local attention branch. In the global attention branch, a 1D global average pooling operation is used to encode channels along the vertical and horizontal spatial directions of the features. In the local attention branch, only one 3×3 kernel convolution is stacked to capture the local inter-channel attention code; this module produces two attention results, one of which is the output of the global attention branch, and the other is the output of the local attention branch; 2D global average pooling and a 1×1 convolution are used to encode the global spatial information in the output of each branch. This global spatial information will be used to cross-fuse the calculation results of the two branches. At the same time, the natural nonlinear function Softmax of the 2D Gaussian mapping is used to fit the linear transformation at the output of the two branches. Finally, the output attention fusion result of this module is the result of adding the two generated spatial attention weight values.

Images