Saliency attention and unsupervised image enhancement double-driven target detection method

Abstract

This invention relates to a dual-driven target detection method combining saliency attention and unsupervised image augmentation. The aim is to improve the accuracy and stability of target detection models in complex and degraded environments by introducing salient region information and a target detection feedback mechanism. This method falls under the fields of computer vision and deep learning. It constructs an unsupervised image augmentation network to enhance the input image and guides the network to focus on optimizing target-related regions in the image through a dual saliency attention mechanism. Simultaneously, the target detection model is embedded in the overall training process, and the detection loss provides inverse constraints and optimizations to the image augmentation process. By combining unsupervised image augmentation and target detection tasks, this invention achieves adaptive optimization of the augmentation results for the detection task, avoiding the problem of traditional augmentation methods focusing only on visual quality while neglecting downstream tasks. It does not rely on paired high-quality reference images, exhibits good generalization ability and robustness, and is suitable for target detection applications in complex underwater visual environments.

Description

Technical Field

[0001] This invention relates to a target detection method driven by salient attention and unsupervised image enhancement. This method combines unsupervised image enhancement technology with target detection tasks, and achieves autonomous optimization of the image enhancement process through salient region guidance and detection feedback. It is particularly suitable for target detection applications in complex underwater degradation environments and belongs to the field of computer vision and deep learning technology. Background Technology

[0002] In practical visual perception applications, complex scenes such as underwater environments, low-light environments, and foggy environments commonly suffer from problems such as low image contrast, blurred details, and color distortion, which seriously affect the performance of subsequent visual tasks. Object detection, as an important research direction in computer vision, relies heavily on the quality of the input image for its detection accuracy. Therefore, how to improve image quality and enhance target detectability in complex environments has become a critical problem that urgently needs to be solved.

[0003] Existing image enhancement methods mainly rely on traditional image processing algorithms or supervised deep learning models. Traditional methods typically depend on manually designed rules, resulting in limited generalization ability; while supervised deep learning methods often require a large number of pairs of high-quality reference images, which are costly to obtain and lack adaptability to different scenarios. Therefore, unsupervised image enhancement methods have gradually become a research hotspot. However, existing unsupervised enhancement methods mostly aim to improve the overall visual quality of the image, ignoring salient region information closely related to the object detection task. This can easily lead to insufficient enhancement of the target region or excessive enhancement of the background region, thus affecting object detection performance.

[0004] On the other hand, object detection models are typically trained as independent modules, and their loss functions cannot be applied back to the image enhancement process, resulting in a lack of effective synergy between the enhancement results and the detection task. In existing technologies, image enhancement and object detection are often treated as two independent stages, making it difficult to achieve adaptive optimization for the detection task, which is particularly evident in complex degradation environments.

[0005] Therefore, how to introduce a target detection feedback mechanism without the need for paired supervised data, and how to use salient region information to guide the image enhancement process to achieve synergistic optimization of image enhancement and target detection, has become an urgent technical problem to be solved. Summary of the Invention

[0006] To address the aforementioned problems, this invention proposes an underwater target detection method based on saliency attention and unsupervised image enhancement. This method constructs an unsupervised image enhancement network and introduces a bisaliency attention mechanism into the network, enabling the model to focus on enhancing features in regions of the image relevant to target detection. Simultaneously, the target detection model is embedded into the overall training process, and the image enhancement network is subjected to inverse constraints through the detection loss, achieving adaptive optimization of the enhancement process for the detection task.

[0007] In this method, an image enhancement network enhances the input image under unsupervised conditions to generate an enhanced image; a target detection model performs detection prediction on the enhanced image and calculates the detection loss based on the prediction results. The detection loss and the image enhancement loss together constitute a joint optimization objective, which guides the image enhancement network to gradually adjust its parameters through a backpropagation mechanism, thereby significantly improving the target detection accuracy.

[0008] An underwater target detection method based on salient attention and unsupervised image enhancement is characterized by the following steps: (1) Obtain the degraded image to be processed, and preprocess the degraded image to construct a training set and a test set.

[0009] (2) Generate a saliency map of the degraded image using a saliency detection model.

[0010] (3) Construct an unsupervised image enhancement network based on UNet and embed a double saliency attention mechanism at the skip connections of the image enhancement network to enhance the salient region features related to object detection in the image.

[0011] The unsupervised image enhancement network first extracts features from the degraded image using an encoder to obtain a feature map. Then, the feature map and the saliency map are input together into a dual saliency attention mechanism. The feature map is weighted and modulated by generating spatial attention weights and channel attention weights to obtain a feature map with enhanced saliency regions. Finally, the feature map with enhanced saliency regions is input into a decoder for reconstruction to generate an image with enhanced saliency regions.

[0012] The dual saliency attention mechanism includes a saliency spatial attention mechanism and a saliency channel attention mechanism. This mechanism is used to guide an unsupervised image enhancement network to enhance salient regions of a degraded image by leveraging saliency maps. It jointly models the importance of image features in both spatial and feature channel dimensions, enabling the enhancement network to adaptively focus on salient regions and strengthen the expression of relevant features.

[0013] The saliency spatial attention mechanism performs max pooling and average pooling operations on the input feature map along the channel dimension to extract saliency statistics at different spatial locations. The two pooling methods respectively characterize the local strongest response and the overall average response features.

[0014] Subsequently, the pooled feature maps are concatenated along the channel dimension, and an initial spatial attention weight map is generated through convolution operations and a non-linear activation function.

[0015] In the process of generating the initial spatial attention weight map, a saliency map is introduced to modulate the saliency of the initial spatial attention weight map, so that the resulting weights not only reflect the statistical characteristics of the feature distribution, but also explicitly integrate the prior saliency information of the target region.

[0016] In this way, spatial attention weights can accurately characterize the importance of different spatial locations in an image, thereby significantly enhancing the response of the target region during feature enhancement, while effectively suppressing the interference of background regions and irrelevant noise on the enhancement results.

[0017] Meanwhile, the saliency channel attention mechanism is used to adaptively modulate different semantic responses in the channel dimension. This saliency channel attention mechanism first performs global average pooling and global max pooling operations on the input feature map in the channel dimension to obtain global contextual description information of each feature channel. Average pooling reflects the overall activation level of the channel, while max pooling emphasizes the most discriminative response in the channel.

[0018] Subsequently, the channel features obtained from the two pooling methods are input into a shared multilayer perceptron for feature transformation, and the output results are fused. Then, an initial channel attention weight map is generated through an activation function.

[0019] In this process, a saliency map is also introduced to modulate the initial channel attention weight map, giving greater response weights to feature channels highly correlated with target semantics and salient regions, while suppressing the responses of channels related to background or redundant information. Through this mechanism, the augmented network can highlight the expressive power of feature channels with strong discriminative ability for target detection tasks.

[0020] (4) Use the enhanced image output by the image enhancement network as the input of the target detection model to establish a joint training structure of the image enhancement network and the target detection model.

[0021] (5) During the joint training process, the detection loss is calculated based on the detection results of the target detection model, and the detection loss is backpropagated to the image enhancement network to guide the image enhancement network to optimize parameters.

[0022] (6) By using the jointly optimized image enhancement network and target detection model, the newly input degraded image is enhanced and the target is detected, thereby improving the target detection accuracy.

[0023] The dual saliency attention mechanism weights the feature map based on the spatial and channel dimensions of the saliency map to enhance the feature representation capability of the target-related region.

[0024] The loss function constructed during the joint training process includes perceptual loss and object detection loss, which are used to constrain the optimization direction of the image enhancement network.

[0025] The joint training strategy for the image enhancement network and the object detection network includes the following five steps: (1) Pre-train an image augmentation network to enable it to recover image structure and details; (2) Train the object detection network using the augmented image to adapt it to the augmented image; (3) The network is reinforced by inverse constraints of perception loss and detection loss, so that the reinforcement effect is more conducive to detection; (4) Train the target detection network to improve robustness; (5) Perform final joint training, specifically, train the augmentation network for 10 epochs and the target detection network for 1 epochs to achieve synergistic optimization of the augmentation network and the detection network.

[0026] The image data comes from complex underwater degradation scenes, and the preferred ratio of the training dataset to the test dataset is 3:1.

[0027] The preprocessing process includes operations such as image size normalization, pixel value normalization, and format conversion to meet the input requirements of subsequent network models.

[0028] The method is applied to target detection in complex underwater degraded environments.

[0029] Advantages of the invention This invention innovatively integrates unsupervised image enhancement technology with object detection tasks. By introducing a bisalience attention mechanism and detection feedback constraints, it achieves autonomous optimization of the image enhancement process. Compared to traditional image enhancement methods that only focus on visual quality, this invention starts from the requirements of object detection tasks, enabling the enhancement results to better represent the features of the target region and effectively improve object detection performance in complex degradation environments.

[0030] The unsupervised image enhancement method employed in this invention does not rely on paired high-quality reference images, avoiding the dependence of supervised methods on large amounts of manually labeled data and significantly reducing data acquisition costs. Simultaneously, through a dual-saliency attention mechanism, the enhancement network can adaptively distinguish between target and background regions, focusing on enhancing the clarity and contrast of target-related regions while avoiding over-enhancing of background regions, thereby improving the overall practicality of the enhancement effect.

[0031] Furthermore, this invention incorporates image enhancement into the target detection model training process and uses the detection loss to apply reverse constraints to the image enhancement network, enabling the image enhancement process to be dynamically adjusted based on the detection results. This joint optimization mechanism breaks through the traditional approach of training image enhancement and target detection independently, allowing them to work collaboratively within the same framework. This significantly improves the accuracy and stability of target detection while enhancing image quality.

[0032] Unlike traditional target detection methods that rely on human experience or fixed preprocessing strategies, this invention can automatically learn enhancement strategies suitable for the detection task through joint training, reducing the uncertainty caused by human intervention and improving the overall robustness and consistency of the system. In complex environments, this method can effectively reduce the probability of missed and false detections, and improve the reliability of detection results.

[0033] Furthermore, this invention possesses excellent versatility and scalability; the salient region image enhancement network and joint optimization strategy can be adapted to target detection models with different structures. Through the application of this invention, the adaptability and intelligence level of target detection systems in practical application environments can be significantly improved, providing an effective technical path for the engineering implementation of complex visual perception systems. Attached Figure Description

[0034] Figure 1 This is a flowchart of the present invention.

[0035] Figure 2 This is a diagram of the deep learning model architecture of the present invention.

[0036] Figure 3 This is a diagram of the saliency spatial attention architecture of the present invention.

[0037] Figure 4 This is a diagram of the saliency channel attention architecture of the present invention.

[0038] Figure 5 This is a flowchart of the bisaliency attention mechanism of the present invention. Detailed Implementation

[0039] This invention addresses the problem of existing unsupervised image enhancement methods neglecting detection task requirements by combining salient region guidance with detection feedback, achieving deep synergy between image enhancement and target detection. This method does not rely on paired high-quality reference images, exhibits good generalization ability and robustness, and can adapt to various complex degradation environments, significantly improving the overall performance and stability of the target detection system. A schematic diagram of the complete workflow framework of this invention is shown below. Figure 1 As shown.

[0040] An underwater target detection method based on salient attention and unsupervised image enhancement includes the following steps: Step 1. Obtain the degraded image data to be processed, and clean and preprocess the image data to construct the training dataset and the test dataset.

[0041] The image data comes from complex underwater degradation scenes, and the preferred ratio of the training dataset to the test dataset is 3:1.

[0042] The preprocessing process includes operations such as image size normalization, pixel value normalization, and format conversion to meet the input requirements of subsequent network models.

[0043] Step 2. Generate a saliency map of the degraded image using a saliency detection model.

[0044] Step 3. Construct a saliency spatial attention mechanism and a channel attention mechanism to guide the modeling of salient regions in image features.

[0045] The saliency spatial attention mechanism is used to weight image features in the spatial dimension, such as... Figure 3 As shown, by performing max pooling and average pooling on the input feature map along the channel dimension, saliency statistics of different spatial locations are extracted. The pooling results are then concatenated and subjected to convolution and nonlinear activation functions to generate a spatial attention weight map. Simultaneously, the spatial attention weights are modulated in conjunction with the saliency map, so that the weights can characterize the importance of different spatial locations in the image. This strengthens the response of the salient region where the target is located during feature extraction and suppresses irrelevant interference from the background region.

[0046] The saliency channel attention mechanism is used to adaptively modulate the feature response along the channel dimension, such as... Figure 4 As shown, global average pooling and global max pooling are performed on the input features respectively to obtain global descriptive information of each feature channel, and the pooled features are input into a nonlinear activation function to generate channel attention weights; then, the channel weights are further modulated in combination with the saliency map so that the feature channels related to the target semantics obtain higher response weights, thereby enhancing the discriminative feature channel expression ability for the target detection task.

[0047] The saliency spatial attention mechanism and the saliency channel attention mechanism work together to enhance image features, such as... Figure 5 As shown, by first weighting and modulating the feature map in the channel dimension to highlight the key feature channels related to salient regions, and then further recalibrating the weighted features in the spatial dimension to enhance the spatial response of salient target regions, a cascaded joint weighting of spatial and channel dimensions is achieved, forming a dual saliency attention mechanism. The weighted result is then fused with the original features to provide guiding information for feature enhancement in subsequent unsupervised image enhancement networks.

[0048] Step 4. Construct an unsupervised image enhancement network, and train the network using the training data from Step 1 in subsequent steps to enable it to enhance degraded images under unsupervised conditions, such as... Figure 2 As shown.

[0049] The unsupervised image enhancement network preferably adopts an encoder-decoder structure based on U-Net, which includes a feature extraction module (encoder) and a feature reconstruction module (decoder). The dual saliency attention mechanism designed in step 3 is introduced at the skip connections of the network to weight the feature map with saliency information, guiding the network to focus on enhancing the regions in the image that are related to target detection.

[0050] The dual saliency attention mechanism is based on the saliency map and modulates the spatial and channel dimensions of the feature map, thereby improving the feature representation capability of the target region.

[0051] Step 5. Using the training data from Step 1, jointly train the image enhancement network and the object detection model, so that the two can be optimized together in the process of continuous iteration.

[0052] The joint training process includes the following steps: (1) Pre-train an image augmentation network to enable it to recover image structure and details; (2) Train the object detection network using the augmented image to adapt it to the augmented image; (3) The network is reinforced by inverse constraints of perception loss and detection loss, so that the reinforcement effect is more conducive to detection; (4) Train the target detection network to improve robustness; (5) Perform final joint training. Specifically, train the augmentation network for one epoch and the target detection network for 10 epochs to achieve synergistic optimization of the augmentation network and the detection network.

[0053] Step 6. Use the trained image enhancement network and object detection model to enhance and detect objects in the new degraded image data.

[0054] In practical applications, newly acquired degraded images are input into the trained image augmentation network to obtain augmented images, which are then input into the target detection model to complete target detection prediction; no model parameter updates are performed during this process.

[0055] By combining salient region guidance and target detection feedback into a joint optimization mechanism, the image enhancement results can continuously adapt to the requirements of target detection tasks, thereby maintaining stable detection performance in complex environments, improving target detection accuracy, and reducing false detection and false negative rates.

[0056] Example (I) Data Acquisition and Preprocessing First, image data is acquired in a real, complex, and degraded environment, specifically from an underwater environment. The acquired image data includes target objects and their corresponding detection annotations. The target detection annotations are used solely for training and evaluating the target detection model, and the image enhancement process does not rely on paired high-quality reference images.

[0057] The collected image data requires standardized preprocessing operations, including image size normalization, pixel value normalization, and format conversion, to meet the input requirements of subsequent network models. Subsequently, the dataset is divided into training and testing datasets according to a preset ratio, preferably 3:1, to ensure the effectiveness of model training and validation.

[0058] (ii) Generate saliency maps of degraded images using a saliency detection model.

[0059] (III) Construction and Training of Unsupervised Image Augmentation Networks An unsupervised image enhancement network is constructed to enhance degraded images. In this embodiment, the image enhancement network preferably adopts an encoder-decoder based network structure, whose overall structure is similar to U-Net, including a downsampling feature extraction path and an upsampling feature reconstruction path.

[0060] A dual saliency attention mechanism is introduced at the skip connections of the network. This mechanism modulates the feature map based on the saliency map and weights the features in both the spatial and channel dimensions, enabling the network to focus on salient regions in the image that are relevant to target detection, thereby enhancing the ability to express the details of the target region.

[0061] During training, the image enhancement network is optimized under unsupervised conditions, and its loss function is perceptual loss.

[0062] (iv) Construction of target detection model An object detection model is constructed to perform object detection on the enhanced image. In this embodiment, different network architectures are used in the selected object detection model to verify the generalization ability of the designed enhancement network.

[0063] (v) Joint optimization of image enhancement and object detection An unsupervised image augmentation network and an object detection model are cascaded to construct a joint training framework. Specifically, the augmented image output by the image augmentation network is used as the input to the object detection model to detect and predict objects.

[0064] The target detection loss is calculated based on the output of the target detection model, and this loss is then used as a feedback signal to propagate back to the image enhancement network, forming a joint loss function together with the image enhancement loss. This joint loss function guides the image enhancement network to improve overall image quality while focusing on optimizing feature representations that are beneficial for target detection.

[0065] The joint training process includes the following steps: Pre-trained image augmentation networks enable them to recover image structure and details; (2) Train the object detection network using the augmented image to adapt it to the augmented image; (3) The network is reinforced by inverse constraints of perception loss and detection loss, so that the reinforcement effect is more conducive to detection; (4) Train the target detection network to improve robustness; (5) Perform final joint training. Specifically, train the augmentation network for one epoch and the target detection network for 10 epochs to achieve synergistic optimization of the augmentation network and the detection network.

[0066] (vi) Model testing and performance evaluation The performance of the jointly trained model was evaluated using a test dataset. Evaluation metrics included object detection accuracy metrics (such as...). mAP 50 , mAP 75 , mAP ,in, mAP 50 and mAP 75 The detection accuracy of the model was measured at different IoU thresholds. mAP This comprehensively reflects the model's overall detection performance under multiple positioning criteria with varying degrees of stringency.

[0067] As shown in Tables 1 and 2 below, compared with the target detection model without image enhancement modules, the method of this invention significantly improves target detection performance in complex underwater degradation environments, while significantly reducing the false negative and false positive rates. Taking the USIS10K dataset as an example, when SAG-UIENet is integrated into Faster R-CNN, the mAP increases from 21.9% to 23.0% (+1.1%). Similar performance improvements are also observed in FCOS (+3.2%), SSD (+1.3%), YOLOv5 (+0.5%), YOLOv8 (+0.8%), YOLOv12 (+1.2%), and RT-DETR (+0.7%), indicating that this method can bring stable gains under various detection frameworks and different detection paradigms. These results highlight the strong generalization ability of SAG-UIENet and verify its good adaptability to different backbone network structures and detection paradigms.

[0068] Table 1. Performance comparison of the object detector on the USIS10K and USIS16K datasets. Table 2. Performance comparison of the object detector on the UIIS and UIIS10K datasets. (vii) Practical application and effect description In practical applications, newly acquired degraded underwater images are input into a trained image enhancement network for enhancement processing. The enhanced images are then input into a target detection model to perform target detection prediction. No model parameter updates are performed during this process.

[0069] Through a joint optimization mechanism combining salient region guidance and detection feedback constraints, the image enhancement results can adaptively serve the target detection task, making the target region clearer and the edges more complete. Experimental results show that, even in underwater degradation environments, the method of this invention can consistently improve target detection accuracy and enhance the consistency and reliability of the detection results.

[0070] Compared with methods without joint optimization, the present invention shows significant advantages in both target detection accuracy and robustness, effectively verifying the practicality and feasibility of the method in complex visual scenes.

Claims

1. A target detection method driven by both saliency attention and unsupervised image enhancement, characterized by: Includes the following steps: (1) Obtain the degraded image to be processed, and preprocess the degraded image to construct a training set and a test set; (2) Generate a saliency map of the degraded image using a saliency detection model; (3) Construct an unsupervised image enhancement network based on UNet and embed a double saliency attention mechanism at the skip connections of the image enhancement network to enhance the salient region features related to object detection in the image; The unsupervised image enhancement network first extracts features from the degraded image through an encoder to obtain a feature map. Then, the feature map and the saliency map are input together into a dual saliency attention mechanism. The feature map is weighted and modulated by generating spatial attention weights and channel attention weights to obtain a feature map with enhanced saliency regions. Finally, the feature map with enhanced saliency regions is input into a decoder for reconstruction to generate an image with enhanced saliency regions. The dual saliency attention mechanism includes a saliency spatial attention mechanism and a saliency channel attention mechanism; Among them, the saliency spatial attention mechanism performs max pooling and average pooling operations on the input feature map in the channel dimension to extract saliency statistical information at different spatial locations. The two pooling methods respectively characterize the local strongest response and the overall average response features. Subsequently, the pooled feature maps are concatenated along the channel dimension, and an initial spatial attention weight map is generated through convolution operations and non-linear activation functions. In the process of generating the initial spatial attention weight map, a saliency map is introduced to modulate the saliency of the initial spatial attention weight map; Meanwhile, the saliency channel attention mechanism first performs global average pooling and global max pooling operations on the input feature map in the channel dimension to obtain global contextual description information for each feature channel. Average pooling reflects the overall activation level of the channel, while max pooling emphasizes the most discriminative response in the channel. Subsequently, the channel features obtained from the two pooling methods are input into a shared multilayer perceptron for feature transformation, and the output results are fused. Then, an initial channel attention weight map is generated through an activation function. In this process, a saliency map is also introduced to saliency modulate the initial channel attention weight map; (4) Use the enhanced image output by the image enhancement network as the input of the target detection model to establish a joint training structure of the image enhancement network and the target detection model; (5) During the joint training process, the detection loss is calculated based on the detection results of the target detection model, and the detection loss is backpropagated to the image enhancement network to guide the image enhancement network to optimize parameters; (6) By using the jointly optimized image enhancement network and target detection model, the newly input degraded image is enhanced and the target is detected, thereby improving the target detection accuracy.

2. The method according to claim 1, characterized in that, The dual saliency attention mechanism weights the feature map based on the spatial and channel dimensions of the saliency map to enhance the feature representation capability of the target-related region.

3. The method according to claim 1, characterized in that, The loss function constructed during the joint training process includes perceptual loss and object detection loss, which are used to constrain the optimization direction of the image enhancement network.

4. The method according to claim 3, characterized in that, The joint training strategy for the image enhancement network and the object detection network includes the following five steps: (1) Pre-train an image augmentation network to enable it to recover image structure and details; (2) Train the object detection network using the augmented image to adapt it to the augmented image; (3) The network is reinforced by inverse constraints of perception loss and detection loss, so that the reinforcement effect is more conducive to detection; (4) Train the target detection network to improve robustness; (5) Perform final joint training, specifically, train the augmentation network for 10 epochs and the target detection network for 1 epochs to achieve synergistic optimization of the augmentation network and the detection network.

5. The method according to claim 1, characterized in that, The degraded images are derived from complex underwater degraded scenes.

6. The method according to claim 1, characterized in that, The preferred ratio of the training dataset to the test dataset is 3:1; the preprocessing process includes operations such as image size normalization, pixel value normalization, and format conversion to meet the input requirements of the subsequent network model.

7. The method of claim 1 is applied to target detection in complex underwater degraded environments.

Images