Underwater target detection method based on irregular patch graph attention

Abstract

The invention discloses an underwater target detection method based on the attention of an irregular patch graph, and the method comprises the following steps: segmenting an original image into irregular patches through a superpixel segmentation algorithm, reducing the noise interference, and reducing the calculation cost; irregular patches are grouped into a plurality of clusters by using a clustering algorithm, a global graph is constructed among clustering centers, and local graphs are constructed in the clusters, so that interaction between global features and local features is facilitated; a graph attention mechanism is used on the global graph and the local graph, fusion of global features and local features is achieved, the fused features are input into a detection head for training optimization, and high-precision underwater target detection is achieved. According to the underwater target detection method provided by the invention, the characteristics of the underwater image are fully utilized, the underwater target detection method based on irregular patch graph attention is provided for the characteristics, and high-precision underwater target detection capable of adapting to a complex environment is realized.

Description

Technical Field

[0001] The present invention belongs to the technical field of underwater target detection, and in particular relates to an underwater target detection method based on irregular patch graph attention. Background Art

[0002] Underwater object detection is an important computer vision task that has become the focus of researchers in recent years. It plays an important role in marine resource management, environmental monitoring, and climate change research. Underwater image capture mainly relies on autonomous underwater vehicles, remotely controlled vehicles, and sonar. However, the captured underwater images have problems such as low contrast conditions, occlusion conditions, light imbalance conditions, and small density objects, which bring a series of challenges to underwater object detection and limit the performance of object detectors in practical applications. At present, deep learning-based methods are widely used in underwater object detection tasks.

[0003] Mainstream object detectors can be divided into two categories: one-stage object detectors and two-stage object detectors. One-stage object detectors, such as YOLO and SSD, aim to detect objects directly from a single network without any additional processing. This approach has a strong advantage in efficiency. However, one-stage models may have difficulty accurately detecting small objects or objects with complex shapes. In contrast, two-stage detectors take a different approach. The approach consists of two main stages: The first stage generates a set of region proposals. In the second stage, these proposals are refined and classified to obtain the final detection. Two-stage detectors usually have higher accuracy, especially in the case of small objects or cluttered backgrounds, due to the separate region proposal and refinement stages, but are usually slower than one-stage models. However, when many excellent general object detection frameworks are directly applied to underwater tasks, their performance is very limited due to the following challenges: (1) Light attenuates rapidly underwater, resulting in reduced contrast and clarity. Underwater objects are irregular in shape and of varying sizes, and usually include many small objects. Underwater objects often have occlusion problems.

[0004] The difference between underwater target detection and ordinary target detection lies in the complex underwater environment. First, underwater optical imaging is limited by the scattering and absorption of light, resulting in a decrease in image quality, especially in deep and turbid waters, where targets are often difficult to capture clearly. Secondly, as a commonly used detection tool, sonar has advantages in underwater environments with low visibility, but problems such as acoustic noise and echo interference still significantly affect the detection accuracy. Secondly, targets in underwater environments usually have complex morphological and motion characteristics, such as fish, submarines, or sunken ships. These targets vary in size, shape, material, and motion mode, making it difficult for traditional target detection algorithms to be efficient and robust in a changing underwater environment. In summary, there are four problems with underwater images: low contrast, occluded targets, unbalanced illumination, and small and dense targets.

[0005] Recent techniques have shown great progress in underwater object detection. RoIAttn adopts the idea of ​​patch-level attention to improve the performance of underwater object detection. However, this method does not fully consider the irregular shapes of underwater objects. Summary of the invention

[0006] In order to overcome the above problems, the present invention provides an underwater target detection method based on irregular patch graph attention.

[0007] The technical solution adopted by the present invention is:

[0008] An underwater target detection method based on irregular patch graph attention includes the following steps:

[0009] Step 1: Clustering-based irregular patch segmentation method: Use the superpixel segmentation method to segment the input underwater image into several irregular patches, and update the information of superpixels and patch boundaries through multiple clustering iterations; aggregate pixels with similar features to suppress noise and reduce computational complexity; this method, as a preprocessing step, helps the model better understand and process local information in the image, thereby improving the robustness and accuracy of detection;

[0010] Step 2: Graph construction method based on irregular patches: Based on the irregular patches generated in step 1, a clustering algorithm is used to cluster superpixels into clusters to reduce computational complexity and obtain global features; a local graph is constructed within the cluster and a global graph is constructed between clusters to facilitate the interaction between local and global features;

[0011] Step 3: Feature fusion method based on graph attention mechanism: Use the attention mechanism to fuse global features and local features on the global graph and local graph constructed in step 2, and optimize parameters through multiple training to improve the accuracy and robustness of the model.

[0012] The specific contents of step one are as follows:

[0013] For the input underwater image, all pixels are divided into two types: ordinary pixels and super pixels. The task of the irregular patch segmentation method is to transform each ordinary pixel P∈R H×W×C Assigned to M superpixels S∈R M×C one of the;

[0014] Among them, H, W, C represent the height, width and number of channels of the input image respectively; S i represents the superpixel of the i-th iteration;

[0015] First, the initial superpixels S are uniformly sampled in a regular grid 0 ∈R M×C ; The number of superpixels can be changed by changing the size of the grid; that is, if the size of the grid is a×b, then the number of superpixels is Then, the association map between ordinary pixels and superpixels is calculated in an iterative manner, and the tth iteration is expressed as:

[0016]

[0017] Among them, A t (ij) is the value at the i-th row and j-th column of the association map; since computing the association map of a normal pixel with all superpixels is computationally expensive, only the superpixels around each normal pixel are computed;

[0018] The features of superpixels and ordinary pixels are interacted again, and the superpixels are clustered to calculate the superpixel cluster center, which is the weighted sum of pixel features:

[0019]

[0020] After T iterations of the above two steps, the final association graph A is obtained t (ij) and superpixel center s t (j); Each superpixel and some normal pixels assigned to it form a patch, and there are M patches in total; by converting each superpixel into a feature vector These features are considered as a set of nodes, represented as Where D is the feature dimension, i = 1, 2,…, M.

[0021] The specific contents of step 2 are as follows:

[0022] In order to achieve multi-scale feature fusion and make the model perform well in complex underwater environments such as unbalanced light and low contrast, a clustering algorithm is used to capture the correlation of pixels in local areas and construct a local graph; the long-range dependency between patches is captured and a global graph is constructed between cluster centers;

[0023] First, the nodes are grouped into clusters with K nodes in each cluster; that is, C clusters are obtained, where The cluster center node set is represented as Each cluster center is then converted into a feature vector

[0024] For each cluster, add a local edge e between the superpixel and the cluster center local ; Get C local graphs G local =(V sp ,e local ) ; add global edges e between each cluster center global , and obtain the global graph G global =(V center ,e global ); The establishment of the global map and the local map is used to fuse the global features and the local features.

[0025] The specific contents of step three are as follows:

[0026] The feature three-way handshake module realizes the interaction of three features between the global graph and the local graph; the spatial attention mechanism is used in the graph attention network to realize the aggregation and distribution of features;

[0027] The first handshake of features is feature aggregation, which aggregates the features inside the cluster to the cluster center. Each cluster contains a cluster center node and K-1 superpixel nodes, and each superpixel node has an edge connected to the cluster center node. The process of feature aggregation is as follows:

[0028]

[0029] in, is the process of attention mechanism, It is a slave node The set of feature vectors collected from the neighbors of and They are the nodes before and after the update. The eigenvector of It is the cluster center node The set of neighbors of ;

[0030] The second feature handshake is global graph self-attention. The purpose of the second feature interaction is to achieve feature propagation between each cluster. Cluster centers are used to represent clusters and to achieve the second feature interaction between clusters. For the updated cluster center node features, All cluster center nodes V center are all its neighbors; so the second feature interaction is as follows:

[0031]

[0032] in, is the feature vector of the cluster center node after the second feature interaction; is the set of feature vectors of other cluster center nodes;

[0033] In this way, remote feature interaction is achieved; each cluster center node aggregates global and local information;

[0034] The third feature handshake is feature distribution. After global self-attention, each cluster center node aggregates global features. In order to enable other nodes in the local graph to obtain global features, the features of the cluster center node are distributed to other nodes in the cluster in the third feature interaction. For the cluster center node in the local graph and its adjacent nodes The process of feature distribution is as follows:

[0035]

[0036] in, and are the feature vectors of the superpixel nodes of the ith cluster before and after the update, yes

[0037] The feature vector of the cluster center to which it belongs; after FA, GSA and FD, each graph node aggregates global and local features.

[0038] The advantages of the present invention are as follows:

[0039] 1. This method can more accurately adapt to the natural boundaries and structures in the image. Through the graph structure, the model can better understand the mutual influence between different regions, thereby improving the overall analysis ability; this method is based on the division of significant regions in the image, which reduces the number of nodes that need to be processed, thereby reducing the computational complexity; this method can be adaptively adjusted according to the image content and the size of the target, which makes this method perform well when processing multi-scale targets.

[0040] 2. The feature three-way handshake method proposed in the present invention can enhance the model's adaptability to changes in target shape and posture, ensuring that the model can still accurately detect the target even if it is rotated, deformed or in a different background; this method integrates local and global features, which is more conducive to the model processing multi-scale targets.

[0041] 3. Compared with the existing underwater target detection algorithms, this algorithm proposes an underwater target detection method based on irregular patch graph attention. The model adopts corresponding feature extraction methods for complex underwater scenes, so that the model can have high accuracy and generalization performance in various complex underwater scenes. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0043] Figure 1 It is a flow chart of an underwater target detection method based on irregular patch graph attention according to the present invention;

[0044] Figure 2 It is a flow chart of a feature three-way handshake method in an underwater target detection method based on irregular patch graph attention described in the present invention;

[0045] Figure 3 This is a comparison chart of the detection effect under low contrast conditions of the UTDAC underwater dataset;

[0046] Figure 4 This is a comparison chart of the detection effect under the condition of unbalanced illumination of the UTDAC underwater dataset;

[0047] Figure 5 This is a comparison chart of the detection effect of small targets in the UTDAC underwater dataset;

[0048] Figure 6 This is a comparison chart of the enhancement effect of the UTDAC underwater dataset under the condition of target occlusion;

[0049] Figure 7 This is a comparison chart of the detection effect under low contrast conditions of the DUO underwater dataset;

[0050] Figure 8 This is a comparison chart of the detection effect under the unbalanced illumination conditions of the DUO underwater dataset;

[0051] Fig. 9This is a comparison chart of the detection effect of small targets in the DUO underwater dataset;

[0052] Fig.10 This is a comparison chart of the enhancement effect of the DUO underwater dataset under the condition of target occlusion;

[0053] Fig.11 This is a comparison chart of the detection effect under low contrast conditions of the RUOD underwater dataset;

[0054] Fig.12 This is a comparison chart of the detection effect under the condition of unbalanced illumination of the RUOD underwater dataset;

[0055] Fig.13 This is a comparison chart of the detection effect of small targets in the RUOD underwater dataset;

[0056] Fig.14 This is a comparison chart of the enhancement effect of the RUOD underwater dataset under the condition of target occlusion;

[0057] Fig.15 This is a comparison chart of the statistical results of the DUO underwater dataset;

[0058] Fig.16 This is a comparison chart of the statistical results of the UTDAC underwater data set;

[0059] Fig.17 This is a comparison chart of the statistical results of the RUOD underwater dataset;

[0060] Fig.18 This is a comparison chart of the statistical results of the Brackish, TrashCan, and WPBB underwater data sets. DETAILED DESCRIPTION

[0061] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0062] like Figure 1 As shown, an underwater target detection method based on irregular patch graph attention includes the following steps:

[0063] Step 1: Clustering-based irregular patch segmentation method:

[0064] For the input underwater image, all pixels are divided into two types: ordinary pixels and super pixels. The task of the irregular patch segmentation method is to transform each ordinary pixel P∈R H×W×CAssigned to M superpixels S∈R M×C one of the;

[0065] Among them, H, W, C represent the height, width and number of channels of the input image respectively; S i represents the superpixel of the i-th iteration;

[0066] First, the initial superpixels S are uniformly sampled in a regular grid 0 ∈R M×C ; You can change the number of superpixels by changing the size of the grid at this step; if the size of the grid is a×b, then the number of superpixels Then, the association map between ordinary pixels and superpixels is calculated in an iterative manner, and the tth iteration is expressed as:

[0067]

[0068] Among them, A t (ij) is the value at the i-th row and j-th column of the association map; since computing the association map of a normal pixel with all superpixels is computationally expensive, only the superpixels around each normal pixel are computed;

[0069] Since it is necessary to interact the features of superpixels and ordinary pixels again and cluster the superpixels, it is necessary to calculate the superpixel cluster center, which is the weighted sum of pixel features:

[0070]

[0071] After T iterations of the above two steps, the final association graph A is obtained t (ij) and superpixel center s t (j); Each superpixel and some normal pixels assigned to it form a patch, and there are M patches in total; by converting each superpixel into a feature vector These features are considered as a set of nodes, represented as Where D is the feature dimension, i = 1, 2,…, M.

[0072] Step 2: Graph construction method based on irregular patches:

[0073] In order to achieve multi-scale feature fusion and make the model perform well in complex underwater environments such as unbalanced light and low contrast, it is necessary to use clustering algorithms to capture the correlation of pixels in local areas and construct local graphs; in order to capture the long-range dependencies between patches, it is necessary to construct a global graph between cluster centers;

[0074] First, the nodes are grouped into clusters with K nodes in each cluster; this results in C clusters, where The cluster center node set is represented as Each cluster center is then converted into a feature vector

[0075] For each cluster, add a local edge e between the superpixel and the cluster center local ; Get C local graphs G local =(V sp ,e local ) ; add global edges e between each cluster center global , and obtain the global graph G global =(V center ,e global ); The establishment of the global map and the local map is used to fuse the global features and the local features.

[0076] Step 3: Feature fusion method based on graph attention mechanism:

[0077] like Figure 2 As shown in the figure, the feature three-way handshake module realizes the three-feature interaction between the global graph and the local graph; the spatial attention mechanism is used in the graph attention network to realize the aggregation and distribution of features;

[0078] like Figure 2 As shown in (a), the first handshake of features is feature aggregation, which aggregates the features inside the cluster to the cluster center; each cluster contains a cluster center node and K-1 superpixel nodes, and each superpixel node has an edge connected to the cluster center node; the process of feature aggregation is as follows:

[0079]

[0080] Among them, M is the process of attention mechanism, It is a slave node The set of feature vectors collected from the neighbors of and They are the nodes before and after the update. The eigenvector of It is the cluster center node The set of neighbors of ;

[0081] like Figure 2 As shown in (b), the second feature handshake is global graph self-attention. The purpose of the second feature interaction is to achieve feature propagation between each cluster. Therefore, cluster centers are used to represent clusters and to achieve the second feature interaction between clusters. For the updated cluster center node features All cluster center nodes V center are all its neighbors; so the second feature interaction is as follows:

[0082]

[0083] in, is the feature vector of the cluster center node after the second feature interaction; is the set of feature vectors of other cluster center nodes;

[0084] In this way, remote feature interaction is achieved; each cluster center node aggregates global and local information;

[0085] like Figure 2 As shown in (c), the third feature handshake is feature distribution. After global self-attention, each cluster center node aggregates global features. In order to enable other nodes in the local graph to obtain global features, the features of the cluster center node are distributed to other nodes in the cluster in the third feature interaction. For the cluster center node in the local graph and its adjacent nodes The process of feature distribution is as follows:

[0086]

[0087] in, and are the feature vectors of the superpixel nodes of the ith cluster before and after the update, yes The feature vector of the cluster center to which it belongs; after FA, GSA and FD, each graph node aggregates global and local features.

[0088] The present invention is applied in underwater image enhancement processing of public underwater image data sets:

[0089] The multi-frequency dual-branch underwater image enhancement method proposed in this invention is implemented by Python and runs on a Windows 10 64-bit system. The hardware configuration used in the experiment is an Intel i5-12400F processor, 32G memory, and an NVIDIA 2080TI graphics card. Figure 3-Figure 6 The comparison results of UTDAC underwater dataset and other advanced algorithms are given. Figure 7-10 The comparison results of DUO underwater dataset and other advanced algorithms are given. Figure 11-Figure 14 The comparison results of RUOD underwater image dataset and other advanced algorithms are given. Figure 15-18 The statistical results on each data set are given.

[0090] Note: Figure 3-Figure 14 The red arrows point to missed detection results, and the blue arrows point to wrong detection results; Figure 15-18The red font indicates the optimal value, and the blue font indicates the suboptimal value.

[0091] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. An underwater target detection method based on irregular patch graph attention, characterized by: The following steps are involved: Step 1: Clustering-based irregular patch segmentation method: Use the superpixel segmentation method to segment the input underwater image into several irregular patches, update the information of superpixel points and patch boundaries through multiple clustering iterations; aggregate pixels with similar features to suppress noise; Step 2: Graph construction method based on irregular patches: Based on the irregular patches generated in step 1, a clustering algorithm is used to cluster superpixels into clusters; a local graph is constructed within the cluster, and a global graph is constructed between clusters; Step 3: Feature fusion method based on graph attention mechanism: Use the attention mechanism on the global graph and local graph constructed in step 2 to fuse global features and local features, and optimize the parameters through multiple training.

2. The underwater target detection method based on irregular patch graph attention according to claim 1, characterized in that: The specific content of step one is as follows: For the input underwater image, all pixels are divided into two types: ordinary pixels and super pixels. The task of the irregular patch segmentation method is to transform each ordinary pixel P∈R H×W×C Assigned to M superpixels S∈R M×C one of the; Among them, H, W, C represent the height, width and number of channels of the input image respectively; S i represents the superpixel of the i-th iteration; First, the initial superpixels S are uniformly sampled in a regular grid 0 ∈R M×C ; The number of superpixels can be changed by changing the size of the grid; that is, if the size of the grid is a×b, then the number of superpixels is Then, the association map between ordinary pixels and superpixels is calculated in an iterative manner, and the tth iteration is expressed as: Among them, A t (ij) is the value at the i-th row and j-th column of the association map; The features of superpixels and ordinary pixels are interacted again, and superpixels are clustered to calculate the superpixel cluster center, which is the weighted sum of pixel features: After T iterations of the above two steps, the final association graph A is obtained t (ij) and superpixel center s t (j); Each superpixel and some normal pixels assigned to it form a patch, and there are M patches in total; by converting each superpixel into a feature vector These features are considered as a set of nodes, represented as Where D is the feature dimension, i = 1, 2,…, M.

3. The underwater target detection method based on irregular patch graph attention according to claim 2 is characterized in that: The specific content of step 2 is as follows: Clustering algorithms are used to capture the correlation of pixels in local areas and construct local graphs; long-range dependencies between patches are captured and a global graph is constructed between cluster centers; First, the nodes are grouped into clusters with K nodes in each cluster; that is, C clusters are obtained, where The cluster center node set is represented as Each cluster center is then converted into a feature vector For each cluster, add a local edge e between the superpixel and the cluster center local ; Get C local graphs G local =(V sp ,e local ) ; Add global edges e between each cluster center global , get the global graph G global =(V center ,e global )。 4. The underwater target detection method based on irregular patch graph attention according to claim 3 is characterized in that: The specific content of step three is as follows: The feature three-way handshake module realizes the interaction of three features between the global graph and the local graph; the spatial attention mechanism is used in the graph attention network to realize the aggregation and distribution of features; The first handshake of features is feature aggregation, which aggregates the features inside the cluster to the cluster center. Each cluster contains a cluster center node and K-1 superpixel nodes, and each superpixel node has an edge connected to the cluster center node. The process of feature aggregation is as follows: Among them, M is the process of attention mechanism, It is a slave node The set of feature vectors collected from the neighbors of and They are the nodes before and after the update. The eigenvector of It is the cluster center node The set of neighbors of ; The second feature handshake is the global graph self-attention. The purpose of the second feature interaction is to achieve feature propagation between each cluster. Cluster centers are used to represent clusters and to achieve the second feature interaction between clusters. For the updated cluster center node features All cluster center nodes V center are all its neighbors; so the second feature interaction is as follows: in, is the feature vector of the cluster center node after the second feature interaction; is the set of feature vectors of other cluster center nodes; In this way, remote feature interaction is achieved; each cluster center node aggregates global and local information; The third feature handshake is feature distribution. After global self-attention, each cluster center node aggregates global features. In order to enable other nodes in the local graph to obtain global features, the features of the cluster center node are distributed to other nodes in the cluster in the third feature interaction. For the cluster center node in the local graph and its adjacent nodes The process of feature distribution is as follows: in, and are the feature vectors of the superpixel nodes of the ith cluster before and after the update, yes The feature vector of the cluster center to which it belongs; after FA, GSA and FD, each graph node aggregates global and local features.

Images