Camouflage target detection method and system based on hierarchical graph interaction and unit clustering

Abstract

The invention provides a camouflage target detection method and system based on hierarchical graph interaction and unit clustering, and the method comprises the steps: obtaining a to-be-detected image, converting the to-be-detected image into a set number of feature graph units through unit division and linear transformation, inputting all the feature graph units into a plurality of continuous region sensing unit focusing attention modules, and carrying out the recognition of the region sensing unit focusing attention modules; and removing redundancy based on the similarity of a self-attention mechanism by using a clustering algorithm, performing self-attention mechanism calculation on the screened feature map units, and outputting a first-level feature map. And inputting the first hierarchy feature maps output by the focusing attention modules of the adjacent region sensing units into a hierarchy map interaction self-attention module in pairs, converting the first hierarchy feature maps into map structures, performing hierarchy feature interaction, and projecting the interacted map structures back to the original space to output a second hierarchy feature map. And inputting the second-level feature maps into a confidence aggregation feature fusion decoder, refining an ambiguity region, fusing all the second-level feature maps, and outputting a camouflage target. The detection precision can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a camouflaged target detection method and system based on hierarchical graph interaction and unit clustering. Background Art

[0002] Camouflaged object detection is a technology in the field of computer vision that aims to identify and locate objects that are camouflaged or concealed in visual images. It requires a combination of multiple advanced computer vision and machine learning methods to effectively identify and locate objects that are cleverly camouflaged.

[0003] In recent years, deep learning technology has made remarkable progress in the task of camouflaged target detection. Most of these methods use attention feature distillation, multi-task joint learning and bionic vision strategies to develop convolutional neural networks, aiming to eliminate the ambiguity caused by the intrinsic similarity between the camouflaged target and the background area. However, the convolutional neural network-based methods rely on stacking convolution blocks with limited receptive fields and cannot utilize long-range information, resulting in poor performance in complex scenes with severe occlusion or difficult to distinguish boundaries. The current camouflaged target detection model uses a visual self-attention encoder to encode the camouflaged image to establish long-range relationships between different units, but this does not solve the problem of interference of the background environment in the camouflaged image on the recognition of the camouflaged target. At the same time, the existing camouflaged target detection model outputs feature maps of different scales, and there is a lack of interaction mechanism between these features to exchange visual semantic information in the feature map. As a result, the output camouflaged target result has poor accuracy and low detail clarity. Summary of the invention

[0004] In view of this, an embodiment of the present invention provides a camouflaged target detection method and system based on hierarchical graph interaction and unit clustering to eliminate or improve the defect in the prior art that the output hierarchical feature graph lacks an interaction mechanism, resulting in unclear details of the camouflaged target detection results.

[0005] A first aspect of the present invention provides a method for training a camouflaged target detection model based on hierarchical graph interaction and unit clustering, the method comprising the following steps:

[0006] Acquire a training sample set, the training sample set including a plurality of sample images, each sample image including one or more sample disguised targets, and mark the sample disguised targets as labels;

[0007] Acquire an initial neural network model, wherein the initial neural network model includes a plurality of continuous region perception unit focus attention modules, a plurality of hierarchical graph interactive self-attention modules and an aggregated feature fusion decoder;

[0008] Convert the sample image to a preset size and normalize it;

[0009] The processed sample image is converted into a set number of feature map units through unit division and linear transformation, and all feature map units are input into a plurality of continuous regional perception unit focus attention modules; each regional perception unit focus attention module uses a clustering algorithm to remove redundant feature map units based on the similarity of the self-attention mechanism, performs self-attention mechanism calculation on the feature map units after clustering screening, and outputs the first-level feature map corresponding to each regional perception unit focus attention module;

[0010] The first-level feature maps output by the adjacent regional perception unit focus attention module are input into the hierarchical graph interactive self-attention module in pairs; the hierarchical graph interactive self-attention module projects the two first-level feature maps into the latent space to obtain two corresponding first graph structures, performs hierarchical feature interaction on the two first graph structures based on the bidirectional alignment matrix, and obtains two corresponding second graph structures after the interaction; the second graph structure is projected back to the original space, and the second-level feature map is output;

[0011] Inputting all second-level feature maps into the confidence aggregation feature fusion decoder, refining the ambiguous areas in the second-level feature maps and fusing all second-level feature maps, and outputting the detected sample disguised target;

[0012] The initial neural network model is trained using the training sample set, and the loss is constructed according to the label and the prediction result to update the parameters of the initial neural network model to obtain a disguised target detection model.

[0013] In some embodiments of the present invention, each regional perception unit focus attention module uses a clustering algorithm to remove redundant feature map units based on the similarity of the self-attention mechanism, performs self-attention mechanism calculation on the feature map units after clustering screening, and outputs the first-level feature map corresponding to each regional perception unit focus attention module. The specific steps include:

[0014] The region perception unit focused attention module projects each feature map unit into corresponding query units, key units, and value units using a linear projection matrix;

[0015] Calculating the similarity between the query unit and the key unit;

[0016] The K nearest neighbor algorithm is used to obtain multiple neighboring units of each feature map unit and calculate the local density and the intimacy between the feature map unit and the neighboring units;

[0017] Based on the local density and the intimacy of each feature map unit, multiple cluster centers with the highest density are selected, all feature map units are classified based on the distances between other feature map units and the cluster centers, and redundant feature map units whose distances to the cluster centers are greater than a preset value are removed;

[0018] The key unit and value unit pairs corresponding to the feature map units after clustering filtering are concatenated, and the self-attention mechanism is calculated for the feature map units after clustering filtering to output the corresponding first-level feature map.

[0019] In some embodiments of the present invention, a self-attention mechanism is calculated for the feature map units after clustering screening, and the specific steps include:

[0020] Calculate the similarity between each query unit and all key units, and use the softmax function to convert the similarity into a probability distribution as the weight corresponding to each key unit;

[0021] The weights are used to perform weighted summation on all value units, and the corresponding first-level feature map is output.

[0022] In some embodiments of the present invention, the hierarchical graph interactive self-attention module projects the first hierarchical feature graph into the latent space to obtain a corresponding first graph structure, and the specific steps include:

[0023] Unifying the first-level feature map into a preset size using a bilinear interpolation method;

[0024] The first-level feature map of a preset size is input into two 1*1 convolutional layers to obtain two matrices, and the two matrices are multiplied to obtain the first map structure.

[0025] In some embodiments of the present invention, the calculation formula of the bidirectional alignment matrix includes:

[0026]

[0027] Among them, S i→(i+1) Indicates that the first graph structure Mapping to the first graph structure The information interaction alignment matrix, S (i+1)→i Indicates that the first graph structure Mapping to the first graph structure The information interaction alignment matrix, θ1 and θ2 represent learnable transformation parameters.

[0028] In some embodiments of the present invention, projecting the second graph structure back to the original space and outputting a second-level feature graph includes:

[0029] Calculate the Laplacian matrix of the second graph structure, and use the Laplacian matrix of the second graph structure as a position code of the self-attention module;

[0030] After the second graph structure is input into the self-attention module, the second-level feature map is output through a 1*1 convolution layer.

[0031] In some embodiments of the present invention, all second-level feature maps are input into a confidence aggregation feature fusion decoder, the ambiguous regions in the second-level feature maps are refined and all second-level feature maps are fused, and the detected camouflaged target is output. The specific steps include:

[0032] Input all second-level feature maps into the confidence aggregation feature fusion decoder, extract features from all second-level feature maps through multiple sets of continuous convolution layers, batch normalization layers, and activation function layers to generate a first prediction result;

[0033] multiplying the first prediction result by the complement of the first prediction result to highlight the ambiguous region to obtain a second prediction result;

[0034] The second prediction result is input into a plurality of groups of continuous convolutional layers, batch normalization layers and activation function layers for refinement processing, and the detected camouflaged target is output.

[0035] A second aspect of the present invention provides a camouflaged target detection method based on hierarchical graph interaction and unit clustering, the method comprising:

[0036] The image to be detected is input into a disguised target detection model such as any of the above-mentioned disguised target detection model training methods based on hierarchical graph interaction and unit clustering, and a disguised target is output.

[0037] The third aspect of the present invention provides a camouflaged target detection method system based on hierarchical graph interaction and unit clustering, comprising a processor, a memory and a computer program stored in the memory, wherein the processor is used to execute the computer program, and when the computer program is executed, the system implements the steps of any of the methods described above.

[0038] A fourth aspect of the present invention provides a computer program product, comprising a computer program, which implements the steps of any of the methods described above when executed by a processor.

[0039] The beneficial effects of the present invention are at least:

[0040] The present invention provides a camouflaged target detection method and system based on hierarchical graph interaction and unit clustering, the method comprising: obtaining an image to be detected, converting it into a set number of feature map units through unit division and linear transformation, inputting all feature map units into multiple continuous regional perception unit focus attention modules, removing redundancy based on the similarity of the self-attention mechanism by using a clustering algorithm, performing self-attention mechanism calculation on the screened feature map units, and outputting a first-level feature map. The first-level feature maps output by the adjacent regional perception unit focus attention modules are input into the hierarchical graph interaction self-attention module in pairs, converted into a graph structure, and then hierarchical feature interaction is performed, and the graph structure after interaction is projected back to the original space to output a second-level feature map. The second-level feature map is input into a confidence aggregation feature fusion decoder, the ambiguous region is refined and all the second-level feature maps are fused, and the camouflaged target is output. The present invention uses unit clustering to suppress the interference of the background on the camouflaged target, and at the same time realizes the interaction and alignment between the hierarchical feature maps output by the encoder, so that the voice information can be fully utilized, the accuracy of camouflaged target detection is improved, and the details of the camouflaged target obtained by the method are clearer.

[0041] Additional advantages, purposes, and features of the present invention will be described in part in the following description, and will become apparent to those skilled in the art after studying the following, or may be learned from the practice of the present invention. The purposes and other advantages of the present invention may be achieved and obtained by the structures specifically indicated in the specification and the accompanying drawings.

[0042] Those skilled in the art will appreciate that the objectives and advantages that can be achieved with the present invention are not limited to the above specific description, and the above and other objectives that can be achieved by the present invention will be more clearly understood from the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The drawings described herein are used to provide a further understanding of the present invention, constitute a part of the present application, and do not constitute a limitation of the present invention. In the drawings:

[0044] Figure 1 The figure is a flow chart of training a camouflaged target detection model based on hierarchical graph interaction and unit clustering in one embodiment of the present invention.

[0045] Figure 2 The figure is a flowchart of an execution of a camouflaged target detection method based on hierarchical graph interaction and unit clustering in one embodiment of the present invention.

[0046] Figure 3 FIG. 4 is a flowchart of an overall execution of a camouflaged target detection method in another embodiment of the present invention. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments and the accompanying drawings. Here, the illustrative embodiments of the present invention and their descriptions are used to explain the present invention, but are not intended to limit the present invention.

[0048] It should also be noted that, in order to avoid obscuring the present invention due to unnecessary details, only structures and / or processing steps closely related to the solutions according to the present invention are shown in the accompanying drawings, while other details that are not closely related to the present invention are omitted.

[0049] It should be emphasized that the term “include / comprises” when used herein refers to the presence of features, elements, steps or components, but does not exclude the presence or addition of one or more other features, elements, steps or components.

[0050] It should also be noted that, unless otherwise specified, the term “connection” herein may refer not only to a direct connection but also to an indirect connection involving an intermediate.

[0051] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. In the accompanying drawings, the same reference numerals represent the same or similar components, or the same or similar steps.

[0052] An embodiment of the present invention provides a method for training a camouflaged target detection model based on hierarchical graph interaction and unit clustering, such as Figure 1 As shown, the training method includes the following steps S101 to S106:

[0053] Step S101: Obtain a training sample set, the training sample set includes multiple sample images, each sample image includes one or more sample disguised targets, and the sample disguised targets are marked as labels. Obtain an initial neural network model, the initial neural network model includes multiple continuous regional perception unit focus attention modules, multiple hierarchical graph interactive self-attention modules and an aggregated feature fusion decoder.

[0054] Step S102: converting the sample image into a preset size and performing normalization processing.

[0055] Step S103: converting the processed sample image into a set number of feature map units through unit division and linear transformation, and inputting all feature map units into a plurality of continuous regional perception unit focus attention modules; each regional perception unit focus attention module uses a clustering algorithm to remove redundant feature map units based on the similarity of the self-attention mechanism, performs self-attention mechanism calculation on the feature map units after clustering screening, and outputs the first-level feature map corresponding to each regional perception unit focus attention module;

[0056] Step S104: The first-level feature maps output by the adjacent regional perception unit focus attention module are input into the hierarchical graph interactive self-attention module in pairs; the hierarchical graph interactive self-attention module projects the two first-level feature maps into the latent space to obtain two corresponding first graph structures, and performs hierarchical feature interaction on the two first graph structures based on the bidirectional alignment matrix to obtain two corresponding second graph structures after the interaction. The second graph structure is projected back to the original space, and the second-level feature map is output.

[0057] Step S105: All second-level feature maps are input into the confidence aggregation feature fusion decoder, the ambiguous areas in the second-level feature maps are refined and all second-level feature maps are fused, and the detected sample disguised targets are output.

[0058] Step S106: Use the training sample set to train the initial neural network model, build the loss according to the label and the prediction result, and update the parameters of the initial neural network model to obtain a disguised target detection model.

[0059] Another aspect of this embodiment provides a camouflaged target detection method based on hierarchical graph interaction and unit clustering, such as Figure 2 As shown, an image to be detected containing a disguised target is input into a disguised target detection model obtained by a disguised target detection model training method based on hierarchical graph interaction and unit clustering. The execution process of the method includes the following steps S201 to S204:

[0060] Step S201: converting the image to be detected into a preset size and performing normalization processing.

[0061] Step S202: The processed image to be detected is converted into a set number of feature map units through unit division and linear transformation, and all feature map units are input into multiple continuous regional perception unit focus attention modules. Each regional perception unit focus attention module uses a clustering algorithm to remove redundant feature map units based on the similarity of the self-attention mechanism, performs self-attention mechanism calculation on the feature map units after clustering screening, and outputs the first-level feature map corresponding to each regional perception unit focus attention module.

[0062] Step S203: Pair the first-level feature maps output by the adjacent regional perception unit focus attention module into the hierarchical graph interactive self-attention module. The hierarchical graph interactive self-attention module projects the two first-level feature maps into the latent space to obtain two corresponding first graph structures, and performs hierarchical feature interaction on the two first graph structures based on the bidirectional alignment matrix to obtain two corresponding second graph structures after the interaction. The second graph structure is projected back to the original space, and the second-level feature map is output.

[0063] Step S204: All second-level feature maps are input into the confidence aggregation feature fusion decoder, the ambiguous areas in the second-level feature maps are refined and all second-level feature maps are fused, and the detected camouflaged targets are output.

[0064] Among them, unit division is to divide the input image to be detected into multiple small blocks, and treat these small blocks as sequence data for processing.

[0065] Among them, the feature map is a high-dimensional and low-resolution representation, which reflects the features extracted from the image to be detected after linear transformation.

[0066] Among them, the level graph represents the hierarchy of different features, for example, from low-level features (such as edges) to high-level features (such as shapes).

[0067] In some embodiments of the present invention, each regional perception unit focus attention module uses a clustering algorithm to remove redundant feature map units based on the similarity of the self-attention mechanism, performs self-attention mechanism calculation on the feature map units after clustering screening, and outputs the first-level feature map corresponding to each regional perception unit focus attention module, specifically including the following steps S2021 to S2025:

[0068] Step S2021: The regional perception unit focus attention module uses a linear projection matrix to project each feature map unit into a corresponding query unit, key unit, and value unit.

[0069] Step S2022: Calculate the similarity between the query unit and the key unit.

[0070] Specifically, the calculation formula for the similarity between the query unit and the key unit is:

[0071]

[0072] in, Represents the attention score matrix, i.e., the similarity between the query unit and the key unit. Indicates a linear mapping of the query unit. Represents a linear mapping of key units.

[0073] Step S2023: Use the K nearest neighbor algorithm to obtain multiple adjacent units of each feature map unit and calculate the local density and the intimacy between the feature map unit and the adjacent units.

[0074] Step S2024: Based on the local density and intimacy of each feature map unit, multiple cluster centers with the highest density are selected, all feature map units are classified based on the distance between other feature map units and the cluster centers, and redundant feature map units whose distance to the cluster centers is greater than a preset value are removed.

[0075] Step S2025: concatenate the key unit and value unit pairs corresponding to the feature map units after clustering and filtering, perform self-attention mechanism calculation on the feature map units after clustering and filtering, and output the corresponding first-level feature map.

[0076] In some embodiments of the present invention, a self-attention mechanism is calculated for the feature map units after clustering screening, and the specific steps include:

[0077] Calculate the similarity between each query unit and all key units, and use the softmax function to convert the similarity into a probability distribution as the weight corresponding to each key unit;

[0078] Use the weights to perform weighted summation on all value units and output the corresponding first-level feature map.

[0079] Among them, the softmax function converts these similarities into probability distribution. The Softmax function is a commonly used activation function used to convert a set of real numbers into a probability distribution, that is, to convert similarities into a probability distribution so that each output value is between 0 and 1, and the sum of all output values ​​is 1.

[0080] In some embodiments of the present invention, the hierarchical graph interactive self-attention module projects the first-level feature graph into the latent space to obtain the corresponding first graph structure, and the specific steps include:

[0081] Use bilinear interpolation to unify the first-level feature map into a preset size;

[0082] The first-level feature map of the preset size is input into two 1*1 convolutional layers to obtain two matrices, and the two matrices are multiplied to obtain the first graph structure.

[0083] In some embodiments of the present invention, the calculation formula of the bidirectional alignment matrix includes:

[0084]

[0085] Among them, S i→(i+1) Indicates that the first graph structure Mapping to the first graph structure The information interaction alignment matrix, S (i+1)→i Indicates that the first graph structure Mapping to the first graph structure The information interaction alignment matrix, θ1 and θ2 represent learnable transformation parameters.

[0086] Specifically, the alignment matrices in two directions are multiplied by two first graph structures that need to interact with hierarchical features, respectively, to obtain two corresponding second graph structures after the interaction.

[0087] In some embodiments of the present invention, projecting the second graph structure back to the original space and outputting a second-level feature graph includes:

[0088] The Laplacian matrix of the second graph structure is calculated, and the Laplacian matrix of the second graph structure is used as the position encoding of the self-attention module.

[0089] After the second graph structure is input into the self-attention module, it passes through a 1*1 convolutional layer to output the second-level feature map.

[0090] Specifically, the Laplacian matrix is ​​calculated from the adjacency matrix and the degree matrix. The adjacency matrix represents the connection relationship between nodes in the graph structure. If there is an edge between two nodes, the corresponding position in the matrix is ​​1, otherwise it is 0. The edge information in the graph structure is the similarity between nodes. The degree matrix is ​​a diagonal matrix, in which each diagonal element represents the degree of the corresponding node, that is, the number of edges connected to the node.

[0091] The Laplacian matrix is ​​calculated from the adjacency matrix and the degree matrix. Its definition is usually: L = DA. D is the degree matrix and A is the adjacency matrix.

[0092] Using the Laplacian matrix as the position encoding of the self-attention module can help the self-attention module better understand the topological structure of the graph, thereby retaining important spatial information during the encoding process.

[0093] In some embodiments of the present invention, all second-level feature maps are input into a confidence aggregation feature fusion decoder, the ambiguous regions in the second-level feature maps are refined and all second-level feature maps are fused, and the detected camouflaged target is output. The specific steps include:

[0094] Input all second-level feature maps into the confidence aggregation feature fusion decoder, extract features from all second-level feature maps through multiple sets of continuous convolution layers, batch normalization layers, and activation function layers to generate a first prediction result;

[0095] The first prediction result is multiplied by the complement of the first prediction result to highlight the ambiguous area, thereby obtaining a second prediction result.

[0096] The second prediction result is input into multiple groups of consecutive convolutional layers, batch normalization layers, and activation function layers for refinement, and the detected camouflaged target is output.

[0097] Among them, the ambiguous area is the fuzzy area between the camouflaged target and the background.

[0098] Another embodiment of the present invention provides a camouflaged target detection method based on hierarchical graph interactive self-attention and dynamic unit clustering, which can detect imperceptible objects through effective interaction between hierarchical features and has better performance than other advanced methods. Figure 3 As shown, the overall execution process of this method is as follows:

[0099] 1) Image preprocessing: The image containing the camouflaged object is input into the data preprocessing module. First, the image is scaled to a fixed size of 512*512; then the image is normalized to make the data distribution of the image adapt to the pre-trained parameters of the visual self-attention backbone network.

[0100] 2) Feature extraction and background suppression: The image is input into the region-aware unit-focused attention backbone network (RTFA) to mine potential distinguishable units in the local area, discard irrelevant units to suppress background interference, and obtain hierarchical feature maps of different sizes.

[0101] 3) Hierarchical feature alignment and interaction: The feature maps of adjacent layers are input into the hierarchical graph interaction self-attention module (HGIT), the feature maps are projected into a graph structure in the latent interaction space, and a bidirectional communication between hierarchical features is constructed in the latent interaction space. The self-attention mechanism is used to enhance the visual semantics in the graph structure.

[0102] 4) Fine Segmentation Decoding: A decoder network (Fusion) with a confidence aggregation feature fusion module is used to refine the fuzzy regions with low confidence and gradually fuse hierarchical interaction features to achieve more accurate and fine segmentation.

[0103] Furthermore, the specific execution modules and algorithms in the execution process are described in detail as follows:

[0104] 1. Efficient camouflage detection algorithm based on hierarchical graph interactive self-attention mechanism

[0105] In order to solve the problems of intrinsic similarity and insufficient hierarchical aggregation in the disguised target detection task, this embodiment proposes an efficient disguised target detection algorithm based on a hierarchical graph interactive self-attention mechanism. The algorithm first adopts a region-aware unit focused attention (RTFA) with dynamic unit clustering to mine potential distinguishable units in local areas, and deletes irrelevant background units to suppress the interference of background areas that are very similar to the disguised target on the segmentation. Then, a hierarchical graph interactive self-attention (HGIT) is used to construct bidirectional communication between hierarchical features in the latent interaction space to enhance visual semantics. Finally, a decoder network with a confidence aggregation feature fusion module is used to refine the local details of the blurred area. A large number of experimental results on the disguised target detection benchmark dataset show that the efficient disguise detection algorithm based on the hierarchical graph interactive self-attention mechanism proposed in this embodiment has better performance than other advanced methods.

[0106] 2. Region-aware focused attention (RTFA) and hierarchical graph interactive self-attention (HGIT) modules

[0107] Since the disguised target and the background environment have inherent similarities, directly using the self-attention mechanism to establish long-range dependencies will inevitably introduce irrelevant interference from the background, resulting in poor segmentation output for distinguishing the disguised target. To solve this problem, this embodiment proposes a region-aware unit focused attention (RTFA) module that allows the model to use a dynamic unit clustering strategy to mine potential distinguishable units, such as Figure 3 As shown in (A).

[0108] For the feature map input to the module, the module first divides it into s*s non-overlapping units and projects these units into query units (Q), key units (K) and value units (V) using a linear projection matrix. Subsequently, the query unit and the key unit are averagely pooled (P) and the similarity (A) is calculated to measure the overall semantic association between the query and key units.

[0109]

[0110] Afterwards, a dynamic unit clustering method based on DPC-KNN (Density Peak Clustering K-Nearest Neighbors) is used to discard the redundancy of units. Specifically, the K nearest neighbor algorithm is used to obtain the first k neighbor units of each unit, and its local density ρ is calculated and the closeness δ between the unit and its neighboring units is measured. δ represents the distance between unit p and the nearest region with higher local density.

[0111]

[0112] This embodiment defines the score as ρ*δ. The higher the score, the richer the information and visual semantics provided by unit p for the recognition of camouflaged targets. Then, the k cluster centers with the highest scores are selected to construct the discriminant cluster unit, and their key-value unit pairs are connected in series to strengthen the unit. Finally, this embodiment performs a self-attention mechanism on the strengthened query unit, key unit, and value unit. It is worth noting that when calculating the above scores and selecting the k units with the highest scores, the model effectively improves the model's ability to distinguish between foreground and background, and suppresses the interference of irrelevant background areas on the detection of camouflaged targets.

[0113] Although the region-aware unit focused attention can construct a discriminative disguised target representation by dynamically suppressing irrelevant key-value units, it still lacks a hierarchical interaction mechanism to exchange visual semantics in multiple region-aware unit focused attention blocks. Furthermore, this embodiment projects the hierarchical feature graphs into a graph structure in the latent space, uses soft attention to align them, and adopts hierarchical graph interactive self-attention to exchange graph information in adjacent levels, assisting the model in building long-range dependencies between graph nodes and realizing two-way message communication. That is, the interaction of the hierarchical graph is realized through the hierarchical graph interactive self-attention (HGIT) module, and the execution process of this module is as follows: Figure 3 As shown in (B), it specifically includes the following steps:

[0114] First, a latent space graph projection operation is performed on the hierarchical feature map. For the multi-scale hierarchical features output by the regional perception unit focus attention module, this embodiment first uses bilinear interpolation to unify them to the same size. The feature maps of uniform size are then input into two 1*1 convolutions respectively to reduce the dimension and aggregate features in different channels. The matrices obtained by the two convolutions are then multiplied to obtain the graph structure in the latent space; among them, the matrix multiplication results in the graph node information, and the edge information of the graph is obtained by calculating the similarity between the graph nodes.

[0115] Secondly, the graph information of the latent space is interactive at the hierarchical level. This embodiment adopts a simple and effective interactive method to create alignment and communication between hierarchical graphs. For the graph node information of adjacent graphs, a learnable transformation is used θ encodes the graph nodes V and obtains their alignment matrix S by calculating the transformed similarity, which indicates the complementary visual semantics corresponding to the hierarchical feature maps.

[0116]

[0117] The alignment matrix is ​​multiplied with the original graph node to obtain the updated graph node information, and residual connection is used to ensure the validity of the learned alignment matrix.

[0118] Finally, the self-attention module is used to model the aligned and interacted graph nodes to mine their high-order dependencies. Specifically, the Laplacian matrix is ​​first calculated through the graph adjacency matrix and the point degree matrix, and the Laplacian matrix is ​​used as the position encoding of the self-attention module to better preserve the structural information of the graph. The learned graph node information containing high-order dependencies will be projected back to the original space through 1*1 convolution.

[0119] 3. Confidence Aggregation Feature Fusion Decoder

[0120] Since disguised target detection has its own unique intrinsic similarity, it is difficult for the model to have a high confidence prediction at each position of the image, making it difficult to obtain accurate segmentation results. Therefore, this embodiment proposes a confidence aggregation feature fusion decoder to refine the local details of the ambiguous area and gradually fuse the hierarchical features. Specifically, this embodiment first uses a series of convolutions, batch normalization, and activation functions to predict the results and obtain the prediction result P; then, this embodiment multiplies P by (1-P) to highlight those uncertain areas in the middle (Note: Disguised target detection is a binary classification task. If the result in P is closer to 1, the model believes that the area is more likely to be the foreground, and the closer it is to 0, the model believes that the area is more likely to be the background area). Subsequently, this embodiment performs a series of convolutions, batch normalization, and activation functions to refine the highlighted fuzzy areas to achieve fine-tuning of the prediction results. The experimental results on the public disguised target detection dataset show that the performance of the algorithm proposed in this embodiment is better than other algorithms.

[0121] In summary, the present invention provides a camouflaged target detection method and system based on hierarchical graph interaction and unit clustering, the method comprising: obtaining an image to be detected, converting it into a set number of feature map units through unit division and linear transformation, inputting all feature map units into multiple continuous regional perception unit focus attention modules, using a clustering algorithm to remove redundancy based on the similarity of the self-attention mechanism, performing self-attention mechanism calculation on the filtered feature map units, and outputting a first-level feature map. The first-level feature maps output by the adjacent regional perception unit focus attention module are input into the hierarchical graph interaction self-attention module in pairs, converted into a graph structure, and then hierarchical feature interaction is performed, and the graph structure after interaction is projected back to the original space to output a second-level feature map. The second-level feature map is input into the confidence aggregation feature fusion decoder, the ambiguous area is refined and all the second-level feature maps are fused, and the camouflaged target is output. The present invention uses unit clustering to suppress the interference of the background on the camouflaged target, and at the same time realizes the interaction and alignment between the hierarchical feature maps output by the encoder, so that the voice information can be fully utilized, the accuracy of camouflaged target detection is improved, and the details of the camouflaged target obtained by the method are clearer.

[0122] Corresponding to the above method, the present invention also provides a system, which includes a computer device, the computer device includes a processor and a memory, the memory stores computer instructions, the processor is used to execute the computer instructions stored in the memory, and when the computer instructions are executed by the processor, the system implements the steps of the method described above.

[0123] The embodiment of the present invention also provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the steps of the aforementioned edge computing server deployment method are implemented. The computer-readable storage medium can be a tangible storage medium, such as a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a floppy disk, a hard disk, a removable storage disk, a CD-ROM, or any other form of storage medium known in the technical field.

[0124] It should be understood by those skilled in the art that the exemplary components, systems and methods described in conjunction with the embodiments disclosed herein can be implemented in hardware, software or a combination of the two. Whether it is performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present invention are programs or code segments used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link via a data signal carried in a carrier.

[0125] It should be clear that the present invention is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present invention is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present invention.

[0126] In the present invention, features described and / or illustrated for one embodiment may be used in the same or similar manner in one or more other embodiments, and / or combined with features of other embodiments or replace features of other embodiments.

[0127] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the embodiments of the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A camouflage target detection model training method based on hierarchical graph interaction and unit clustering, characterized in that: The method comprises the following steps: Acquire a training sample set, the training sample set including a plurality of sample images, each sample image including one or more sample disguised targets, and mark the sample disguised targets as labels; Acquire an initial neural network model, wherein the initial neural network model includes a plurality of continuous region perception unit focus attention modules, a plurality of hierarchical graph interactive self-attention modules and an aggregated feature fusion decoder; Convert the sample image to a preset size and normalize it; The processed sample image is converted into a set number of feature map units through unit division and linear transformation, and all the feature map units are input into a plurality of continuous regional perception unit focusing attention modules; The focused attention module of each regional perception unit uses a clustering algorithm to remove redundant feature map units based on the similarity of the self-attention mechanism, performs self-attention mechanism calculation on the feature map units after clustering screening, and outputs the first-level feature map corresponding to the focused attention module of each regional perception unit; Input the first-level feature maps output by the adjacent region perception unit focus attention module into the level map interactive self-attention module in pairs; The hierarchical graph interaction self-attention module projects two first-level feature graphs into the latent space to obtain two corresponding first graph structures, performs hierarchical feature interaction on the two first graph structures based on the bidirectional alignment matrix, and obtains two corresponding second graph structures after the interaction; the second graph structures are projected back to the original space, and the second-level feature graph is output; Inputting all second-level feature maps into the confidence aggregation feature fusion decoder, refining the ambiguous regions in the second-level feature maps and fusing all second-level feature maps, and outputting the prediction result of the sample disguised target; The initial neural network model is trained using the training sample set, and the loss is constructed according to the label and the prediction result to update the parameters of the initial neural network model to obtain a disguised target detection model.

2. The method for training a camouflaged target detection model based on hierarchical graph interaction and unit clustering according to claim 1, characterized in that: Each regional perception unit focus attention module uses a clustering algorithm to remove redundant feature map units based on the similarity of the self-attention mechanism, performs self-attention mechanism calculation on the feature map units after clustering screening, and outputs the first-level feature map corresponding to each regional perception unit focus attention module. The specific steps include: The region perception unit focused attention module projects each feature map unit into corresponding query units, key units, and value units using a linear projection matrix; Calculating the similarity between the query unit and the key unit; The K nearest neighbor algorithm is used to obtain multiple neighboring units of each feature map unit and calculate the local density and the intimacy between the feature map unit and the neighboring units; Based on the local density and the intimacy of each feature map unit, multiple cluster centers with the highest density are selected, all feature map units are classified based on the distances between other feature map units and the cluster centers, and redundant feature map units whose distances to the cluster centers are greater than a preset value are removed; The key unit and value unit pairs corresponding to the feature map units after clustering filtering are concatenated, and the self-attention mechanism is calculated for the feature map units after clustering filtering to output the corresponding first-level feature map.

3. The method for training a camouflaged target detection model based on hierarchical graph interaction and unit clustering according to claim 2, characterized in that: The self-attention mechanism is calculated for the feature map units after clustering screening. The specific steps include: Calculate the similarity between each query unit and all key units, and use the softmax function to convert the similarity into a probability distribution as the weight corresponding to each key unit; The weights are used to perform weighted summation on all value units, and the corresponding first-level feature map is output.

4. The method for training a camouflaged target detection model based on hierarchical graph interaction and unit clustering according to claim 1, characterized in that: The hierarchical graph interactive self-attention module projects the first hierarchical feature graph into the latent space to obtain a corresponding first graph structure, and the specific steps include: Unifying the first-level feature map into a preset size using a bilinear interpolation method; The first-level feature map of a preset size is input into two 1*1 convolutional layers to obtain two matrices, and the two matrices are multiplied to obtain the first map structure.

5. The method for training a camouflaged target detection model based on hierarchical graph interaction and unit clustering according to claim 1, characterized in that: The calculation formula of the bidirectional alignment matrix includes: Among them, S i→(i+1) Indicates that the first graph structure Mapping to the first graph structure The information interaction alignment matrix, S (i+1)→i Indicates that the first graph structure Mapping to the first graph structure The information interaction alignment matrix, θ1 and θ2 represent learnable transformation parameters.

6. The method for training a camouflaged target detection model based on hierarchical graph interaction and unit clustering according to claim 1, characterized in that: The second graph structure is projected back to the original space, and the second-level feature graph is output, including: Calculate the Laplacian matrix of the second graph structure, and use the Laplacian matrix of the second graph structure as a position code of the self-attention module; After the second graph structure is input into the self-attention module, the second-level feature map is output through a 1*1 convolution layer.

7. The method for training a camouflaged target detection model based on hierarchical graph interaction and unit clustering according to claim 1, characterized in that: Inputting all second-level feature maps into the confidence aggregation feature fusion decoder, refining the ambiguous areas in the second-level feature maps and fusing all second-level feature maps, and outputting the prediction result of the disguised target, the specific steps include: Input all second-level feature maps into the confidence aggregation feature fusion decoder, extract features from all second-level feature maps through multiple sets of continuous convolution layers, batch normalization layers, and activation function layers to generate a first prediction result; multiplying the first prediction result by the complement of the first prediction result to highlight the ambiguous region to obtain a second prediction result; The second prediction result is input into a plurality of groups of continuous convolution layers, batch normalization layers and activation function layers for refinement processing, and a prediction result of the disguised target is output.

8. A camouflaged target detection method based on hierarchical graph interaction and unit clustering, characterized in that: The method comprises: The image to be detected is input into the disguised target detection model in the disguised target detection model training method based on hierarchical graph interaction and unit clustering as described in any one of claims 1 to 7, and the disguised target is output.

9. A method system for detecting camouflaged targets based on hierarchical graph interaction and unit clustering, comprising a processor, a memory, and a computer program stored in the memory, characterized in that: The processor is used to execute the computer program. When the computer program is executed, the system implements the steps of the method according to any one of claims 1 to 7.

10. A computer program product comprising a computer program / instructions, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images