A method for infrared small target detection based on edge topology guidance

Abstract

This invention relates to the fields of image processing and machine vision applications, and provides a method for detecting small infrared targets based on edge topology guidance. This method aims to eliminate intra-class distances through target edge detection and edge topology, introduce an edge topology feature layer into the segmentation network, and thus improve the model's detection performance for various targets. The specific steps are: first, edge detection is performed on the input infrared image; then, the edges are topologically transformed to construct a candidate graph structure composed of multiple subgraphs. The network then generates an edge topology feature layer containing both image features and graph topology features; this layer is then introduced into the segmentation network to participate in model training, resulting in the final detection model.

Description

Technical Field

[0001] The present invention relates to the fields of image processing and machine vision applications, and provides an infrared small target detection method based on edge topology guidance, especially the problem of small target detection under infrared imaging, which is of great significance to the military defense system. Background Art

[0002] Infrared small target detection is the core of infrared search and tracking systems and a crucial technology in military defense systems. Automatic detection and defense systems for targets with odd shapes, such as drones, are also crucial. However, current academic research has paid scant attention to these odd-shaped small targets, with more focus on detecting long-range elliptical targets. Consequently, current mainstream algorithms, such as deep learning, low-rank sparse algorithms, and saliency algorithms, are insufficiently capable of detecting these targets. Infrared small target detection can be viewed as a binary classification problem involving target and background. Most small infrared targets have similar features, often forming an elliptical shape. However, in near- and medium-range infrared imaging, odd-shaped small targets can appear in various shapes due to varying observation angles or their inherent morphology. This can significantly differentiate them from the principal component targets, even though they fall within the same small target category. This large intra-class distance inevitably makes it difficult for models to detect these small targets. Therefore, the key to solving this problem lies in how to eliminate this intra-class distance. After analyzing a large number of infrared target images, we found that regardless of their shape, infrared targets all have a common feature, which we call the edge-surrounding convergence characteristic. That is, the gradient direction on the target edge points to the interior of the target. Therefore, we perform target edge detection and topologically transform the edges. Almost all targets, regardless of their shape, can be transformed into a ring topological state, thereby eliminating intra-class distances and improving the model's detection ability for odd-shaped targets. Summary of the Invention

[0003] The present invention provides an infrared small target detection method based on edge topology guidance. The technical problem to be solved is to eliminate the intra-class distance through target edge detection and edge topology, introduce the edge topology feature layer into the segmentation network, and thus improve the detection performance of the model for various types of targets.

[0004] In order to solve the above technical problems and achieve the above objectives, the technical solutions adopted by the present invention are as follows.

[0005] A method for detecting small infrared targets based on edge topology guidance includes the following steps:

[0006] Step 1: Perform edge detection on the input infrared image to obtain an edge image and proceed to step 2;

[0007] Step 2: Construct a candidate graph structure of multiple subgraphs through the edge image obtained in step 1, update the features of each node through a multi-layer message passing network, obtain the edge topology feature layer, and proceed to step 3;

[0008] Step 3: Embed the edge topology feature layer into the target detection network model to construct an edge topology guidance network. Through model training, the final infrared small target detection model with odd-shaped target detection capability is obtained.

[0009] In the above technical solution, step 1 includes the following steps:

[0010] Step 1.1: Input infrared image I of size w×h, and use rotatable r×r convolution kernel W r Get edge features B in different directions r , as shown in formula (1). Formula (1) is as follows:

[0011] B r =CONV(I,rot(W r ))#(1)

[0012] Where CONV(*) represents the normal convolution operation, rot(W r ) is of size r×r×d, where d=2π / θ represents the convolution kernel W r The number of rotations is determined by the single rotation angle θ. This convolution will not change the size of the input image, so the output edge feature B r The size is w×h×d;

[0013] Step 1.2: Considering that the target has different scales, multiple edge features of different scales are obtained through step 1.1, and channel connection is performed through formula (2) to obtain multi-scale edge features B. Formula (2) is as follows:

[0014] B=CONCAT(B r1 ,B r2 ,...,B rs )#(2)

[0015] Among them, CONCAT(*) represents channel connection, r1, r2, ..., rs represent different scales of convolution kernel. The initial weight of the convolution kernel with rotation θ and scale r can be set to -G(xcosθ+ysinθ) / r 2 , G is the Gaussian kernel. The size of the output multi-scale edge feature B is w×h×(d*s);

[0016] Step 1.3: The multi-scale edge feature B obtained in step 1.2 is convolved with a 1×1 convolution kernel W 1×1 Combine the features of each channel, reduce the number of channels to 1, and obtain the edge prediction image Edge through formula (3)pre °Formula (3) is as follows:

[0017] Edge pre =RELU(CONV1(B,W1×1))#(3)

[0018] Among them, CONV1(*) represents a 1×1 convolution operation, RELU(*) is the activation function, and W 1×1 The size is 1×1×(d*s), and finally, the edge prediction image Edge pre The size is w×h×1;

[0019] Step 1.4: The edge prediction image Edge obtained in step 1.3 pre and the true target edge image Edge label The loss function is constructed by formula (4), which is as follows:

[0020]

[0021] Among them, y i,j and p i,j Represents Edge label and Edge pre The value corresponding to coordinate (i, j).

[0022] Step 1.5: Use a large number of training samples and train the model using the loss function of step 1.4 to obtain W in step 1.1. r and W in step 1.3 1×1 The optimal weight parameter of , and then the optimal edge prediction model MED is obtained. The edge prediction process can be briefly summarized as shown in formula (5), which is as follows:

[0023]

[0024] Among them, I represents the input infrared image, E is the edge result output by MED, and are the optimal parameters obtained through training.

[0025] In the above technical solution, step 2 includes the following steps:

[0026] Step 2.1: Use formula (6) to perform global threshold segmentation on the edge result E obtained in step 1.5 to obtain the edge template image M. Formula (6) is as follows:

[0027]

[0028] Among them, Tre is the global threshold used to extract edges.

[0029] Step 2.2: For the edge template image M obtained in step 2.1, a value of 1 indicates an edge region. Connected domain features are expressed for the edge template image M. Each connected domain represents the edge of a candidate target, and the edge set UE of the edge template image M is obtained. The connectivity criterion uses 4-connectivity, that is, pixels with a value of 1 and adjacent in the vertical, horizontal, and vertical directions belong to the same connected domain. The feature contains position information and intensity information. The edge set UE can be expressed by equation (7), which is as follows:

[0030] UE={C1, C2, ..., C n}#(7)

[0031] Among them, C i Represents the i-th connected domain in M, and C i ={P i , F i}, P and F represent the position information and features of each pixel in the connected domain, respectively, where P = [p1, p2, ..., p k ], p i =(x i ,y i ), x, y are the horizontal and vertical coordinates of the pixel respectively, k is the number of pixels in the connected domain; F = [f1, f2..., f k ], f i =(I(x i ,y i ), B(x i ,y i )),

[0032] That is, the original image and the pixel values ​​corresponding to the multi-scale edge feature B obtained in step 1.2 are used as features. The final size of P is k×2, and the size of F is k×(1+d*s).

[0033] Step 2.3: Convert the edge set UE obtained in step 2.2 into a multi-subgraph hybrid candidate graph UG, eliminating the influence of the geometric structure and retaining the topological structure and features. The multi-subgraph hybrid candidate graph UG is shown in formula (8):

[0034] UG={G1,G2,...,G n}#(8)

[0035] Among them, the edge graph topology G i From step 2.2UE corresponding to C i G = (V, A), where V contains k nodes, corresponding to the number of pixels in C, and This means that each node will store a 1+d*s dimension feature h, corresponding to f in C; A∈{0,1} k×kIs a sparse adjacency matrix that describes the connection relationship between nodes. i，j =1 indicates node V i With node V j There is an edge connecting them, which means they are adjacent on the image. i，j =0 means there is no edge connecting them.

[0036] Step 2.4: For each edge graph topology G in the multi-subgraph hybrid candidate graph UG in step 2.3, in order to enable each node in G to obtain the topological information of the entire graph, each node will calculate its message passing vector from its neighbors through formula (9): Formula (9) is as follows:

[0037]

[0038] in, is the information received by node v at layer t+1, M t is the message function, represents the node feature of the t-th layer node v, N(v) represents the neighbor set of node v, Represents the node features of the neighbor node w of node v in layer t.

[0039] Step 2.5: Using the message passing vector obtained in step 2.4, update the node features by using formula (9), which is as follows:

[0040]

[0041] Among them, U t For the node update function, by combining the current node status and messages You can get the new node status

[0042] Step 2.6: For each edge graph topology G in the multi-subgraph hybrid candidate graph UG in step 2.3, perform steps 2.4 and 2.5 multiple times. Each node will receive messages from all nodes. At this point, the features of each node in the edge graph topology G will contain not only image structure information but also edge topology information. The final stopping edge graph topology is recorded as The multi-sub mixed candidate graph is denoted as

[0043] Step 2.7: Each time steps 2.4 and 2.5 are repeated, messages are passed to each node to update its features. After the update stops as in step 2.6, the multi-layer joint feature h of each node is calculated using formula (10): v , formula (10) is shown as follows.

[0044]

[0045] Among them, σ0(*) is the activation function, which can map the values ​​in the brackets to a specific range; CONCAT(*) is the connection function, K is the number of message transmission or feature updates, corresponding to the number of repetitions of steps 2.4 and 2.5, the original The feature dimension is 1+d*s. After the CONCAT(*) function, the feature vector of (1+d*s)*(K+1) will be obtained. Let L=(1+d*s)*(K+1), W G is the parameter matrix, size is L×O, T represents transpose. Finally h v The dimension is O.

[0046] Step 2.8: Use the multi-layer joint feature h obtained in step 2.7 v , use formula (11) to obtain the entire graph Topological characteristics of It not only includes the edge features at the original image level, but also the topological features at the graph structure level.

[0047]

[0048] Among them, V is the graph Node set, k represents the number of nodes, Maxpooling (*) represents the maximum value pooling, and finally The dimension is O.

[0049] Step 2.9: Using the graph obtained in step 2.8 Formula (12) is used to predict the category (target / false alarm) corresponding to the image. Formula (12) is as follows:

[0050]

[0051] Among them, σ1(*) is the activation function, which can map the value in the bracket to 0~1, K G is a parameter vector with a dimension of O. Ultimately, each graph G will output a predicted value gpre G ;

[0052] Step 2.10: The predicted value gpre obtained in step 2.9 G and the true label glabel G The loss function is constructed by formula (13), and the model is trained to solve the optimal W G and K G Formula (13) is as follows:

[0053]

[0054] Among them, UGt rain represents the training graph sample set, and num represents the number of training samples.

[0055] Step 2.11: After the model training is completed, substitute the relevant optimal parameters. Steps 2.1, 2.2, 2.3, 2.6, 2.7, and 2.8 can be used to obtain the edge result E from step 1.5 to obtain the topological characteristics of each edge. Construct a tensor of size w×h×O with a value of 0, and input the topological features of the graph or edge at the corresponding position of each node (each pixel on the edge) The edge topology feature layer FG is obtained as shown in formula (14):

[0056]

[0057] Among them, E represents the edge result obtained in step 1.5, and its size is w×h×1. EDTO represents the edge topology feature layer extraction process. FG is the edge topology feature layer output by EDTO, and its size is w×h×O. and are the optimal parameters obtained through training.

[0058] In the above technical solution, step 3 includes the following steps:

[0059] Step 3.1: Construct a segmentation model. Common segmentation models are mainly based on a U-shaped structure. Multi-scale features are obtained through convolution and downsampling, and then the original scale is restored through convolution and upsampling. There may be jump links in the middle. Due to the existence of various segmentation structures, the present invention summarizes the original segmentation process as shown in formula (15):

[0060] FS=SEG(I,W S )#(15)

[0061] Among them, SEG represents the feature extraction process of the segmentation network, I is the input infrared image, and W S is the trainable parameter involved in the segmentation network, FS is the output semantic feature layer, and its size is w×h×Se.

[0062] Step 3.2: The edge topology feature layer FG obtained in step 2.11 and the semantic feature layer FS obtained for detecting the target in step 3.1 are connected through equation (16) to obtain the mixed feature layer FC. Equation (16) is as follows:

[0063] FC=CONCAT(FG,FS)#(16)

[0064] Among them, CONCAT(*) represents channel connection, FC is the mixed feature layer, and the size is w×h×(Se+O)

[0065] Step 3.3: The mixed feature layer FC obtained in step 3.2 is used to make the final pixel-level prediction using formula (17), which is as follows:

[0066] Target pre =RELU(CONV(RELU(CONV(B,W s1 )), W s2 ))#(17)

[0067] Among them, Target pre Represents the final pixel-level prediction result, CONV(*) represents the normal convolution operation without changing the input size, W s1 and W s2 Respectively represent the corresponding training parameters under the two layers of convolution, and the final target pre The size is w×h×1.

[0068] Step 3.4: Among the infrared small target detection samples, odd-shaped targets often account for a minority. In order to enhance the detection capability of such targets, we will also construct the odd-shapedness measurement function SHA through formula (18). Formula (18) is as follows:

[0069]

[0070] Among them, ∈ is a parameter that controls the stability of the model, p i,j Indicates Target pre The value corresponding to the coordinate (i, j). UG is the edge map set corresponding to the current image, k G and gpre G They represent the number of nodes in graph G and the predicted value of graph category in step 2.9 respectively.

[0071] Step 3.5: Combine the odd shape measurement function SHA in step 3.4 and transform the edge prediction image Target obtained in step 3.3 pre and the true target edge image Target label The loss function can be constructed by formula (19), which is as follows:

[0072]

[0073] Among them, y i，j and p i，j Respectively represent Target label and Target pre The value corresponding to the coordinate (i, j). (1-SHA) is used to control the model to focus more on odd-shaped targets. ε is a parameter that controls the stability of the model.

[0074] Step 3.6: Using a large number of training samples, the model is trained using the loss function in step 3.5 to obtain the final object detection model DECT, which is summarized as shown in formula (20). Formula (20) is as follows:

[0075]

[0076] Among them, I represents the input infrared image, Out is the final detection result, Represents the optimal parameters obtained by training in the segmentation network.

[0077] Because the present invention adopts the above technical solution, it has the following beneficial effects:

[0078] 1. The edge detection method proposed in this invention combines the advantages of model-driven and data-driven approaches, that is, it has the interpretability and controllability of the model-driven approach and the high detection performance of the data-driven approach.

[0079] 2. This method introduces graph-related processing techniques into the traditional image-level object detection task.

[0080] This technology combines image features and graph features to improve the detection performance of the model in multiple dimensions.

[0081] 3. This invention introduces an edge topology feature layer into the traditional semantic segmentation feature layer, and proposes an oddness measurement function and introduces it into the loss function, which greatly improves the detection performance of small odd-shaped targets that are difficult to detect.

[0082] 4. The edge topology idea and corresponding method proposed in this method are extremely easy to embed into other infrared small target detection models and improve their performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0083] Figure 1 Design a process for the entire algorithm;

[0084] Figure 2 Images related to detecting small infrared targets for this process. (a) is the original infrared image with an odd-shaped target, (b) is the edge detection result, (c) is the edge label, and (d) is the pixel-level detection label.

[0085] Figure 3 This is a specific network structure diagram, which includes three modules: edge detection, edge graph topology and detection network;

[0086] Figure 4 This is a schematic diagram of edge topology. The edges of odd-shaped objects and regular small objects will reduce intra-class differences after topology. DETAILED DESCRIPTION

[0087] The present invention will be further described in detail below in conjunction with test examples and specific embodiments. However, this should not be understood as limiting the scope of the present invention to the following embodiments, and all technologies implemented based on the present invention fall within the scope of the present invention.

[0088] The present invention provides an infrared small target detection method based on edge topology guidance. The technical problem to be solved is to eliminate the intra-class distance through target edge detection and edge topology, introduce the edge topology feature layer into the segmentation network, and thus improve the detection performance of the model for various targets. The entire algorithm design structure framework is as follows Figure 1 As shown, the steps include:

[0089] Step 1.1: Input infrared image I of size w×h, and use rotatable r×r convolution kernel W r Get edge features B in different directions r , as shown in formula (1). Formula (1) is as follows:

[0090] B r =CONV(I,rot(W r ))#(1)

[0091] Where CONV(*) represents the normal convolution operation, the size of rot(W) is r×r×d, and d=2π / θ represents the convolution kernel W r The number of rotations is determined by the single rotation angle θ. This convolution will not change the size of the input image, so the output edge feature B r The size is w×h×d; in this example, the image size is 600×400, the single rotation angle is 60 degrees, and the number of rotations is 6 times, so the edge feature B of the single size output is r The size of the image is 600×400×6. The rotatable convolution kernel technique employed in step 1.1 effectively solves the problem of extracting multi-scale edge features in infrared images. This technique, by using convolution kernels of different scales and orientations, can more comprehensively capture edge information in the image, thereby improving the accuracy and robustness of edge detection. Specifically, the rotatable convolution kernel can adapt to targets of varying shapes and scales, enabling the model to more accurately identify target edges when processing infrared images. Furthermore, by rotating the convolution kernel, the model can capture edge information from multiple angles, further improving edge detection accuracy.

[0092] Step 1.2: Considering that the target has different scales, multiple edge features of different scales are obtained through step 1.1, and channel connection is performed through formula (2) to obtain multi-scale edge features B. Formula (2) is as follows:

[0093] B=CONCAT(B r1 ,B r2,...,B rs )#(2)

[0094] Among them, CONCAT(*) represents channel connection, r1, r2, ..., rs represent different scales of convolution kernel. The initial weight of the convolution kernel with rotation θ and scale r can be set to -G(xcosθ+ysinθ) / r 2 , G is the Gaussian kernel. The size of the output multi-scale edge feature B is w×h×(d*s); in this example, three different scales are used, namely 3×3, 5×5, and 7×7. The final size of the output multi-scale edge feature B is 600×400×18;

[0095] Step 1.3: The multi-scale edge feature B obtained in step 1.2 is convolved with a 1×1 convolution kernel W 1×1 Combine the features of each channel, reduce the number of channels to 1, and obtain the edge prediction image Edge through formula (3) pre Formula (3) is as follows:

[0096] Edge pre =RELU(CONV1(B,W 1×1 ))#(3)

[0097] Among them, CONV1(*) represents a 1×1 convolution operation, RELU(*) is the activation function, and W 1×1 The size is 1×1×(d*s), and finally, the edge prediction image Edge pre The size of is w×h×1; in this example, W 1×1 The size is 1×1×18, the edge prediction image Edge pre The size is 600×400×1.

[0098] Step 1.4: The edge prediction image Edge obtained in step 1.3 pre and the true target edge image Edge label The loss function is constructed by formula (4), which is as follows:

[0099]

[0100] Among them, y i，j and p i，j Represents Edge label and Edge pre The value corresponding to coordinate (i, j).

[0101] Step 1.5: Use a large number of training samples and train the model using the loss function of step 1.4 to obtain W in step 1.1. r and W in step 1.3 1×1The optimal weight parameter of , and then the optimal edge prediction model MED is obtained. The edge prediction process can be briefly summarized as shown in formula (5), which is as follows:

[0102]

[0103] Among them, I represents the input infrared image, E is the edge result output by MED, and are the optimal parameters obtained through training.

[0104] Step 2.1: Use formula (6) to perform global threshold segmentation on the edge result E obtained in step 1.5 to obtain the edge template image M. Formula (6) is as follows:

[0105]

[0106] Among them, Tre is the global threshold used to extract edges.

[0107] Step 2.2: For the edge template image M obtained in step 2.1, a value of 1 indicates an edge region. Connected domain features are expressed for the edge template image M. Each connected domain represents the edge of a candidate target, and the edge set UE of the edge template image M is obtained. The connectivity criterion uses 4-connectivity, that is, pixels with a value of 1 and adjacent in the vertical, horizontal, and vertical directions belong to the same connected domain. The feature contains position information and intensity information. The edge set UE can be expressed by equation (7), which is as follows:

[0108] UE={C1, C2, ..., C n}#(7)

[0109] Among them, C i Represents the i-th connected domain in M, and C i ={P i , F i}, P and F represent the position information and features of each pixel in the connected domain, respectively, where P = [p1, p2, ..., p k ], p i =(x i ,y i ), x, y are the horizontal and vertical coordinates of the pixel respectively, k is the number of pixels in the connected domain; F = [f1, f2..., f k ], f i =(I(x i ,y i ), B(x i ,y i)), that is, the original image, and the values ​​of the pixel positions corresponding to the multi-scale edge features B obtained in step 1.2 are used as features. The final size of P is k×2, and the size of F is k×(1+d*s). In this example, the size of F is k×19. The connected domain feature expression of the edge template image in step 2.2 effectively solves the problem of edge structure representation of small irregular targets. By expressing the connected domain features of the edge template image, the model can give position information and intensity information to each connected domain, enhance the representation ability of the target edge, and facilitate the subsequent conversion to a graph topology structure. This method is particularly suitable for processing small irregular targets, because the edge structure of small irregular targets is relatively complex, and traditional edge detection methods are difficult to accurately represent. Through the connected domain feature expression, the model can convert the edge structure of small irregular targets into a graph topology structure, thereby more accurately identifying and representing these targets.

[0110] Step 2.3: Convert the edge set UE obtained in step 2.2 into a multi-subgraph hybrid candidate graph UG, eliminating the influence of the geometric structure and retaining the topological structure and features. The multi-subgraph hybrid candidate graph UG is shown in formula (8):

[0111] UG={G1,G2,...,G n}#(8)

[0112] Among them, the edge graph topology G i From step 2.2UE corresponding to C i G = (V, A), where V contains k nodes, corresponding to the number of pixels in C, and This means that each node will store a 1+d*s dimension feature h, corresponding to f in C; A∈{0,1} k×k Is a sparse adjacency matrix that describes the connection relationship between nodes. i，j =1 indicates node V i With node V j There is an edge connecting them, which means they are adjacent on the image. i，j = 0 means there is no edge connecting them. The multi-subgraph hybrid processing in step 2.3 effectively solves the problem of converting edge structure into graph topology. By converting the edge set UE into a multi-subgraph hybrid candidate graph UG, the model can eliminate the influence of geometric structure and retain the topology and features. This method is particularly suitable for processing small irregular targets in infrared images, because the edge structure of small irregular targets is relatively complex, and different small irregular targets are difficult to distinguish from the perspective of geometric structure. After multi-subgraph hybrid processing, the model can convert complex and different target edge structures into similar ring graph topology structures, thereby more accurately identifying and characterizing these targets.

[0113] Step 2.4: For each edge graph topology G in the multi-subgraph hybrid candidate graph UG in step 2.3, in order to enable each node in G to obtain the topological information of the entire graph, each node will calculate its message passing vector from its neighbors through formula (9): Formula (9) is as follows:

[0114]

[0115] in, is the information received by node v at layer t+1, M t is the message function, represents the node feature of the t-th layer node v, N(v) represents the neighbor set of node v, Represents the node features of the neighbor node w of node v in layer t.

[0116] Step 2.5: Using the message passing vector obtained in step 2.4, update the node features by using formula (9), which is as follows:

[0117]

[0118] Among them, U t For the node update function, by combining the current node status and messages You can get the new node status

[0119] Step 2.6: For each edge graph topology G in the multi-subgraph hybrid candidate graph UG in step 2.3, perform steps 2.4 and 2.5 multiple times. Each node will receive messages from all nodes. At this point, the features of each node in the edge graph topology G will contain not only image structure information but also edge topology information. The final stopping edge graph topology is recorded as The multi-sub mixed candidate graph is denoted as In this example, steps 2.4 and 2.5 were repeated three times.

[0120] Step 2.7: Each time steps 2.4 and 2.5 are repeated, messages are passed to each node to update its features. After the update stops as in step 2.6, the multi-layer joint feature h of each node is calculated using formula (10): v , formula (10) is shown as follows.

[0121]

[0122] Among them, σ0(*) is the activation function, which can map the values ​​in the brackets to a specific range; CDNCAT(*) is the connection function, K is the number of message transmission or feature updates, corresponding to the number of repetitions of steps 2.4 and 2.5, the original The feature dimension is 1+d*s. After the CONCAT(*) function, the feature vector of (1+d*s)*(e+1) will be obtained. Let L=(1+d*s)*(K+1), W G is the parameter matrix, size is L×O, T represents transpose. Finally h v The dimension of is O. In this example, K=3, which means that steps 2.4 and 2.5 are repeated 3 times. The dimension of the feature vector is 19*4=76, W G The size is 76×4.

[0123] Step 2.8: Use the multi-layer joint feature h obtained in step 2.7 v , use formula (11) to obtain the entire graph Topological characteristics of It not only includes the edge features at the original image level, but also the topological features at the graph structure level.

[0124]

[0125] Among them, V is the graph Node set, k represents the number of nodes, Maxpooling (*) represents the maximum value pooling, and finally The dimension of is O. In this example, The dimension is 4. The multi-layer joint features and graph topology features calculated in steps 2.7 and 2.8 effectively solve the problem of feature fusion in infrared small target detection. By calculating the multi-layer joint features and graph topology features of each node, the model can fuse the edge features at the original image level with the topological features at the graph structure level, thereby improving the model's detection performance for infrared small targets. This method is particularly suitable for processing small, irregular targets in infrared images, because the edge structure of irregular small targets is relatively complex, which makes it difficult for traditional edge detection methods to accurately represent them. Through feature fusion, the model can more comprehensively capture the features of irregular small targets, thereby improving detection accuracy.

[0126] Step 2.9: Using the graph obtained in step 2.8 Formula (12) is used to predict the category (target / false alarm) corresponding to the image. Formula (12) is as follows:

[0127]

[0128] Among them, σ1(*) is the activation function, which can map the value in the bracket to 0~1, K Gis a parameter vector with a dimension of O. Ultimately, each graph G will output a predicted value gpre G ;

[0129] Step 2.10: The predicted value gpre obtained in step 2.9 G and the true label glabel G The loss function is constructed by formula (13), and the model is trained to solve the optimal W G and K G Formula (13) is as follows:

[0130]

[0131] UGtrain represents the training graph sample set, and num represents the number of training samples. In this example, the value of num is 1600.

[0132] Step 2.11: After the model training is completed, substitute the relevant optimal parameters. Steps 2.1, 2.2, 2.3, 2.6, 2.7, and 2.8 can be used to obtain the edge result E from step 1.5 to obtain the topological characteristics of each edge. Construct a tensor of size w×h×O with a value of 0, and input the topological features of the graph or edge at the corresponding position of each node (each pixel on the edge) The edge topology feature layer FG is obtained as shown in formula (14):

[0133]

[0134] Among them, E represents the edge result obtained in step 1.5, and its size is w×h×1. EDTO represents the edge topology feature layer extraction process. FG is the edge topology feature layer output by EDTO, and its size is w×h×O. and are the optimal parameters obtained through training. In this example, the size of the edge topology feature layer FG is 600×400×4.

[0135] Step 3.1: Construct a segmentation model. Common segmentation models are mainly based on a U-shaped structure. Multi-scale features are obtained through convolution and downsampling, and then the original scale is restored through convolution and upsampling. There may be jump links in the middle. Due to the existence of various segmentation structures, the present invention summarizes the original segmentation process as shown in formula (15):

[0136] FS=SEG(I,W S )#(15)

[0137] Among them, SEG represents the feature extraction process of the segmentation network, I is the input infrared image, and W Sis the trainable parameter involved in the segmentation network, and FS is the output semantic feature layer, with a size of w × h × Se. In this example, the size of the output semantic feature layer FS is 600 × 400 × 3.

[0138] Step 3.2: The edge topology feature layer FG obtained in step 2.11 and the semantic feature layer FS obtained for detecting the target in step 3.1 are connected through equation (16) to obtain the mixed feature layer FC. Equation (16) is as follows:

[0139] FC=CONCAT(FG,FS)#(16)

[0140] CONCAT(*) represents channel connection, and FC is the mixed feature layer with a size of w×h×(Se+O). In this example, the size of the mixed feature layer FC is 600×400×7.

[0141] Step 3.3: The mixed feature layer FC obtained in step 3.2 is used to make the final pixel-level prediction using formula (17), which is as follows:

[0142] Target pre =RELU(CONV(RELU(CONV(B,W s1 )),W s2 ))#(17)

[0143] Among them, Target pre Represents the final pixel-level prediction result, CONV(*) represents the normal convolution operation without changing the input size, W s1 and W s2 Respectively represent the corresponding training parameters under the two layers of convolution, and the final target pre The size of target is w×h×1. In this example, target pre The size is 600×400×1.

[0144] Step 3.4: Among the infrared small target detection samples, odd-shaped targets often account for a minority. In order to enhance the detection capability of such targets, we will also construct the odd-shapedness measurement function SHA through formula (18). Formula (18) is as follows:

[0145]

[0146] Among them, ∈ is a parameter that controls the stability of the model, p i，j Indicates Target pre The value corresponding to the coordinate (i, j). UG is the edge map set corresponding to the current image, k G and gpre GRepresent the number of nodes in graph G and the predicted value of the graph category in step 2.9, respectively. The oddness measure function introduced in step 3.4 effectively solves the problem of detecting a small number of odd-shaped small targets. By constructing the oddness measure function SHA, the model can focus more on identifying a small number of odd-shaped small targets with irregular shapes, thereby improving the detection accuracy of these targets. This method is particularly suitable for processing small, irregular-shaped targets in infrared images, because these targets are in the minority in the image and are difficult to accurately identify with traditional detection methods. Through the oddness measure function, the model can focus more on identifying small, odd-shaped targets with irregular shapes, thereby improving detection accuracy.

[0147] Step 3.5: Combine the odd shape measurement function SHA in step 3.4 and transform the edge prediction image Target obtained in step 3.3 pre and the true target edge image Target label The loss function can be constructed by formula (19), which is as follows:

[0148]

[0149] Among them, y i，j and p i，j Respectively represent Target label and Target pre The value corresponding to the coordinate (i, j). (1-SHA) is used to control the model to focus more on odd-shaped targets. ε is a parameter that controls the stability of the model.

[0150] Step 3.6: Using a large number of training samples, the model is trained using the loss function in step 3.5 to obtain the final object detection model DECT, which is summarized as shown in formula (20). Formula (20) is as follows:

[0151]

[0152] Among them, I represents the input infrared image, Out is the final detection result, represents the optimal parameters obtained from training the segmentation network. The model training in step 3.6 effectively solves the optimization problem of the infrared small target detection model. By using a large number of training samples, the model can be optimized using the loss function, thereby improving overall detection performance. This method is particularly suitable for processing small, irregularly shaped targets in infrared images, as these targets are a minority in the image and difficult to accurately identify using traditional detection methods. Through model training, the model can better focus on identifying small, irregularly shaped targets, thereby improving detection accuracy.

Claims

1. A method for detecting small infrared targets based on edge topology guidance, characterized in that: The steps include: Step 1: Perform edge detection on the input infrared image to obtain an edge image and proceed to step 2; Step 2: Construct a candidate graph structure of multiple subgraphs through the edge image obtained in step 1, update the features of each node through a multi-layer message passing network, obtain the edge topology feature layer, and proceed to step 3; Step 3: Embed the edge topology feature layer into the target detection network model to construct an edge topology guidance network. Through model training, the final infrared small target detection model with odd-shaped target detection capability is obtained. In step 2, the geometric edges of the odd-shaped targets are transformed into a ring topology structure through the edge wrapping convergence feature to eliminate the intra-class distance difference. The step 2 comprises the following steps: Step 2.1: Obtain the edge result of the edge image obtained in step 1 , use formula (6) to perform global threshold segmentation and obtain the edge template image , formula (6) is as follows: in, is the global threshold, used to extract edges; Step 2.2: For the edge template image obtained in step 2.1 , a value of 1 indicates an edge area, for edge template images Perform connected domain feature expression, each connected domain represents the edge of a candidate target, and obtain the edge template image Edge collection , where the connectivity criterion uses 4 connectivity, that is, pixels with a value of 1 and adjacent to each other in the up, down, left, and right directions belong to the same connected domain. The feature contains position information and intensity information, and the edge set It can be expressed by formula (7), which is as follows: in, express Middle connected domains, and , and Respectively represent the location information and features of each pixel in the connected domain, where , , are the horizontal and vertical coordinates of the pixel respectively, is the number of pixels in the connected domain; , , which is the original image and multi-scale edge features The value of the corresponding pixel position is used as the feature, and finally The size is , The size is ; Step 2.3: Set the edge set obtained in step 2.2 Convert to a multi-subgraph hybrid candidate graph , multiple subgraph mixed candidate graph As shown in formula (8): Among them, the edge graph topology From step 2.2 Corresponding Converted to ,here, Contains a total of nodes, and corresponding The number of pixels corresponds to , which means that each node will store Dimensional characteristics ,and middle correspond; It is a sparse adjacency matrix that describes the connection relationship between nodes. Representation node With node There is an edge connecting them, and they are adjacent on the image. It means there is no edge connecting them; Step 2.4: Mix candidate graphs for multiple subgraphs in step 2.3 Each edge graph topology in , in order to Each node in the graph obtains the topological information of the entire graph, and each node calculates its message passing vector from its neighbors through formula (9): , formula (9) is as follows: in, For nodes In the The information received by the layer, is the message function, Indicates the Layer Node The node characteristics of Representation node The neighbor set of express Nodes in the layer Neighbor nodes Node characteristics; Step 2.5: Using the message passing vector obtained in step 2.4, update the node features by using formula (9), which is as follows: in, For the node update function, by combining the current node status and messages , get the new node status ; Step 2.6: For step 2.3, multiple subgraphs are mixed into candidate graphs Each edge graph topology in , and then perform steps 2.4 and 2.5 multiple times. Each node will get the information of all nodes. At this point, the edge graph topology The features of each node in the image not only contain image structure information, but also edge topology information. The final stopped edge graph topology is recorded as , the multi-submix candidate graph is denoted as ; Step 2.7: Each time steps 2.4 and 2.5 are repeated, messages are passed to each node to update its features. After the update stops as in step 2.6, the multi-layer joint features of each node are calculated using formula (10) , formula (10) is shown as follows: in, It is an activation function that can map the values ​​in the brackets to a specific range; is the connection function, is the number of message transmissions or feature updates, corresponding to the number of repetitions of steps 2.4 and 2.5 for the original The feature dimension is ,go through After the function, you will get The characteristic vector of , is the parameter matrix, size is , Indicates transposition, and finally The dimension is ; Step 2.8: Use the multi-layer joint features obtained in step 2.7 , use formula (11) to obtain the entire graph Topological characteristics of , which includes not only the edge features at the original image level, but also the topological features at the graph structure level: in, For the picture A collection of nodes, Indicates the number of nodes, Represents maximum pooling, and finally The dimension is ; Step 2.9: Using the graph obtained in step 2.8 , use formula (12) to predict the category corresponding to the image, the category includes target and false alarm, formula (12) is as follows: in, is an activation function that maps the values ​​in the brackets to , is a parameter vector with dimension , finally, each graph Will output a predicted value ; Step 2.10: The predicted values ​​of the graph obtained in step 2.9 and the true label The loss function is constructed by formula (13) and the model is trained to find the optimal and , formula (13) is as follows: in, represents the training graph sample set, Indicates the number of training samples; Step 2.11: After the model training is completed, substitute the relevant optimal parameters, and steps 2.1, 2.2, 2.3, 2.6, 2.7, and 2.8 are used to obtain the edge results. , get the topological features of each edge , the build size is , a tensor with a value of 0, at the corresponding position of each node in the graph, input the topological characteristics of the graph or edge , get the edge topology feature layer , as shown in formula (14): in, Represents the edge result, the size is , represents the edge topology feature layer extraction process, For passing The output edge topology feature layer has a size of , and are the optimal parameters obtained through training.

2. The infrared small target detection method based on edge topology guidance according to claim 1 is characterized in that: The step 1 comprises the following steps: Step 1.1: Input size is Infrared image , using a rotatable Convolution kernel Get edge features in different directions , as shown in formula (1): in represents the normal convolution operation, The size is , Represents the convolution kernel The number of rotations is determined by the single rotation angle The convolution will not change the size of the input image, so the output edge features The size is ; Step 1.2: Considering that the target has different scales, edge features of multiple scales are obtained through step 1.1, and channel connection is performed through formula (2) to obtain multi-scale edge features , formula (2) is as follows: in, Indicates channel connection, Indicates different scales and rotations of convolution kernels The scale is The initial weight of the convolution kernel is set to , is a Gaussian kernel, and the output multi-scale edge features The size is ; Step 1.3: The multi-scale edge features obtained in step 1.2 ,use Convolution kernel Combine the features of each channel, reduce the number of channels to 1, and obtain the edge prediction image through formula (3) , formula (3) is as follows: in, express Convolution operation, is the activation function, The size is ,Finally, the edge prediction image The size is ; Step 1.4: The edge prediction image obtained in step 1.3 and the true target edge image The loss function is constructed by formula (4), which is as follows: in, and Respectively and coordinate The corresponding value at Step 1.5: Use a large number of training samples and train the model using the loss function of step 1.4 to obtain the value in step 1.

1. and in step 1.3 The optimal weight parameter of , the edge prediction process is shown in formula (5), which is as follows: in, represents the input infrared image, is the edge result output by MED, and They are trained respectively.

3. The infrared small target detection method based on edge topology guidance according to claim 2 is characterized in that: The step 3 comprises the following steps: Step 3.1: Construct a segmentation model. The segmentation model is mainly based on a U-shaped structure. Multi-scale features are obtained through convolution and downsampling, and then the original scale is restored through convolution and upsampling. The segmentation process is summarized as shown in formula (15): in, represents the feature extraction process of the segmentation network, is the input infrared image, are the trainable parameters involved in the segmentation network, is the output semantic feature layer, with a size of ; Step 3.2: Get the edge topology feature layer from step 2.11 The semantic feature layer obtained in step 3.1 for detecting the target Through the channel connection through formula (16), the mixed feature layer is obtained , formula (16) is as follows: in, Indicates channel connection, is the mixed feature layer, with a size of ; Step 3.3: Mix the feature layer obtained in step 3.2 ,The final pixel-level prediction is performed through formula (17), which is as follows: in, Represents the final pixel-level prediction result, represents a normal convolution operation that does not change the input size, and Represent the corresponding training parameters under the two layers of convolution, and finally The size is ; Step 3.4: In the infrared small target detection sample, the odd shape measurement function SHA is constructed by formula (18), which is as follows: in, is a parameter that controls the stability of the model. express coordinate The corresponding value at is the edge map set corresponding to the current image, and Respectively represent the graph The number of nodes and the predicted value of the graph category in step 2.9; Step 3.5: Combine the odd shape measurement function SHA in step 3.4 and transform the edge prediction image obtained in step 3.3 and the true target edge image The loss function can be constructed by formula (19), which is as follows: in, and Respectively and coordinate The corresponding value at Used to control the model to focus more on odd-shaped targets, A parameter that controls the stability of the model; Step 3.6: Use a large number of training samples and train the model using the loss function in step 3.5 to obtain the final target detection model. , which is summarized as shown in formula (20). Formula (20) is as follows: in, represents the input infrared image, For the final test results, Represents the optimal parameters obtained by training in the segmentation network.

Images