Intensive small target detection method based on fuzzy perception

Abstract

The invention provides a dense small target detection method based on fuzzy perception. The method comprises the following steps: averagely segmenting an infrared image into a plurality of regions, blocks and sub-blocks; utilizing a nested four-way Mama network to respectively extract the features of the regions, the blocks and the sub-blocks, and sequentially embedding and fusing the extracted features to obtain an initial feature map; transmitting the initial feature map to a multi-scale context collaborative network, and outputting a final prediction feature map through information transmission and feature fusion operation; inputting the final prediction feature map into a detection head, and outputting arbitrarily distributed detection frame representation and detection frame quality estimation; and weighting the loss function and fusing the directivity to obtain a boundary fuzzy perception loss function for optimizing the whole process of small target detection. According to the method, the thermal radiation target with relatively high spatial distribution density can be accurately positioned in a complex scene, and the accuracy of infrared dense small target detection is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of target detection and recognition, and in particular to a dense small target detection method based on fuzzy perception. Background Art

[0002] Infrared imaging has the characteristics of good concealment, strong anti-interference ability, strong ability to capture details, and can work all-weather, playing an important role in the field of remote sensing. It is widely used in detection and guidance, surveillance and monitoring, search and tracking, etc. However, due to the long imaging distance, the targets in infrared images are usually very small, lacking color and texture information, which brings great difficulties to target detection. And in practical applications, the scene is complex, and the targets in the image are characterized by multiple and dense. Their edges are blurred, and they are severely coupled with the environmental background, resulting in unclear target contour information. Therefore, infrared dense small target detection has become a very challenging task. The main difficulties lie in how to fully extract global and local features and make good use of the relationship between the two, as well as solve the problem of the ambiguity and uncertainty of each target bounding box.

[0003] Current infrared small target detection algorithms mainly include two categories: traditional model-driven methods and deep learning data-driven methods. 1) Traditional model-driven methods usually extract low-level features such as edges, textures, and shapes in the image, and then classify and detect according to the characteristics of the target and the background. However, the effectiveness of these methods highly depends on the accuracy of prior knowledge and assumptions, and the generalization ability is limited. Once the application environment changes or the image features do not meet expectations, the detection performance of the model may decrease significantly. 2) Deep learning data-driven methods mainly include convolutional neural networks and vision transformers. Compared with convolutional neural network methods, vision transformer networks show better learning ability on large-scale data due to the integration of self-attention mechanisms. However, the computational complexity of the self-attention mechanism grows quadratically with the increase in the number of input tokens, thus introducing a significant computational overhead in downstream tasks. In contrast, the Mamba network based on the state space model successfully reduces the computational burden by maintaining linear complexity and has achieved remarkable performance in various modeling tasks. However, when directly transferring vision Mamba to infrared small target detection, the detection accuracy is not high because the target usually only occupies a small number of pixels in the image, unable to fully mine local features, let alone make full use of the correlation between global and local features, resulting in still weak ability to capture detailed information.

[0004] Regarding the above problems, the widely used bounding box representation can be regarded as a Dirac delta distribution of the target box coordinates. However, this distribution is concentrated only at one point and cannot effectively utilize the rich information near the target boundary. Although some recent studies represent arbitrary distributions by directly learning their discretized probability distributions in the continuous space without introducing other stronger priors, this improved representation method poses challenges to optimization. At the same time, the traditional loss function Focal Loss, although dealing with the class imbalance problem by reshaping the standard cross-entropy loss, only supports discrete {1, 0} class labels and is not suitable for the continuous labels in this method. In summary, the existing technologies for detecting infrared dense small targets are still inaccurate. Summary of the Invention

[0005] In order to overcome the deficiencies of the prior art, the purpose of the present invention is to provide a method for detecting dense small targets based on fuzzy perception, which can accurately locate hot radiation targets with a large spatial distribution density from complex scenes and improve the accuracy of infrared dense small target detection.

[0006] To achieve the above object, the present invention provides the following solution: A method for detecting dense small targets based on fuzzy perception, comprising the following steps:

[0007] Obtain an infrared image, evenly divide the infrared image into multiple regions, evenly divide each region into multiple blocks, and then evenly divide each block into multiple sub-blocks;

[0008] Use a nested four-way Mamba network to extract the features of the regions, blocks, and sub-blocks respectively, obtain shallow features, middle features, and deep features, and then embed and fuse the shallow features, middle features, and deep features in order from shallow to deep to obtain an initial feature map; wherein, the nested four-way Mamba network includes a convolutional stem for generating region maps, block maps, and sub-block maps and an encoding structure for feature extraction;

[0009] Transmit the initial feature map into a multi-scale context collaboration network, and output a final prediction feature map through information transmission and feature fusion operations;

[0010] Input the final prediction feature map into a detection head, and output a detection box representation of an arbitrary distribution and a detection box quality estimation; wherein, the detection head includes a detection box position regression branch of an arbitrary distribution and a detection box quality prediction branch;

[0011] Weight the detection box position regression branch and the detection box quality prediction branch and fuse the directivity to obtain a boundary fuzzy perception loss function for optimizing the whole process of small target detection.

[0012] Optionally, the convolution trunk generates regions, blocks, and sub-blocks of different dimensions from the infrared image, maps the sub-blocks into the blocks using a 2×2 sliding window with a stride of 2, and maps the blocks into the regions using a 4×4 sliding window with a stride of 4; wherein, each block consists of 2×2 sub-blocks, and each region consists of 4×4 blocks;

[0013] The encoding structure sets the feature extraction processes of the sub-blocks, the blocks, and the regions as the shallow stage, the middle stage, and the deep stage respectively, and each layer of each stage consists of two four-way Mamba structures.

[0014] Optionally, the feature extraction process of the shallow stage includes:

[0015] Using the four-way Mamba structure of the shallow stage, by calculating the interaction magnitude between two adjacent sub-blocks, constructing the relationship representation between the sub-blocks; the relationship representation formula between the sub-blocks is:

[0016]

[0017] where X m is the sub-block representation mapped to the block, a represents the a-th four-way Mamba structure in each stage, a = 1, 2, LN represents layer normalization, and DP represents regularization operation;

[0018] Combining the relationship representation between the sub-blocks, embedding the sub-blocks mapped to the block into the block, and the sub-block embedding expression is:

[0019]

[0020] where y m is the block representation mapped to the region, and FC represents the fully connected layer.

[0021] Optionally, the feature extraction process of the middle stage includes:

[0022] Using the four-way Mamba structure of the middle stage to express the relationship between the blocks, and the relationship expression between the blocks is:

[0023]

[0024] where Y n is the set of block mappings of the n-th region;

[0025] Combining the relationship between the blocks, embedding the blocks mapped to the region into the region, and the block embedding expression is:

[0026]

[0027] Among them, z n represents the nth region.

[0028] Optionally, the feature extraction process in the deep stage includes:

[0029] Using the four-way Mamba structure in the deep stage to represent the relationship between the regions, and the relationship expression between the regions is:

[0030]

[0031] Optionally, the process of calculating using the four-way Mamba structure includes:

[0032] S1. Transpose the input and divide the input into a first branch and a second branch for calculation;

[0033] S2. Use a 2D convolution with a kernel of 1×1 to divide the input into multiple non-overlapping spatial blocks and flatten and project them into the embedding dimension to obtain an initial embedding vector;

[0034] S3. Add learnable spatial positions to the initial embedding vector, perform normalization operations, forward calculations, and backward calculations on the initial position embedding vector after adding the spatial positions, and then linearly project the calculated initial embedding vector into sub-blocks and regions of a new dimension to obtain new dimension sub-blocks and new dimension regions;

[0035] S4. Perform per-channel 1D convolution, linear projection, and smooth activation on the new dimension sub-blocks in sequence, and then introduce learnable parameters for discretization to obtain continuously updated hidden layer states and outputs of each layer;

[0036] S5. Activate the new dimension regions and stack the outputs of each layer, and use the activated new dimension region outputs to enhance the results;

[0037] S6. For the first branch and the second branch, perform the operations of S2 - S5 respectively, and then add the output results of the first branch and the second branch, output the final result, and obtain the initial feature map.

[0038] Optionally, transfer the initial feature map to a multi-scale context collaboration network, and through information transfer and feature fusion operations, output the final predicted feature map, including:

[0039] Use the convolutional layer of the multi-scale context collaboration network to unify the dimension of the initial feature map to maintain the same number of channels;

[0040] Through upsampling operations, lift the low-resolution feature map to the same size as the high-resolution feature map to perform multi-scale feature fusion of the initial feature map;

[0041] Extract information from the initial feature map after multi-scale feature fusion to output the final predicted feature map.

[0042] Optionally, input the final predicted feature map into the detection head to output detection box representations and detection box quality estimates with arbitrary distributions, including:

[0043] Input the final predicted feature map into a combined module including 4 Conv2d + GN + ReLU. Divide the output of the combined module into two branches. Both branches go through convolutional layers respectively and then are input into the detection box position regression branch and the detection box quality prediction branch respectively;

[0044] Output a basic generalized distribution through the detection box position regression branch and represent the predicted detection box. Calculate the sigmoid function through the detection box quality prediction branch to obtain the detection box quality prediction value.

[0045] Optionally, weight the detection box position regression branch and the detection box quality prediction branch and fuse the directionality to obtain a boundary ambiguity perception loss function for optimizing the entire process of small target detection, including:

[0046] Introduce GIoU Loss and SIoU Loss into the loss function of the detection box position regression branch to obtain the detection box representation weighted loss function. Introduce BCELoss into the loss function of the detection box quality prediction branch to obtain the detection box quality weighted loss function. Combine the detection box representation weighted loss function and the detection box quality weighted loss function to obtain the boundary ambiguity perception loss function; The expression of the boundary ambiguity perception loss function is:

[0047]

[0048] Among them, (x, y) represents the feature map position, p x,y represents the estimated detection box quality score at this point, represents the IoU score between the predicted box and the ground truth box at this point, t x,y represents the predicted box information at this point, represents the ground truth box information at this point, s x,y represents the probability of two discrete points predicted to be close to the ground truth box at this point, represents the distance between this point and the ground truth box, N pos represents the number of positive samples, λ1 represents the weight of L GS and λ2 represents the weight of L R . represents the indicator function, when it is 1, otherwise it is 0.

[0049] The present invention provides a method for detecting dense small targets based on fuzzy perception, and discloses the following technical effects:

[0050] 1. By nesting the Mamba network, it is possible to extract global and local features and make full use of the relationship between the two. Local mapping features and global features are extracted from the shallow, middle, and deep layers respectively, progressing layer by layer and concatenating local details and the local-global relationship through an embedding method to improve the accuracy of infrared dense small target detection.

[0051] 2. Through the four-way Mamba structure, it is possible to extract targets under complex background interferences such as platform movement and environmental light changes. This structure realizes data-dependent global visual context modeling by constructing forward, backward, lateral, and longitudinal state space models, thereby enhancing the ability to understand complex backgrounds. In addition, a position embedding mechanism is introduced to strengthen the model's spatial position perception ability, further improving the accuracy and robustness of target extraction. While retaining the correlation of spatial information, it can effectively overcome the interference of background dynamic changes, providing an efficient solution for the infrared small target detection task.

[0052] 3. It is capable of accurately locating the task of thermal radiation targets with a large spatial distribution density in complex scenes. The present invention makes improvements to the traditional detection head, which is divided into two branches. One branch outputs detection frames representing arbitrary distributions. It converts the integral in the continuous domain into a discrete representation, does not introduce prior shapes, but allows the model to learn the shape from the data itself. And to be more suitable for small and weak targets, the interval of discrete values is set very small. The other branch outputs the detection frame quality estimation. Different from the traditional method, its positive sample labels are dynamically calculated during training instead of fixed values, providing support for improving the perception ability of boundary uncertainties.

[0053] 4. The present invention solves the problem of blurred and uncertain target boundaries in infrared images caused by the severe overlap of dense small targets and their strong coupling with the environmental background through the bounding box fuzzy perception loss function. Considering that the real distribution usually does not deviate too far from the annotated position, this loss function enables the network to efficiently and intensively learn the probabilities of discrete values near the positions of target bounding boxes under any flexible distribution, improving the accuracy of locating the uncertain boundaries of infrared dense small targets. Moreover, it integrates directionality, enhances the scale perception and angle perception abilities, and improves the convergence speed.

[0054] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Description of the Drawings

[0055] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required in the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0056] Figure 1 Schematic diagram of the method flow provided by the embodiment of the present invention;

[0057] Figure 2 Schematic diagram of the infrared dense small target detection process provided by the embodiment of the present invention;

[0058] Figure 3 Schematic diagram of the four-way Mamba structure framework provided by the embodiment of the present invention;

[0059] Figure 4 Schematic diagram of the detection box representation provided by the embodiment of the present invention. Detailed implementation manners

[0060] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, rather than all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present invention.

[0061] To make the above objects, features, and advantages of the present invention more obvious and understandable, the present invention will be further described in detail below with reference to the drawings and specific implementation manners.

[0062] As Figure 1-2 shown, the present invention provides a dense small target detection method based on fuzzy perception, including the following steps:

[0063] 1. Obtain an infrared image, evenly divide the infrared image into multiple regions, evenly divide each region into multiple blocks, and then evenly divide each block into multiple sub-blocks.

[0064] 2. Use a nested four-way Mamba network to extract the features of the region, the block, and the sub-block respectively, obtain shallow features, middle features, and deep features, and then embed and fuse the shallow features, the middle features, and the deep features in sequence from shallow to deep to obtain an initial feature map; wherein, the nested four-way Mamba network includes a convolutional trunk for generating region mapping, block mapping, and sub-block mapping and an encoding structure for feature extraction.

[0065] 2.1 Convolutional trunk

[0066] The convolution stem generates regions, blocks, and sub-blocks of different dimensions from the infrared image, maps the sub-blocks into the blocks using a 2×2 sliding window with a stride of 2, and maps the blocks into the regions using a 4×4 sliding window with a stride of 4; where each block consists of 2×2 sub-blocks, and each region consists of 4×4 blocks. For example:

[0067] Assume an infrared image is input First, use stacked 3×3 convolution stems to generate regions blocks sub-blocks where M, N, and P are the dimensions of the region, block, and sub-block respectively. Then each block consists of 2×2 sub-blocks, and a 2×2 sliding window with a stride of 2 is used to map the sub-blocks into the blocks, so the mapped sub-blocks Each region consists of 4×4 blocks, and a 4×4 sliding window with a stride of 4 is used to map the blocks into the regions, so the mapped blocks That is, the resolution of each region is 8×8, the resolution of each block is 2×2, and the resolution of each sub-block is 1×1. Since the present invention adopts a hierarchical encoder structure, the spatial shapes of the regions, blocks, and sub-blocks are not fixed and gradually decrease as the network layer deepens.

[0068] 2.2 Encoding Structure

[0069] As Figure 3 shown, the encoding structure sets the feature extraction processes of the sub-blocks, blocks, and regions as the shallow stage, middle stage, and deep stage respectively, and each layer in each stage consists of two four-way Mamba structures. For example:

[0070] The encoding structure is a three-stage hierarchical encoder. In these three stages, the spatial shapes of the sub-blocks are successively: H / i×W / i, i = 1, 2, 4, the spatial structures of the blocks are successively: H / j×W / j, j = 2, 4, 8, and the spatial structures of the regions are successively: H / k×W / k, k = 8, 16, 32. Block merging is used as the downsampling operation between different stages, and each layer in each stage consists of two four-way Mamba structures, which process the sub-block, block, and region features. The sub-block is represented as:

[0071] x = [x1, x2, …, x H×W

[0072] The sub-block mapped to the block is represented as:

[0073] X m = [x m,1 , x m,2 , x m,3 , x​m,4

[0074] Among them, x m,p represents the mapping of the p-th sub-block in the m-th block, and X m is the set of sub-block mappings of the m-th block.

[0075] 2.3 The feature extraction process in the shallow stage includes:

[0076] Using the four-way Mamba structure in the shallow stage, by calculating the interaction magnitude between two adjacent sub-blocks, construct the relationship representation between the sub-blocks; the relationship representation formula between the sub-blocks is:

[0077]

[0078] Among them, X m is the sub-block representation mapped to the block, a represents the a-th four-way Mamba structure in each stage, a = 1, 2, LN represents layer normalization, and DP represents the regularization operation.

[0079] The block is represented as:

[0080] y = [y1, y2, …, y m , …, y H / 2×W / 2

[0081] Among them, y m represents the m-th block.

[0082] Combined with the relationship representation between the sub-blocks, embed the sub-blocks mapped to the block into the block, and the sub-block embedding expression is:

[0083]

[0084] Among them, y m is the block representation mapped to the region, and FC represents the fully connected layer.

[0085] The block representation mapped to the region is:

[0086] Y n = [y n,1 , y n,2 , …, y n,16

[0087] Among them, y n,q represents the mapping of the q-th block in the n-th region, and Y n is the set of block mappings in the n-th region.

[0088] 2.4 The feature extraction process in the middle layer stage includes:

[0089] ​​​Express the relationship between the blocks using the four-way Mamba structure in the middle layer. The relationship expression between the blocks is:

[0090]

[0091] where Y n is the set of block mappings in the nth region.

[0092] The region is expressed as:

[0093] z = [z1, z2, …, z n , …, z H / 8×W / 8

[0094] where z n represents the nth region. Combining the relationship between the blocks, embed the blocks mapped to the region into the region. The block embedding expression is:

[0095]

[0096] 2.5 The feature extraction process in the deep layer includes:

[0097] Express the relationship between the regions using the four-way Mamba structure in the deep layer. The relationship expression between the regions is:

[0098]

[0099] 2.6 The calculation process of the four-way Mamba structure:

[0100] S1. First, transpose the input U' and divide the input U (including X, Y, z) into two branches (the first branch and the second branch) for calculation:

[0101] U' = transpose(U)

[0102] S2. For each branch, use a 2D convolution with a kernel of 1×1 to divide the input into L non-overlapping spatial blocks and flatten and project them onto the embedding dimension (D dimensions) to obtain the initial embedding vector:

[0103] Q = flatten(Conv2d(U))

[0104] where the embedding vector is Q ∈ R L×D , where L = h × w and D is the embedding dimension.

[0105] S3. Add the learnable spatial position embedding q s ∈ R (h×w)×D to the initial embedding vector to retain the position information, as shown in the following formula:

[0106] Q = Q + q​s

[0107] Perform a normalization operation on the initial position embedding vector after adding the spatial position to obtain Q'.

[0108] Q' = RMSNorm(Q)

[0109] Then calculate separately from the forward and backward directions. Keep the original position in the forward direction and flip the vector in the backward direction:

[0110] P' backward = flip(P' forward )

[0111] Then linearly project the calculated initial embedding vector onto the sub - blocks and regions of the new dimension E to obtain the new - dimension sub - block x and the new - dimension region z:

[0112] x = Linear x (P')

[0113] z = Linear z (P')

[0114] S4. Perform channel - wise 1D convolution on the new - dimension sub - block x in sequence to obtain x', so as to reduce the computational complexity:

[0115] x' = SiLU(DWConv(x))

[0116] Linearly project x' onto B, C, Δ, and perform a smooth activation on Δ:

[0117] B = Linear B (x')

[0118] C = Linear C (x')

[0119] Δ = log(1 + exp(Linear Δ (x')+Parameter Δ ))

[0120] A is used as a learnable parameter, and then A, B and Δ are responded to achieve the purpose of discretization:

[0121]

[0122] Obtain the continuously updated hidden - layer state and the output of each layer. The hidden - layer state h o and the output y o of each layer are continuously updated through the following formula:

[0123]

[0124] where \(o = 0, 1, \ldots, L - 1\).

[0125] S5. Activate the new - dimension region z as the subsequent gating parameter to adjust the final output, so as to better capture and express the spatio - temporal sequence feature information:

[0126] \(z'=\text{SiLU}(z)\)

[0127] Finally, stack \(y\) o and regulate it by \(z'\) to generate the output \(y'\):

[0128]

[0129] Add the results obtained from the forward and backward directions, and pass through a linear layer. \(Q\) performs feature fusion through skip connections to finally obtain the enhanced output:

[0130] \(Q=\text{Linear}(y'\) forward +y'\) backward ) + Q

[0131] S6. For the first branch and the second branch, perform the operations of S2 - S5 respectively, then add the output results of the first branch and the second branch, and output the final result to obtain the initial feature map.

[0132] \(U=\text{rearrange}(\text{BDMamba}(Q))\)

[0133] \(U'=\text{rearrange}(\text{BDMamba}(Q\) transpose ))

[0134] \(U = U'+U\)

[0135] That is, perform calculations in four directions: forward, backward, horizontal, and vertical on the input, which can be expressed as:

[0136] \(U=\text{FDMamba}(U)\)

[0137] In this way, the global context relationship can be fully utilized to strengthen the model's perception of spatial positions and understanding of local regions of small targets.

[0138] 3. Transmit the initial feature map into the multi - scale context collaborative network, and output the final predicted feature map through information transmission and feature fusion operations. It includes:

[0139] (1) Convolution processing to unify the dimension of the feature map

[0140] The multi - scale context collaborative network receives the regional feature maps from the above three different scales Each feature map first passes through a convolutional layer with a kernel of 3×3 and 256 channels to maintain the same number of channels:

[0141] z'1 = SiLU(Conv2d(Norm(z1)))

[0142] z'2 = SiLU(Conv2d(Norm(z2)))

[0143] z'3 = SiLU(Conv2d(Norm(z3)))

[0144] Then

[0145] (2) Collaborative fusion to align multi-scale features

[0146] For z'3, perform upsampling using a transposed convolution with a kernel of 3×3 and a stride of 2:

[0147] z”3 = W * z'3

[0148] where W is the transposed convolution weight, then Then add it element-wise to z'2 to achieve feature fusion:

[0149] z 2r = z'2 + z”3

[0150] Then Then perform upsampling on z 2r continuously using bilinear interpolation:

[0151]

[0152] where w is the bilinear interpolation weight, then Then add it element-wise to z'1 to achieve feature fusion:

[0153] z 1r = z'1 + z”2

[0154] Then

[0155] (3) Information reconstruction to enhance the ability to capture small target details

[0156] For z 1r ,z 2r , use a convolutional layer with a kernel of 3×3 and a stride of 1 to further extract the fused feature information to obtain the final output feature map:

[0157] z 1out = SiLU(Conv2d(Norm(z 1r )))

[0158] z 2out =SiLU(Conv2d(Norm(z 2r )))

[0159] For z3, the convolutional layer output with a kernel of 3×3 and a stride of 1 is also used, and then the output is continuously downsampled through two convolutions to enhance feature abstraction and semantic information extraction:

[0160] z 3out =SiLU(Conv2d(Norm(z'3)))

[0161] z 4out =SiLU(Conv2d(Norm(z 3out )))

[0162] z 5out =SiLU(Conv2d(Norm(z 4out )))

[0163] In this way, by upsampling, the low-resolution feature map is increased to the same size as the high-resolution feature map, and feature alignment at different scales is achieved, so that fine-grained spatial information can be captured; by combining multi-scale information, it helps to capture cross-scale contextual information, so that the network can better understand the relationship between global information and local details; by downsampling the output, excessive information loss is avoided while improving the network's expression and generalization capabilities, making the features smooth and enhancing robustness.

[0164] 4. Input the final predicted feature map into the detection head, and output the arbitrarily distributed detection box representation and the detection box quality estimation; wherein the detection head includes an arbitrarily distributed detection box position regression branch and a detection box quality prediction branch. Including:

[0165] The final predicted feature map is input into a combination module including 4 Conv2d+GN+ReLU, and the output of the combination module is divided into two branches. Both branches pass through a convolution layer with a convolution kernel size of 3×3 and a step size of 1, and then are input into the detection frame position regression branch and the detection frame quality prediction branch for output.

[0166] (1) Detection box representation:

[0167] like Figure 3 As shown, let the minimum and maximum distances of the positive sample from the bounding box be z min and z max , the distance between this position and the true bounding box is z label , the distance to the predicted bounding box is z pre , take z0 so that it satisfies Take zm Make it satisfy The integral on the continuous domain [z0, z m is converted to a discrete representation {z0, z1, …, z m}, and the interval ω = 0.5 is set. Given the discrete distribution property Then:

[0168]

[0169] That is,

[0170]

[0171] z pre 's P pre (z q ) is obtained by calculating the softmax function on the m + 1 numbers output by the detection head. Therefore, this branch represents the predicted detection box by outputting the basic generalized distribution P(z q ) and further calculation (such as Figure 4 ).

[0172] (2) Detection box quality estimation:

[0173] Since the dense object detector ranks all candidate samples sampled from the entire image, the present invention uses the IoU score between 0 and the positive sample prediction box and the corresponding ground truth box as the ground truth label. Then, the negative samples will be supervised with a quality score of 0, making the overall quality prediction more reliable. The label of the positive sample is not just 1, but a number in the range (0, 1], and this score is dynamically calculated during training. The detection box quality prediction value is obtained by calculating the sigmoid function on the output of the detection head.

[0174] 5. Weight the detection box position regression branch and the detection box quality prediction branch and fuse the directionality to obtain a boundary blur perception loss function for optimizing the whole process of small object detection. It includes:

[0175] Let z j and z j+1 be two numbers near z label , and z j ≤ z label ≤ z j+1 . Then:

[0176]

[0177] Among them:

[0178]

[0179] The detection box representation loss function only needs to calculate the predicted value P pre(z j ), P pre (z j+1 ) and the true label value P label (z j ), P label (z j+1 ) is the weighted loss (as shown in Figure 4 ). The formula is:

[0180] L R = -(P label (z j ) ln(P pre (z j )) + P label (z j+1 ) ln(P pre (z j+1 )))

[0181] Considering the characteristics of dense and small targets, a small positioning error will also cause a significant decrease in the traditional IoU. Therefore, the loss function of the present invention introduces GIoU Loss, which not only focuses on the overlapping area but also can focus on other non-overlapping areas, and can better reflect the intersection of two bounding boxes in the closure area. The following formula:

[0182]

[0183] Among them, A and B represent the predicted bounding box and the true bounding box respectively, and C represents the smallest bounding box that can enclose them. However, it will also bring the problems of slow convergence speed and low efficiency. Therefore, on this basis, SIoU Loss is introduced again. It has the ability of scale perception and angle perception, and improves the accuracy of target detection in complex environments. The following formula:

[0184]

[0185] Among them, Δ represents the distance loss, which includes the angle loss, and Ω represents the shape loss. The weighted combination of the two is obtained:

[0186]

[0187] According to the experiment, η1 = 0.8 and η2 = 0.2.

[0188] The quality loss of the detection bounding box is improved on the basis of BCELoss. The formula is:

[0189] L Q = -|z pre - z label | α ((1 - z label ) ln(1 - z pre ) + zlabel ln(z pre ))

[0190] where z label represents the true label, and z pre represents the predicted value. It is measured experimentally that the best effect is achieved when α = 2.

[0191] Finally, the loss function of fuzzy perception is:

[0192]

[0193] where the entire summation is calculated over all feature map positions (x, y), p x,y represents the estimated detection box quality score at that point, represents the IoU score between the predicted box and the true box corresponding to that point, t x,y represents the predicted box information at that point, represents the true box information at that point, s x,y represents the probability of two discrete points predicted to be adjacent to the true box at that point, represents the distance between that point and the true box, N pos represents the number of positive samples, λ1 represents the weight of L GS set to 2, λ2 represents the weight of L R Since it is averaged in four directions, it is set to 1 / 4, represents the indicator function, when it is 1, otherwise 0.

[0194] Therefore, the present invention provides a dense small target detection method based on fuzzy perception. By constructing a multi-level nested Mamba network, global and local features are extracted and fused layer by layer, the correlation between the two is deeply mined and fully utilized, and the ability to capture detailed information is enhanced to solve the problem of small targets. At the same time, by designing a boundary box fuzzy perception loss function that fuses directionality, the network can efficiently and intensively learn the probability of adjacent continuous values of the target boundary box position under any flexible distribution, and then accurately locate the thermal radiation targets with a relatively large spatial distribution density from complex scenes, improving the accuracy of infrared dense small target detection.

[0195] In this specification, each embodiment is described in a progressive manner. The key point of each embodiment is to illustrate the differences from other embodiments. The same or similar parts among the embodiments can be referred to each other.

[0196] In this text, specific examples are used to illustrate the principles and implementation manners of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for those of ordinary skill in the art, according to the idea of the present invention, there will be changes in the specific implementation manners and application scopes. In summary, the content of this specification should not be construed as a limitation on the present invention.

Claims

1. A dense small target detection method based on fuzzy perception, characterized in that It includes the following steps: Obtain an infrared image, evenly divide the infrared image into multiple regions, evenly divide each region into multiple blocks, and then evenly divide each block into multiple sub-blocks; Use a nested four-way Mamba network to extract the features of the region, the block, and the sub-block respectively, obtain shallow features, middle-level features, and deep features, and then embed and fuse the shallow features, the middle-level features, and the deep features in order from shallow to deep to obtain an initial feature map; wherein, the nested four-way Mamba network includes a convolutional trunk for generating region maps, block maps, and sub-block maps and an encoding structure for feature extraction; Transfer the initial feature map into a multi-scale context collaboration network, and output a final predicted feature map through information transfer and feature fusion operations; Input the final predicted feature map into a detection head, and output detection box representations and detection box quality estimates with arbitrary distributions; wherein, the detection head includes a detection box position regression branch and a detection box quality prediction branch with arbitrary distributions; Weight the detection box position regression branch and the detection box quality prediction branch and fuse the directivity to obtain a boundary blur perception loss function for optimizing the entire process of small target detection.

2. The method for detecting dense small targets based on blur perception according to claim 1, wherein: The convolutional trunk generates regions, blocks, and sub-blocks with different dimensions from the infrared image, maps the sub-blocks into the blocks using a 2×2 sliding window with a stride of 2, and maps the blocks into the regions using a 4×4 sliding window with a stride of 4; wherein, each block is composed of 2×2 sub-blocks, and each region is composed of 4×4 blocks; The encoding structure sets the feature extraction processes of the sub-blocks, the blocks, and the regions as the shallow stage, the middle-level stage, and the deep stage respectively, and each layer of each stage is composed of two four-way Mamba structures.

3. The method for detecting dense small targets based on fuzzy perception according to claim 2, wherein The feature extraction process of the shallow stage includes: Using the four-way Mamba structure in the shallow stage, by calculating the interaction magnitude between two adjacent sub-blocks, construct a relationship representation between the sub-blocks; the relationship representation formula between the sub-blocks is: Among them, X m is the sub-block representation mapped to the block, a represents the a-th four-way Mamba structure in each stage, a = 1, 2, LN represents layer normalization, and DP represents the regularization operation; Combined with the relationship representation between the sub-blocks, embed the sub-blocks mapped to the block into the block, and the sub-block embedding expression is: Among them, y m is the block representation mapped to the region, and FC represents the fully connected layer.

4. A method for detecting dense small targets based on fuzzy perception according to claim 3, characterized in that The feature extraction process of the middle-level stage includes: Using the four-way Mamba structure in the middle-level stage to express the relationship between the blocks, the relationship expression between the blocks is: where Y n is the set of block mappings for the n-th region; Combined with the relationship between the blocks, embed the blocks mapped to the region into the region, and the block embedding expression is: Among them, z n represents the nth region.

5. A method for detecting dense small targets based on fuzzy perception according to claim 4, characterized in that, The feature extraction process of the deep stage includes: Using the four-way Mamba structure in the deep stage to express the relationship between the regions, the relationship expression between the regions is:

6. A dense small target detection method based on fuzzy perception according to claim 5, characterized in that, The process of calculating using the four-way Mamba structure includes: S1. Transpose the input and divide the input into a first branch and a second branch for calculation; S2. Use a 2D convolution with a kernel of 1×1 to divide the input into multiple non-overlapping spatial blocks and flatten and project them into the embedding dimension to obtain an initial embedding vector; S3. Add learnable spatial positions to the initial embedding vectors, perform normalization operations, forward calculations, and backward calculations on the initial position embedding vectors after adding spatial positions, and then linearly project the calculated initial embedding vectors into sub-blocks and regions of a new dimension to obtain sub-blocks of the new dimension and regions of the new dimension; S4. Perform per-channel 1D convolution, linear projection, and smooth activation on the sub-blocks of the new dimension in sequence, and then introduce learnable parameters for discretization to obtain continuously updated hidden layer states and outputs of each layer; S5. Activate the regions of the new dimension, stack the outputs of each layer, and use the activated outputs of the regions of the new dimension to enhance the results; S6. For the first branch and the second branch, perform the operations of S2 - S5 respectively, then add the output results of the first branch and the second branch, output the final result, and obtain the initial feature map.

7. A dense small target detection method based on fuzzy perception according to claim 6, characterized in that, Transfer the initial feature map into the multi-scale context collaboration network, and through information transfer and feature fusion operations, output the final predicted feature map, including: Use the convolutional layer of the multi-scale context collaboration network to unify the dimensions of the initial feature map to maintain the same number of channels; Through upsampling operations, lift the low-resolution feature map to the same size as the high-resolution feature map for multi-scale feature fusion of the initial feature map; Extract information from the initial feature map after multi-scale feature fusion to output the final predicted feature map.

8. A dense small target detection method based on fuzzy perception according to claim 7, characterized in that, Input the final predicted feature map into the detection head to output detection box representations with arbitrary distributions and detection box quality estimates, including: Input the final predicted feature map into a combined module including 4 Conv2d + GN + ReLU, divide the output of the combined module into two branches, both branches pass through convolutional layers respectively, and then are input into the detection box position regression branch and the detection box quality prediction branch respectively; Output a basic generalized distribution through the detection box position regression branch and represent the predicted detection box, and perform sigmoid function calculation through the detection box quality prediction branch to obtain the detection box quality prediction value.

9. A method for detecting dense small targets based on fuzzy perception according to claim 8, characterized in that, Weight the detection box position regression branch and the detection box quality prediction branch and fuse the directionality to obtain a boundary ambiguity perception loss function for optimizing the entire process of small target detection, including: Introduce GIoU Loss and SIoU Loss into the loss function of the detection box position regression branch to obtain a detection box representation weighted loss function, introduce BCELoss into the loss function of the detection box quality prediction branch to obtain a detection box quality weighted loss function, and combine the detection box representation weighted loss function and the detection box quality weighted loss function to obtain the boundary ambiguity perception loss function; the expression of the boundary ambiguity perception loss function is: Among them, (x, y) represents the position of the feature map, and p x,y represents the estimated detection box quality score at this point, represents the IoU score between the predicted box and the ground truth box corresponding to this point, and t x,y represents the predicted box information at this point, represents the ground truth box information at this point, and s x,y represents the probability of two discrete points predicted to be adjacent to the ground truth box at this point, represents the distance between this point and the ground truth box, and N pos represents the number of positive samples, λ1 represents the weight of L GS and λ2 represents the weight of L R represents the indicator function, when it is 1, otherwise it is 0.​

Images