Interpretable small object detection method based on Gaussian curvature

Abstract

The present invention discloses an interpretable small target detection method based on Gaussian curvature, which mainly solves the problem of low detection accuracy of existing methods. The implementation steps are as follows: obtaining a training sample set and a test sample set; constructing an interpretable small target detection model based on Gaussian curvature: including a backbone network and a structural feature enhancement network arranged in parallel with it; initializing parameters; training the small target detection model; updating the parameters of the small target detection model; and obtaining small target detection results. In the present invention, the backbone network is used to extract features of different scales. In the structural feature enhancement network, a Gaussian curvature-guided structure extraction block is constructed in combination with mathematical theory, making the model highly interpretable. It effectively extracts texture structure information through Gaussian curvature calculation. The feature fusion block then fuses the texture structure information with multi-scale feature information to compensate for target features lost in the downsampling operation of the backbone network, thereby improving the accuracy of small target detection.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and in particular relates to an interpretable small target detection method based on Gaussian curvature, which can be used for marine rescue and monitoring security systems. Background Art

[0002] With the continuous development of deep learning technology, existing algorithms have shown good detection effects on medium and large targets. However, for application scenarios that mainly detect small targets, since the area of ​​the detection object in the sample image accounts for a very small proportion, while the background noise accounts for a large proportion, missed detection or false detection problems are prone to occur, which poses a challenge to small target detection.

[0003] For example, in its patent application, "A Small Target Detection Method with Multi-Scale Features and Adaptive Weights" (patent application number: CN202310205418.5, application publication number: CN116258940A), Anhui University of Information Engineering adds a cross-scale feature layer at each scale of the ResNet network and introduces a large convolution kernel with a channel attention mechanism to facilitate the extraction of target features. The feature maps of the four scales obtained by the backbone network are then fused and an improved FPN+PAN structure with adaptive weights of multi-scale features is added. The FPN structure passes down the strong semantic features of the high-level layers and enhances the semantic information of the entire pyramid. PAN passes the strong positioning information of the low-level network to the high-level features, thereby facilitating the improvement of positioning accuracy. In this invention, the model has low interpretability. After the data is down-sampled, the area of ​​the detection target is further compressed, the feature information of small targets is easily lost, and the extracted texture structure details are unclear, resulting in low detection accuracy. Summary of the Invention

[0004] In order to solve the above problems existing in the prior art, the present invention provides an interpretable small target detection method based on Gaussian curvature, which is used to solve the technical problem of low detection accuracy existing in the existing small target detection methods.

[0005] To achieve the above object, the technical solution adopted by the present invention includes the following steps:

[0006] (1) Obtain training sample set and test sample set:

[0007] Obtain K small target images and label the small target objects in each small target image. Then, combine the M small target images and their corresponding labels into a training sample set R1, and combine the remaining KM small target images and their corresponding labels into a test sample set E1, where K≥500.

[0008] (2) Construct an interpretable small target detection model O based on Gaussian curvature:

[0009] A small target detection network model O is constructed, which includes a backbone network and a structural feature enhancement network arranged in parallel with the backbone network; the backbone network includes a Stem block, N feature extraction blocks, an upsampling block and a Head block connected in sequence; the structural feature enhancement network is loaded between the Stem block and the Head block of the backbone network, and includes a Gaussian curvature-guided structural extraction block, N feature fusion blocks, and a normalization block connected in sequence; the output end of the nth feature extraction block in the backbone network is connected to the input end of the nth feature fusion block in the structural feature enhancement network; the Gaussian curvature-guided structural extraction block includes a Gaussian curvature operator and two deconvolution layers connected in sequence; the feature fusion block includes two residual blocks connected in sequence and a convolution block composed of three stacked convolution layers, the input end of the convolution block is also connected to a stacked convolution layer and an upsampling layer, the residual block includes a plurality of composite layers connected in sequence, and each composite layer includes a convolution layer, a normalization layer and a nonlinear activation layer connected in sequence; wherein N ≥ 1;

[0010] (3) Initialization parameters:

[0011] The number of initialization iterations is t, the maximum number of iterations is T, T ≥ 1000, and the small target detection model O of the tth iteration t The weight and bias parameters in are w t 、b t , and let t = 0, O t =O;

[0012] (4) Training the small target detection model O:

[0013] Randomly select L training samples with replacement from the training sample set R1 as the input of the small target detection model O for forward propagation, and obtain L small target detection results, where 1≤L≤M;

[0014] (5) Update the parameters of the small target detection model:

[0015] The L small target detection results obtained in step (4) are used to detect the small target model O t The weight and bias parameter w t 、b t Update and get the network model O of this iteration t ; Determine whether t≥T holds true. If so, obtain the trained small target detection model O*. Otherwise, set t=t+1 and execute step (4);

[0016] (6) Obtain small target detection results:

[0017] The test sample set E1 is used as the input of the trained small target detection model O* for forward propagation to obtain the small target detection results corresponding to KM test samples.

[0018] Compared with the prior art, the present invention has the following advantages:

[0019] The interpretable Gaussian curvature-based small target detection model constructed by the present invention includes a backbone network and a structural feature enhancement network. In the process of training the model and obtaining small target detection results, the backbone network is used to extract features of different scales. In the structural feature enhancement network, the Gaussian curvature-guided structure extraction block combines mathematical theory to make the model highly interpretable. It extracts texture structure information from shallow features in the backbone network through Gaussian curvature calculation. The feature fusion block fuses the texture structure information with feature information of different scales to compensate for the target features lost in the downsampling operation in the backbone network and enhance the detailed features of the target. Experimental results show that the present invention can effectively improve the accuracy of small target detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 Flowchart for the implementation of the present invention;

[0021] Figure 2 A schematic diagram of the structure of an interpretable Gaussian curvature-based small target detection model used in an embodiment of the present invention;

[0022] Figure 3 A schematic diagram of the structure of a feature extraction block used in an embodiment of the present invention;

[0023] Figure 4 This is a schematic diagram of the structure of the feature fusion block used in an embodiment of the present invention;

[0024] Figure 5 This is a schematic diagram of the structure of the residual block used in an embodiment of the present invention. DETAILED DESCRIPTION

[0025] The present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0026] Reference Figure 1 , the present invention comprises the following steps:

[0027] (1) Obtain training sample set and test sample set:

[0028] Obtain 1000 small target images contained in the IRSTD-1k dataset, and label the small targets in each small target image. Then, 600 small target images and their corresponding labels form the training sample set R1, and the remaining 400 small target images and their corresponding labels form the test sample set E1;

[0029] (2) Construct an interpretable small target detection model O based on Gaussian curvature:

[0030] Reference Figure 2 , further describing the interpretable Gaussian curvature-based small target detection model adopted in the embodiment of the present invention.

[0031] The feature fusion block includes two sequentially connected residual blocks and a convolution block consisting of three stacked convolutional layers. The input end of the convolution block is also connected to a stacked convolutional layer and an upsampling layer. The residual block includes multiple sequentially connected composite layers, each of which includes a sequentially connected convolutional layer, a normalization layer, and a nonlinear activation layer.

[0032] A small target detection network model O is constructed, which includes a backbone network and a structural feature enhancement network arranged in parallel with the backbone network; the backbone network includes a Stem block, three feature extraction blocks, an upsampling block and a Head block connected in sequence; the structural feature enhancement network is loaded between the Stem block and the Head block of the backbone network, including a Gaussian curvature-guided structural extraction block, three feature fusion blocks, and a normalization block connected in sequence; the output end of the first feature extraction block in the backbone network is also connected to the input end of the first feature fusion block, the output end of the second feature extraction block is also connected to the input end of the second feature fusion block, and the output end of the third feature extraction block is also connected to the input end of the third feature fusion block; wherein, the Stem module includes a first convolutional layer, a second convolutional layer, a third convolutional layer, and a pooling layer connected in sequence; the three feature extraction blocks adopt the ResNet-20 structure, and each feature extraction block includes a fourth convolutional layer, a fifth convolutional layer, and a pooling layer connected in sequence. The sixth convolutional layer, the seventh convolutional layer, the eighth convolutional layer, and the ninth convolutional layer; the upsampling block adopts the interpolate operation, and the output space size is the input image size; the Gaussian curvature guided structure extraction block includes the Gaussian curvature operator, the first deconvolution layer, and the second deconvolution layer connected in sequence; the feature fusion block includes the first residual block, the second residual block, the eleventh convolutional layer, the twelfth convolutional layer, and the thirteenth convolutional layer connected in sequence, and the input end of the eleventh convolutional layer is also connected to the stacked tenth convolutional layer and the upsampling layer. The residual block includes the fourteenth convolutional layer, the first normalization layer, the first nonlinear activation layer, the fifteenth convolutional layer, the second normalization layer, and the second nonlinear activation layer connected in sequence; the normalization block includes the sixteenth convolutional layer and the third nonlinear activation layer connected in sequence; the Head module includes the seventeenth convolutional layer, the third normalization layer, the fourth nonlinear activation layer, the random dropout layer, and the eighteenth convolutional layer connected in sequence;

[0033] The specific parameters are set as follows: the convolution kernel size of the first convolution layer is 3*3, the step size is 2, and the padding is 1; the convolution kernel size of the second and third convolution layers is 3*3, the step size is 1, and the padding is 1; the convolution kernel size of the convolution layer in the first feature extraction block is 3*3, the step size is 1, and the padding is 1; the convolution kernel size of the convolution layer in the second and third feature extraction blocks is 3*3, the step size is 2, and the padding is 1; the convolution kernel size of the tenth, eleventh, twelfth, and thirteenth convolution layers is 1*1, the step size is 1; the convolution kernel size of the fourteenth and fifteenth convolution layers is 3*3, the step size is 1, and the padding is 1 ; The convolution kernel size of the sixteenth convolution layer is 1*1, and the stride is 1; the convolution kernel size of the seventeenth convolution layer is 3*3, the stride is 1, and the padding is 1; the convolution kernel size of the eighteenth convolution layer is 1*1, and the stride is 1; the convolution kernel size of the first deconvolution layer and the second deconvolution layer are both 4*4, with a stride of 2 and a padding of 1; the pooling layer uses maximum pooling; the upsampling layer uses interpolate operation, and the output space size is the input image size; the normalization layers all use layer normalization; the first nonlinear activation layer, the second nonlinear activation layer, and the fourth nonlinear activation layer all use ReLU function, and the third nonlinear activation layer uses Sigmoid function;

[0034] In the structural feature enhancement network, the Gaussian curvature-guided structure extraction block uses the Gaussian curvature operator. The model combines mathematical theory to make it highly interpretable. It extracts gradient-based information from the image through Gaussian curvature calculation, which contains richer texture structure information. According to differential geometry, the Gaussian curvature operator K is used to calculate the Gaussian curvature of the feature map I, which can be defined as:

[0035]

[0036] Among them, I x is the first-order gradient of the feature map I in the x direction, I y is the first-order gradient of the feature map I in the y direction, I xx is the second-order gradient of feature map I in the x direction, I yy is the second-order gradient of feature map I in the y direction, I xy is the second-order gradient of feature map I in the x and y directions, and the calculation formula is:

[0037] I x =I*G x '(2)

[0038] I y =I*G y '(3)

[0039] I xx =I*G x ” x(4)

[0040] I yy =I*G y ” y (5)

[0041] I xy =I*G x ” y (6)

[0042]

[0043]

[0044]

[0045]

[0046]

[0047] Among them, G' x is the first-order partial derivative of the Gaussian function with respect to x, G' y is the first-order partial derivative of the Gaussian function with respect to y, G' x ' x is the second-order partial derivative of the Gaussian function with respect to x, G' y ' y is the second-order partial derivative of the Gaussian function with respect to y, G' x ' y is the second-order partial derivative of the Gaussian function with respect to x and y.

[0048] The Gaussian curvature-guided structure extraction block calculates Gaussian curvature and upsamples the shallow feature map x1 output by the Stem block to obtain the coarse structure information feature map g1, which can be expressed as:

[0049] g1=Deconv(K(x1)) (12)

[0050] Among them, K(·) is the Gaussian curvature operator that calculates the Gaussian curvature of the shallow feature map x1, and Deconv(·) is the deconvolution layer that performs upsampling operations.

[0051] The feature fusion block fuses and refines the multi-scale features and coarse structural information feature maps obtained by the backbone network. The output of the third feature fusion block is normalized as the weight of the structural information and fused with the global features obtained by the backbone network to enhance the target detail information. It is used to constrain the segmentation branch and improve the small target detection performance.

[0052] (3) Initialization parameters:

[0053] The number of initialization iterations is t, the maximum number of iterations is T = 1000, and the small target detection model O of the tth iteration t The weight and bias parameters in are w t 、b t , and let t = 0, O t =O;

[0054] (4) Training the small target detection model O:

[0055] 32 training samples are randomly selected with replacement from the training sample set R1 as the input of the small target detection model O for forward propagation:

[0056] (4a) The input image size is 512*512 and the number of channels is 3. The Stem block in the backbone network downsamples each image to obtain a shallow feature map x1, whose size is 128*128 and the number of channels is 16. The first feature extraction block extracts features from the feature map to obtain a local feature map x2, whose size is 128*128 and the number of channels is 16. The second feature extraction block extracts features from the feature map to obtain a local feature map x3, whose size is 64*64 and the number of channels is 32. The third feature extraction block extracts features from the feature map to obtain a local feature map x4, whose size is 32*32 and the number of channels is 64. The upsampling block upsamples the local feature map x4 to obtain a global feature map x5, whose size is 512*512 and the number of channels is 64.

[0057] (4b) The Gaussian curvature guided structure extraction block in the structural feature enhancement network calculates the Gaussian curvature of the shallow feature map x1 and upsamples it to obtain the coarse structure information feature map g1, whose size is 512*512 and the number of channels is 16. The first feature fusion block fuses and refines the local feature map x2 with the coarse structure information feature map g1 to obtain the fine structure information feature map g2, whose size is 512*512 and the number of channels is 32. The second feature fusion block fuses and refines the local feature map x3 with the fine structure information feature map g1 to obtain the fine structure information feature map g2. The feature fusion and refinement of the image g2 are performed to obtain the fine structure information feature map g3, whose size is 512*512 and the number of channels is 64. The third feature fusion block performs feature fusion and refinement on the local feature map x4 and the fine structure information feature map g3 to obtain the fine structure information feature map g4, whose size is 512*512 and the number of channels is 128. The normalization block normalizes the fine structure information feature map g4 to obtain the target structure enhancement feature map g5, whose size is 512*512 and the number of channels is 1;

[0058] (4c) After element-by-element multiplication of the global feature map x5 and the target structure enhanced feature map g5, a fused feature map x6 is obtained. The fused feature map x6 is added element-by-element to the target structure enhanced feature map g5 to obtain a structure information guided fused feature map x7, which has a size of 512*512 and 64 channels. The Head block performs target detection on the structure information guided fused feature map x7 and obtains 32 small target detection results.

[0059] (5) Update the parameters of the small target detection model:

[0060] The detection results of 32 small targets obtained by step (4) are used to test the small target detection model O t The weight and bias parameter w t 、b t Update and get the network model O of this iteration t :

[0061] (5a) Using the Dice loss function, the detection loss value L is calculated by generating 32 small target detection results and the labels of the corresponding L images. seg , the structural information loss value L is calculated by generating 32 target structure enhancement feature maps and the labels of the corresponding L images cur , L seg and L cur Add up to get the total loss L t :

[0062]

[0063]

[0064] L t =L seg +L cur

[0065] Among them, r l represents the category of each pixel in the input image, p l Indicates the probability that each pixel in the small target detection result belongs to the label category, q l It represents the probability that each pixel in the target structure enhancement feature map belongs to the label category, and ε represents the correction factor, which is any real number in the range of (0, 0.1). Its function is to prevent the denominator of the fraction from being zero.

[0066] (5b) Calculate L by the chain rule t For the weight parameter ω t and bias parameter b t Partial derivative of and Finally, according to Right t、b t To update:

[0067]

[0068]

[0069] Among them, ω t 、b t Indicates O t The weights and bias parameters of all learnable parameters, w t '、b t ' indicates ω t 、b t The update result of , α represents the learning rate; determine whether t ≥ T is established, if so, obtain the trained small target detection model O*, otherwise, set t = t + 1 and execute step (4).

[0070] (6) Obtain small target detection results:

[0071] The test sample set E1 is used as the input of the trained small target detection model O* for forward propagation to obtain the small target detection results corresponding to 400 test samples.

[0072] The following is a description of the technical effects of the present invention in conjunction with simulation experiments.

[0073] Simulation conditions, content and result analysis:

[0074] The hardware platform for the simulation experiment is an Intel(R) Core i9-9900K CPU with a main frequency of 3.5GHz, 32GB of memory, and an NVIDIA GeForce RTX 2080Ti graphics card. The software platform for the simulation experiment is an Ubuntu 16.04 operating system, Python version 3.7, and PyTorch version 1.7.1.

[0075] The intersection over union (IoU) and accuracy evaluation indicators are used to compare the detection performance of the patent document "A small target detection method with multi-scale features and adaptive weights" (patent application number: CN202310205418.5, application publication number: CN116258940A) and the present invention on the test samples. The intersection over union (IoU) of the small target detection results of the existing method is 62.16%, and the accuracy is 94.64%. The intersection over union (IoU) of the small target detection results of the present invention is 65.24%, and the accuracy is 97.58%. Compared with the existing technology, the detection accuracy of the present invention is significantly improved.

Claims

1. An interpretable small target detection method based on Gaussian curvature, characterized by: The following steps are involved: (1) Obtain training sample set and test sample set: Obtain K small target images and label the small target objects in each small target image. Then, combine the M small target images and their corresponding labels into a training sample set R1, and combine the remaining KM small target images and their corresponding labels into a test sample set E1, where K≥500. (2) Construct an interpretable small target detection model O based on Gaussian curvature: Construct a small target detection network model O including a backbone network and a structural feature enhancement network arranged in parallel with it; the backbone network includes a Stem block, N feature extraction blocks, an upsampling block and a Head block connected in sequence; The structural feature enhancement network is loaded between the Stem block and the Head block of the backbone network, and includes a Gaussian curvature-guided structure extraction block, N feature fusion blocks, and a normalization block connected in sequence. The output of the nth feature extraction block in the backbone network is connected to the input of the nth feature fusion block in the structural feature enhancement network. The Gaussian curvature-guided structure extraction block includes a Gaussian curvature operator and two deconvolution layers connected in sequence. The feature fusion block includes two residual blocks connected in sequence and a convolution block composed of three stacked convolution layers. The input of the convolution block is also connected to a stacked convolution layer and an upsampling layer. The residual block includes multiple composite layers connected in sequence, and each composite layer includes a convolution layer, a normalization layer, and a nonlinear activation layer connected in sequence. Wherein, N ≥ 1. (3) Initialization parameters: The number of initialization iterations is t, the maximum number of iterations is T, T ≥ 1000, and the small target detection model O of the tth iteration t The weight and bias parameters in are w t 、b t , and let t = 0, O t =O; (4) Training the small target detection model O: Randomly select L training samples with replacement from the training sample set R1 as the input of the small target detection model O for forward propagation, and obtain L small target detection results, where 1≤L≤M; (5) Update the parameters of the small target detection model: The L small target detection results obtained in step (4) are used to detect the small target model O t The weight and bias parameter w t 、b t Update and get the network model O of this iteration t ; Determine whether t≥T holds true. If so, obtain the trained small target detection model O*. Otherwise, set t=t+1 and execute step (4); (6) Obtain small target detection results: The test sample set E1 is used as the input of the trained small target detection model O* for forward propagation to obtain the small target detection results corresponding to KM test samples.

2. The interpretable small target detection method based on Gaussian curvature according to claim 1, characterized in that: The small target detection network model O described in step (2), wherein: The backbone network includes a Stem block, which includes multiple convolutional layers and a pooling layer connected in sequence; a feature extraction block includes multiple convolutional layers connected in sequence; and a Head module includes a convolutional layer, a normalization layer, a nonlinear activation layer, a random dropout layer, and a convolutional layer connected in sequence. The structural feature enhancement network includes a normalization block, a convolutional layer and a nonlinear activation layer connected in sequence.

3. The interpretable small target detection method based on Gaussian curvature according to claim 1, characterized in that: The small target detection model O described in step (4) is trained by the following steps: (4a) The Stem block in the backbone network downsamples each image to obtain L shallow feature maps, N feature extraction blocks extract features from each shallow feature map to obtain L local feature maps, and the upsampling block upsamples each local feature map to obtain L global feature maps; (4b) The Gaussian curvature-guided structure extraction block in the structural feature enhancement network calculates Gaussian curvature and upsamples each shallow feature map output by the Stem block to obtain L coarse structural information feature maps; the first feature fusion block fuses and refines each local feature map output by the first feature extraction block and the coarse structural information feature map output by the structure extraction block. The second to N feature fusion blocks sequentially fuse and refine the output results of the feature fusion block of the previous level and the output results of the corresponding feature extraction block to obtain L fine structural information feature maps; The normalization block normalizes L fine structure information feature maps to obtain L target structure enhancement feature maps; (4c) Each global feature map output by the backbone network is fused with each target structure enhancement feature map output by its corresponding Gaussian curvature network to obtain L structure information guided fusion feature maps. The Head block performs target detection on the L structure information guided fusion feature maps to obtain L small target detection results.

4. The interpretable small target detection method based on Gaussian curvature according to claim 3, characterized in that: The Gaussian curvature-guided structure extraction block described in step (4b) calculates the Gaussian curvature of each shallow feature map output by the Stem block. The specific process is: According to differential geometry, the Gaussian curvature calculation of the feature map I using the Gaussian curvature operator K can be defined as: Among them, I x is the first-order gradient of the feature map I in the x direction, I y is the first-order gradient of the feature map I in the y direction, I xx is the second-order gradient of feature map I in the x direction, I yy is the second-order gradient of feature map I in the y direction, I xy is the second-order gradient of feature map I in the x and y directions, and the calculation formula is: I x =I*G x ' I y =I*G y ' I xx =I*G x ” x I yy =I*G y ” y I xy =I*G x ” y Among them, G' x is the first-order partial derivative of the Gaussian function with respect to x, G' y is the first-order partial derivative of the Gaussian function with respect to y, G' x ' x is the second-order partial derivative of the Gaussian function with respect to x, G' y ' y is the second-order partial derivative of the Gaussian function with respect to y, G' x ' y is the second-order partial derivative of the Gaussian function with respect to x and y.

5. The interpretable small target detection method based on Gaussian curvature according to claim 1, characterized in that: The parameters of the small target detection model described in step (5) are updated as follows: (5a) Using the Dice loss function, the detection loss value L is calculated by generating L small target detection results and the labels of the corresponding L images. seg , the structural information loss value L is calculated by generating L target structure enhancement feature maps and the labels of the corresponding L images cur , L seg and L cur Add up to get the total loss L t : L t =L seg +L cur Among them, r l Represents the label category of each pixel in the input image, p l Indicates the probability that each pixel in the small target detection result belongs to the label category, q l represents the probability that each pixel in the target structure enhancement feature map belongs to the label category, and ε represents the correction factor; (5b) Calculate L by the chain rule t For the weight parameter ω t and bias parameter b t Partial derivative of and Finally, according to Right t 、b t To update: Among them, ω t 、b t Indicates O t The weights and bias parameters of all learnable parameters, w t '、b t ' indicates ω t 、b t The update result of , α represents the learning rate.

Images