Small target detection method based on deformable attention feature fusion

Abstract

The present invention discloses a small target detection method based on deformable attention feature fusion, step S100, obtaining a data set and performing data preprocessing; step S200, constructing an enhanced small target lung nodule detection network model based on a deformable attention mechanism; step S300, training an enhanced small target lung nodule detection network model based on a deformable attention mechanism; step S400, inputting an image to be detected into the trained enhanced small target lung nodule detection network model based on a deformable attention mechanism, and outputting a detection result. Aiming at the characteristic that the features of small-sized targets are concentrated in shallow features, the present invention adds a four-fold down-sampling detection layer in the feature fusion pyramid, supplements the spatial position and other information of small targets omitted in the original detection layer, helps to extract and enhance the feature representation of small targets, and at the same time reduces the interference of background redundant features, thereby improving the detection accuracy of small targets.

Description

Technical Field

[0001] The present invention relates to the field of image detection technology, and in particular to a small target detection method based on deformable attention feature fusion. Background Art

[0002] In image detection, especially in the field of small target detection in images, such as lung nodule detection, since the diameter of lung nodules is generally less than 3cm, and some are even only a few millimeters in size, they are easily overlooked or mistaken for normal structures during detection. In addition, since lung nodules have diverse shapes, are multi-scale and multi-morphological, irregular and have blurred edges, and the lung tissue itself contains a large number of tiny blood vessels and bronchi, they will interfere with the detection of lung nodules. These are the main reasons for the low accuracy of lung nodule detection.

[0003] Traditional lung nodule detection methods are mainly based on artificially extracted features and classifier models for lung nodule detection and identification, mainly introducing multi-scale filters to enhance and detect lung nodules; lung nodule detection based on region growing algorithm, effectively extracting lung nodules, and being able to handle problems such as noise and edge blur in images; and introducing random forest models in machine learning algorithms to extract features related to lung nodules by analyzing local areas in images; training decision trees to achieve lung nodule identification and detection, etc. These methods are easily affected by artificial factors, and it is difficult to extract features of complex nodules, which has great limitations.

[0004] With the development of deep learning, methods based on convolutional neural networks have also been introduced into the field of lung nodule detection. The network structure is generally designed based on two-dimensional convolutional neural networks and three-dimensional convolutional neural networks. The three-dimensional lung nodule detection algorithm can use the contextual information of nodules in three-dimensional space for detection. However, due to the need to process three-dimensional volume data, there are problems such as long model training time, high requirements for computing resources, huge model parameters, large data sample requirements, and easy overfitting, resulting in poor detection results. At present, the two-dimensional lung nodule target detection algorithms are all based on deep learning target detection networks, such as the two-stage Faster-RCNN series and the single-stage YOLO series. The Faster-RCNN framework is applied to lung nodule detection, and the candidate region generation network is optimized. A densely connected network is used for feature extraction, and feature information is effectively transmitted and reused. However, due to the complexity of the network structure, it is difficult to meet the real-time requirements of lung nodule detection. In the single-stage lung nodule detection algorithm, the attention mechanism and multi-scale feature fusion method are mainly introduced to automatically learn the salient areas in the image and improve the detection effect of lung nodules. However, the extracted lung nodule features are not further processed before fusion to reduce the negative impact of irrelevant redundant features on the detection results. Summary of the invention

[0005] The purpose of the present invention is to provide a small target detection method based on deformable attention feature fusion, which is used to solve the problems in the detection methods in the prior art that it is difficult to extract complex nodule features, the introduction of redundant information when fusion of feature layers has a negative impact on the detection results, it is difficult to meet real-time detection requirements or the detection effect is poor.

[0006] The present invention solves the above problems through the following technical solutions:

[0007] A small target detection method based on deformable attention feature fusion, comprising:

[0008] Step S100, obtaining a data set and performing data preprocessing;

[0009] Step S200, constructing an enhanced small target lung nodule detection network model based on a deformable attention mechanism;

[0010] Step S300, training an enhanced small target lung nodule detection network model based on a deformable attention mechanism;

[0011] Step S400: input the image to be detected into the trained enhanced small target lung nodule detection network model based on deformable attention mechanism, and output the detection result.

[0012] Furthermore, the step S100 specifically includes: obtaining a lung nodule target detection data set, segmenting the lung area in the original image by corrosion and expansion, and then enhancing the data set by random rotation, center rotation, translation, cropping, and size scaling.

[0013] Specifically:

[0014] Acquire lung nodule CT images to form a data set; each lung nodule CT image includes multiple axial slices of the chest;

[0015] Label the CT images of lung nodules, including non-nodule areas, <a mm结节区和>=amm nodule area, a is the set nodule length; preferably, a is 3.

[0016] According to the Z-axis coordinate of the nodule location, obtain the 2D slice of a single nodule, re-annotate it using the annotation format of the data set, and then perform data format conversion;

[0017] The dataset was segmented into lung parenchyma by erosion and dilation methods, retaining only the entire lung area, obtaining image samples, and dividing them into training set, test set, and validation set;

[0018] The training set is subjected to Mosaic data enhancement, Mixup data enhancement, random cropping, flipping, and scaling. Finally, the K-means clustering algorithm is used to re-cluster the anchors to generate anchors of appropriate sizes.

[0019] Furthermore, the step S200 specifically includes:

[0020] Step S210: construct a deformable attention feature fusion module for targeted fusion of shallow features and deep features of the image;

[0021] Step S220: construct a small target detection layer for detecting the location and category information of the target.

[0022] Based on YOLOv7, the feature fusion method based on deformable idea and self-attention is improved, a small target detection layer is added, and an enhanced small target lung nodule detection network model is constructed. The model specifically includes: a deformable attention feature fusion module, which is used for the targeted fusion of shallow features and deep features of the image; a small target detection layer, which is used to supplement the effective and rich spatial information of tiny lung nodules.

[0023] Furthermore, the deformable attention feature fusion module includes a shallow feature sampling module and an attention calculation module, wherein:

[0024] The shallow feature sampling module is composed of the offset network, which samples the shallow features X∈R H×W×C For learning, H and W represent the height and width of the feature layer respectively, and C represents the number of feature channels. We get 4 groups of size The offset domain, 2 is the number of channels, which represents the offset variables of each sampling point in the X and Y directions. The shallow feature sampling points are obtained by bilinear interpolation.

[0025] The extraction of points of interest in the shallow feature layer is done with the help of the deformable idea. The offset network learns the input shallow features to obtain offset variables of multiple groups of sampling points, and then calculates the position of the sampling points in the shallow features by bilinear interpolation. Finally, the feature points of interest are extracted from the shallow features. In order to improve the diversity of deformable sampling points and provide different levels of semantic information for the attention mechanism, the present invention uses deep separable convolution to group feature channels, adjusts the number of sampling points by the average pooling downsampling layer, and finally adjusts the number of channels to generate offsets in the X and Y directions for each group. Taking the input shallow feature X∈R H×W×C (H, W represent the height and width of the feature layer, respectively, and C represents the number of feature channels) as an example, and the number of groups G is set to 4. The offset network finally generates 4 groups of size The offset domain, the number of channels is 2, which means the offset variables of each sampling point in the X and Y directions. Finally, the sampling point is obtained by bilinear interpolation.

[0026] Deep features are calculated by stacking deeper layers of the network. For example, four ELAN layers are used to obtain a feature map with a higher resolution on the top and a smaller resolution on the bottom. Since the feature extraction level is higher, the smaller size on the bottom is the deep feature.

[0027] The sampling points and deep features are put into the self-attention module for attention calculation. The attention module is designed with reference to the general multi-headed attention mechanism (MSA). The number of groups of sampling points corresponds to the number of heads of multi-headed attention, and X 1 ∈R H×W×C represents the deep features of the input, X 2 ∈R Hg×Wg×C Represents the shallow feature sampling points of the input, where H g , W g Represent the height and width of the feature layer respectively. Through mapping, the deep feature X 1 and the sampled features X 2 Respectively through the trainable transformation matrix W q , W k and W v , we get the Q, K, V needed to calculate the attention. The calculation process is as follows:

[0028] Q(X 1 )=X 1 W q ,K(X 2 )=X 2 W k ,V(X 2 )=X 2 W V

[0029] Where W q , W k and W v These are all weight matrices that need to be learned;

[0030] Q: Query: The query vector is used to obtain the correlation with other vectors. In the attention mechanism, Query represents the target we pay attention to or the vector that needs to be compared.

[0031] K: Key (K): The key vector is used to calculate the similarity between Query and Value. In the attention mechanism, Key is used to measure the degree of association between Query and other vectors.

[0032] V: Value (V): The value vector contains the information that needs to be weighted and aggregated according to the query. In the attention mechanism, Value represents the vector that we want to weight and aggregate according to the importance of the query Q.

[0033] According to the set number of groups, Q, K, and V are divided equally according to the number of groups to obtain each head 1 ,head 2 ,……,head h The corresponding Q i , K i , V i , h is the number of heads, and the attention between them is calculated. The formula for calculating attention is as follows:

[0034]

[0035] Among them, Q i , K i 、V i Corresponding to Query, Key and Value respectively, Q i With each K i Matching calculates the correlation between two vectors. The greater the correlation, the greater the corresponding attention weight. ki Represents vector K i length, and finally, V i The output of the attention mechanism is obtained by weighted summing with the corresponding attention weights, and then the weighted results of each group of attention are concatenated and then trained through the weight matrix W. z Fusion is performed to obtain the final multiple groups of attention results:

[0036] MultiHead(Q,K,V)=Concatenate(head 1 ,head 2 ,...,head h )W Z

[0037]

[0038] Among them, Concatenate(.) means that all groups are concatenated along the depth dimension.

[0039] Furthermore, the target detection head of the small target detection layer is composed of a convolutional layer and a linear layer. The input feature map is first processed and feature extracted through a convolutional layer, and then features are extracted at different scales by applying pooling operations at different scales to capture target information of different sizes. Then, a convolutional layer and two linear layers are used to output the predicted bounding box coordinates, category confidence, and category information corresponding to each feature point in the four feature layers.

[0040] Furthermore, the method further includes step S230: performing post-processing operations on the target detection results to obtain final target detection results.

[0041] Furthermore, the post-processing operation includes:

[0042] Non-maximum suppression removes duplicate detections;

[0043] Set the confidence threshold to remove boxes with lower confidence;

[0044] Then, each retained bounding box is assigned to the corresponding category according to the predicted category confidence to obtain the final target detection result.

[0045] Furthermore, in step S300, based on the data set obtained in step S100 and based on the enhanced small target detection network model constructed in step S200, training of the pulmonary nodule detection network model is performed, specifically including:

[0046] Step S310, the classification loss and bounding box regression loss of the enhanced small target lung nodule detection network model based on the deformable attention mechanism both use the binary cross entropy loss function and the Adam optimizer, and the hyperparameters are set as follows: the initial learning rate is 0.0001, the total number of iterations is 3300, the batch size is 8, and the L2 weight regularization is 0.0005; the best model is saved in each round of training; wherein, the bounding box regression loss function calculation formula is:

[0047]

[0048] Among them, c represents the minimum bounding rectangle of the predicted box and the real box, b and b gt Represent the centroids of the predicted box and the real box respectively, ρ represents the Euclidean distance between the two centroids, and W c , H c Indicates the width and height of the bounding rectangle c, l IOU Represents the IOU loss between the predicted box and the real box, w gt is the width of the real box, h gt is the height of the real box; the EIOU loss function calculates the length and width of the object box respectively, and continuously optimizes the model parameters by calculating the loss between the predicted result and the actual result through the back propagation algorithm;

[0049] Step S320: The lung nodule detection performance evaluation indexes include precision, recall and average precision (AP), where:

[0050]

[0051]

[0052]

[0053] Among them, TP represents the total number of true positives, i.e., true shadow pixels that are correctly identified; FN represents the total number of false negatives, i.e., true shadow pixels that are correctly identified; FP represents false positives, i.e., non-shadow pixels that are mistakenly identified as shadow pixels; TN represents the number of true negatives, i.e., the number of true non-shadow pixels that are correctly identified;

[0054] Step S330: The data obtained from the data set is input into the enhanced small target lung nodule detection network model based on the deformable attention mechanism, and then the back propagation algorithm is used to train the model.

[0055] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0056] (1) Aiming at the fact that the features of small targets are concentrated in shallow features, the present invention adds a detection layer with four times downsampling in the feature fusion pyramid to supplement the information such as the spatial position of small targets that is originally omitted in the detection layer, which helps to extract and enhance the feature representation of small targets and improve the detection accuracy of small targets.

[0057] (2) The present invention constructs a deformable attention feature fusion module, uses deformable convolution to improve the local problem of traditional convolutional feature learning, effectively extracts key features in shallow features, and reduces the redundant information introduced when fusion of feature layers, especially the large amount of irrelevant structures and background information in the shallow feature layer that is extremely critical for small target detection, reduces the interference of background redundant features, and enhances the feature fusion effect in an attention manner, promotes the effective fusion of shallow features and deep features, maximizes the use of learned features, and improves the precision and accuracy of small target detection.

[0058] (3) The present invention adds a small target detection layer to supplement the rich detail features of small targets. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 It is the overall network framework diagram of the present invention;

[0060] Figure 2 Schematic diagram of the variability attention module;

[0061] Figure 3 Schematic diagram of the small target detection layer;

[0062] Figure 4 This is a comparative analysis chart of different models in the experiment. DETAILED DESCRIPTION

[0063] The present invention is further described in detail below in conjunction with examples, but the embodiments of the present invention are not limited thereto.

[0064] Embodiment 1:

[0065] Combined with Figure 1 , Figure 2 and Figure 3 As shown, a small target detection method based on deformable attention feature fusion includes:

[0066] Step 100, data preprocessing stage:

[0067] Step 110, a dataset for lung nodule detection is obtained from the public LUNA16 (Aerial Imagery Shadow Detection) dataset, which includes 888 low-dose lung CT images (mhd format), each of which contains a series of multiple axial slices of the chest. The nodule area is annotated according to three label categories, including non-nodule area, nodule area <3mm, and nodule area >=3mm, and 36,378 annotation information is obtained, including 1,186 lung nodule areas, as experimental data for detection;

[0068] Step 120, first obtain the two-dimensional slice of a single nodule according to the Z-axis coordinate of the nodule location, re-annotate it in PASCAL VOC format, and then convert it into the dataset format required by YOLOv7. In order to optimize the detection effect, we segmented the lung parenchyma of the dataset by erosion and expansion methods, retaining only the entire lung area. Among the 1175 image samples obtained in total, we divided 70% of the lung nodule samples as training sets, 15% as test sets, and 15% as validation sets, and performed Mosaic data enhancement, Mixup data enhancement, random cropping, flipping, scaling and other processing on the training set. Finally, the K-means clustering algorithm was used to re-cluster the anchors to generate 12 anchors of appropriate size.

[0069] Step 200, constructing an enhanced small target lung nodule detection network model based on a deformable attention mechanism, includes:

[0070] Step 210: construct a feature extractor based on an efficient layer aggregation network

[0071] The basic module for feature learning, extraction and fusion in the YOLOv7 network structure is the efficient layer aggregation network module ELAN. ELAN is a network structure designed based on gradient path optimization. In the current neural network structure, the loss function guides the network to update parameters through back propagation. Therefore, the structure designed based on the gradient path has a more efficient and stable network learning ability by adjusting the gradient propagation path and gradient source, and can better learn image features.

[0072] Image feature extraction includes four stages. In each stage, each block outputs a feature map of the same size. In this embodiment, a maximum pooling layer with a span of 2 is used to downsample the feature map in each stage. The ELAN module is used to extract and fuse the feature maps after each layer of sampling. Before entering the next stage, the size of the feature map will be halved, but the number of channels will be doubled. The feature extractor outputs four layers of feature maps F1, F2, F3, and F4. The output f of each stage is respectively The feature map of size C k Indicates the number of channels of the feature map of the kth output, k = 1, 2, 3, 4. In this embodiment, the sizes of the feature maps output at each stage are [160×160×256], [80×80×512], [40×40×1024], and [20×20×1024], respectively, so that the entire network model can capture objects of multiple scales.

[0073] The deep features are calculated by stacking deeper layers of the network. The upper resolution is higher, 160*160, and the lowest resolution is 20*20. Because the feature extraction level is higher, the smaller size below is the deep feature.

[0074] Step 220: Feature pyramid fusion module based on deformable attention module

[0075] The feature pyramid network detects the target object at different scales by fusing multiple feature maps of different scales generated by the extractor. The processed feature layer contains four parts F1, F2, F3, and F4, respectively: [160×160×64], [80×80×128], [40×40×256], and [20×20×512].

[0076] Further deformable attention enhancement fusion is performed, in which the feature layer F1 with the largest size is the shallowest feature, containing rich small target detail features, and needs to be enhanced and fused with the slightly larger F2 feature layer. The deformable attention module is mainly divided into two parts, namely the feature selection part and the attention enhancement part. The input feature F1 is adjusted to 128 channels through a convolution layer with a convolution kernel size of 1×1, keeping the same number of channels as the F2 feature layer. F1 is learned through the offset network to obtain a feature map of size 10×10×2, representing the key position information of 10×10 and its offset in the X and Y directions. Then, the learned key features are obtained on F1 through bilinear interpolation, and the shallow feature f1 used to calculate the attention is obtained. The feature map size of f1 is 10×10×128. Finally, the self-attention is calculated for the feature maps of f1 and F2. F2 is used as q, v; f1 is used as k, the correlation between f1 and F2 is calculated, and the feature of F2 is enhanced based on the attention weight, and finally the weighted fusion feature of F1 and F2 is obtained. The weighted fusion feature sampling of F2 and F3 and F3 and F4 is obtained in the same way. The sizes of the multi-scale feature maps obtained by feature fusion are [160×160×128], [80×80×128], [40×40×256], and [20×20×512], respectively, which are input to the corresponding target detection head to detect the position and category information of the target.

[0077] Step 230, in the target detection stage, the detection head consists of a series of convolutional layers and linear layers, and the target detection results generated include three parts: based on the anchor boxes of pre-set different scales, the predicted bounding box coordinates, category confidence, and target score are obtained respectively. These three parts enable the model to accurately detect the target in the image. First, through a RepConv layer, the input feature map is further processed and feature extracted through a convolutional layer, and then the features are extracted at different scales by applying pooling operations at different scales to capture target information of different sizes. Then, through a convolutional layer and two linear layers, the predicted bounding box coordinates, category confidence, and category information corresponding to each feature point in the four feature layers are output. The detection results of the four feature layers are [20×20×21], [40×40×21], [80×80×21], and [160×160×21], which require further post-processing operations to remove invalid detection data.

[0078] Step 240, output the predicted target information, and perform post-processing operations on the four groups of detection results to obtain the final detection results. The post-processing operations include non-maximum suppression to remove duplicate detections; setting a confidence threshold to remove boxes with low confidence; and then assigning each retained bounding box to the corresponding category according to the predicted category confidence to obtain the final target detection result.

[0079] Step 300, training phase

[0080] Step 310, using the deep learning framework Pytorch1.7 to build a network architecture for enhanced small target detection. In the experiment, all model classification losses and bounding box regression losses use binary cross entropy loss function and Adam optimizer, and hyperparameters are set (the initial learning rate is 0.0001, the total number of iterations is 3300, the batch size is 8, and the L2 weight regularization is 0.0005). The best model is saved in each round of training; among them, the bounding box regression loss function calculation formula is:

[0081]

[0082] Among them, c represents the minimum bounding rectangle of the predicted box and the real box, b and b gt represents the centroid of the predicted box and the true box, ρ represents the Euclidean distance between the two centroids, and W c , H c Indicates the width and height of the bounding rectangle C. IOU Represents the IOU loss between the predicted box and the real box, w gt is the width of the real box, h gt is the height of the real box; the EIOU loss function calculates the length and width of the object box respectively, avoiding the problem of large calculation errors in the horizontal and vertical directions, improving the convergence speed and regression accuracy, and continuously optimizing the model parameters by calculating the loss between the predicted results and the actual results through the back propagation algorithm.

[0083] Step 320: Pulmonary nodule detection performance evaluation index

[0084] In order to intuitively and effectively analyze the detection accuracy of the proposed model, the target detection method was used. Currently, the commonly used indicators for evaluating model training performance and robustness are mainly precision, recall and average precision.

[0085] The formula for precision is as follows:

[0086]

[0087] The formula for recall is as follows:

[0088]

[0089] The formula for average precision AP is as follows:

[0090]

[0091] Among them, TP: True Positive, that is, the total number of true shadow pixels correctly identified. FN: False Negative, that is, the total number of true shadow pixels correctly identified. FP: False Positive, that is, non-shadow pixels that are mistakenly identified as shadow pixels; TN: True Negative, that is, the number of true non-shadow pixels correctly identified. Three object detection evaluation indicators, the higher the value, the better the detection result.

[0092] Step 330: The data obtained from the data set is input into the enhanced small target detection network model constructed in step 200, and then the model is trained using the back propagation algorithm.

[0093] Step 400, prediction phase

[0094] The data augmented dataset is input into the trained network model for lung nodule detection, and the test image is used for prediction, and the lung nodules in the image are detected to obtain the detection results.

[0095] The effect of the present invention can be further illustrated by the following simulation experiment.

[0096] 1. Simulation experiment content and result analysis:

[0097] The simulation experiment of the present invention adopts the method of the present invention and the ablation experiment method of the prior art, and the simulation experiment is carried out according to the following steps respectively;

[0098] The comparative experimental method uses Adam as the optimizer and binary cross entropy as the loss function to train a small target detection network model based on a deformable attention feature fusion module;

[0099] 3300 iterations of training were completed in the same experimental environment. After 300 iterations, the model reached convergence. The evaluation index results of six common target detection networks SSD, Faster Rcnn, YOLOv3, YOLOv5, Deformable-DETR, and YOLOv7 are compared as shown in Table 1. The data of various evaluation indicators show that this model has achieved good performance on the LUNA16 dataset, and has better results in lung nodule target detection. Compared with the lightweight model SSD, YOLOv3, and YOLOv5 models, the detection accuracy of the model is about 0.81. The two-stage Faster R-CNN model and the Deformable-detr model based on the Transformer model have a detection accuracy of about 0.84. Compared with the two-stage model, this model has higher detection efficiency and higher lung nodule detection accuracy. In the task of lung nodule target detection, the algorithm proposed in this paper has better results.

[0100] Table 1 Evaluation indicators of target detection for each model

[0101]

[0102] Figure 4 The lung nodule detection results of the network of the present invention and other models are compared to provide a more intuitive display of the detection effect of the present invention. Three nodules of different sizes are selected as detection objects. Figure 4 (a) is the original image of the lung nodule slice in the test set. Figure 4 (b) is the lung nodule detection result of the YOLOv5 network. Figure 4 (c) is the lung nodule detection result of the YOLOv7 network. Figure 4 Middle (d) is a diagram showing the lung nodule detection results of the network of the present invention.

[0103] The test results show that the present invention can effectively reduce the false detection and missed detection of tiny lung nodules ranging from 3mm to 10mm, and the detection effect is better than other algorithms. At the same time, it can accurately detect nodules of various sizes, and the confidence is better than other models.

[0104] 3. Ablation experiment:

[0105] In order to further analyze the role of the deformable attention module in the entire network, an ablation study was conducted. The present invention divides the experiment into three parts. The first is to use the deformable attention feature fusion module to complete the feature fusion of the small target detection layer and the next feature layer, which is called method a; the second is to use the deformable attention feature fusion module to complete the feature fusion of the middle layer and the deep layer, which is called method b; the third is to use the deformable attention feature fusion module to complete the feature fusion of the entire feature pyramid, which is called method c.

[0106] Table 2 Pulmonary nodule detection results of the ablation experimental method of the present invention in the simulation experiment

[0107] Model AP@0.5 / % Precision / % Recall / % YOLOv7 87.50 88.23 89.08 With a 91.46 90.63 91.20 With b 89.53 88.54 89.60 With c 93.50 93.79 92.00

[0108] In Table 2, various evaluation indicators show that the deformable attention fusion module in the present invention can improve the accuracy of lung nodule detection, and the enhancement effect of shallow feature fusion is better than that of middle and deep features, and the detection effect of small targets is better, the missed detection rate is lower, and the accuracy is higher.

[0109] Although the present invention is described herein with reference to the illustrative embodiments of the present invention, the above embodiments are only preferred embodiments of the present invention, and the embodiments of the present invention are not limited to the above embodiments. It should be understood that those skilled in the art can design many other modifications and embodiments, which will fall within the scope and spirit of the principles disclosed in this application.

Claims

1. A small target detection method based on deformable attention feature fusion, characterized in that: include: Step S100, obtaining a data set and performing data preprocessing; Step S200, constructing an enhanced small target lung nodule detection network model based on a deformable attention mechanism; Step S300, training an enhanced small target lung nodule detection network model based on a deformable attention mechanism; Step S400, input the image to be detected into the trained enhanced small target lung nodule detection network model based on deformable attention mechanism, and output the detection result; The step S200 specifically includes: Step S210: construct a deformable attention feature fusion module for targeted fusion of shallow features and deep features of the image; Step S220: construct a small target detection layer for detecting the position and category information of the target; The deformable attention feature fusion module includes a shallow feature sampling module and an attention calculation module, wherein: The shallow feature sampling module is composed of the offset network, which samples the shallow features X∈R H×W×C For learning, H and W represent the height and width of the feature layer respectively, and C represents the number of feature channels. We get 4 groups of size The offset domain, 2 is the number of channels, which represents the offset variables of each sampling point in the X and Y directions. The shallow feature sampling points are obtained by bilinear interpolation. The sampling points and deep features are put into the self-attention module for attention calculation. The number of groups of sampling points corresponds to the number of heads of multi-head attention, and X1∈R H×W×C represents the deep features of the input, Represents the shallow feature sampling points of the input, where H g , W g Respectively represent the height and width of the feature layer. Through mapping, the deep feature X1 and the sampled feature X2 are respectively transformed through the trainable transformation matrix W q , W k and W v , we get the Q, K, and V needed to calculate the attention. The calculation process is as follows: Q(X1)=X1W q ,K(X2)=X2W k ,V(X2)=X2W V Where W q , W k and W v These are all weight matrices that need to be learned; According to the set number of groups, Q, K, and V are divided equally according to the number of groups to obtain each head head1, head2, ..., head h The corresponding Q i , K i , V i , h is the number of heads, and the attention between them is calculated. The formula for calculating attention is as follows: Among them, Q i , K i 、V i Corresponding to Query, Key, Value, Q i With each K i Matching calculates the correlation between two vectors. The greater the correlation, the greater the corresponding attention weight. ki Represents vector K i length, and finally, V i The output of the attention mechanism is obtained by weighted summing with the corresponding attention weights, and then the weighted results of each group of attention are concatenated and then trained through the weight matrix W. z Fusion is performed to obtain the final multiple groups of attention results: MultiHead(Q,K,V)=Concatenate(head1,head2,...,head h )W Z where head i =Attention(QW i Q ,KW i K ,VW i V ) Among them, Concatenate(.) means that all groups are concatenated along the depth dimension.

2. The small target detection method based on deformable attention feature fusion according to claim 1 is characterized in that: The step S100 specifically includes: A lung nodule target detection dataset is obtained, and the lung area in the original image is segmented by erosion and dilation. The dataset is then enhanced by random rotation, center rotation, translation, cropping, and resizing.

3. The small target detection method based on deformable attention feature fusion according to claim 1 is characterized in that: The target detection head of the small target detection layer consists of a convolutional layer and a linear layer. The input feature map is first processed and feature extracted through a convolutional layer. Then, feature extraction is achieved at different scales by applying pooling operations at different scales to capture target information of different sizes. Then, a convolutional layer and two linear layers are used to output the predicted bounding box coordinates, category confidence, and category information corresponding to each feature point in the four feature layers.

4. The small target detection method based on deformable attention feature fusion according to claim 3 is characterized in that: The method further includes step S230: performing post-processing operations on the target detection results to obtain final target detection results.

5. The small target detection method based on deformable attention feature fusion according to claim 4 is characterized in that: The post-processing operations include: Non-maximum suppression removes duplicate detections; Set the confidence threshold to remove boxes with lower confidence; Then, each retained bounding box is assigned to the corresponding category according to the predicted category confidence to obtain the final target detection result.

6. The small target detection method based on deformable attention feature fusion according to claim 1 is characterized in that: The step S300 specifically includes: Step S310, the classification loss and bounding box regression loss of the enhanced small target lung nodule detection network model based on the deformable attention mechanism both use the binary cross entropy loss function and the Adam optimizer, and the hyperparameters are set as follows: the initial learning rate is 0.0001, the total number of iterations is 3300, the batch size is 8, and the L2 weight regularization is 0.0005; the best model is saved in each round of training; wherein, the bounding box regression loss function calculation formula is: Among them, c represents the minimum bounding rectangle of the predicted box and the real box, b and b gt Represent the centroids of the predicted box and the real box respectively, ρ represents the Euclidean distance between the two centroids, and W c , H c Indicates the width and height of the bounding rectangle c, l IOU Represents the IOU loss between the predicted box and the real box, w gt is the width of the real box, h gt is the height of the real box; the EIOU loss function calculates the length and width of the object box respectively, and continuously optimizes the model parameters by calculating the loss between the predicted result and the actual result through the back propagation algorithm; Step S320: The lung nodule detection performance evaluation indexes include precision, recall, and average precision (AP), where: Among them, TP represents the total number of true positives, i.e., true shadow pixels that are correctly identified; FN represents the total number of false negatives, i.e., true shadow pixels that are correctly identified; FP represents false positives, i.e., non-shadow pixels that are mistakenly identified as shadow pixels; TN represents the number of true negatives, i.e., the number of true non-shadow pixels correctly identified; Step S330: The data obtained from the data set is input into the enhanced small target lung nodule detection network model based on the deformable attention mechanism, and then the back propagation algorithm is used to train the model.

Images