A target detection method for nuclear medicine imaging equipment based on improved DETR model

Through the improved DETR model, combined with dynamic geometric position coding and Transformer structure, the problem of object detection accuracy and training time in nuclear medical imaging equipment is solved, and efficient object recognition and detection is achieved.

CN118486029BActive Publication Date: 2024-11-15JIANGSU INST OF METROLOGY +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410947240.6
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-07-16
Publication Date
2024-11-15
Estimated Expiration
2044-07-16

AI Technical Summary

Technical Problem

The existing object detection methods are not widely used in nuclear medical imaging equipment, especially traditional methods based on convolutional neural networks have limitations of anchor points and non-maximum suppression, making it difficult to effectively identify and locate nuclear medical imaging equipment.

Method used

Adopting an improved DETR model, including feature extraction network, query generation module, encoder module, decoder module, multi-scale global and local attention mechanism, and classification and bounding box regression module, it is generated through dynamic geometric position coding and single query vectors. Combining the ResNet-152 network and Transformer structure, an NMED-DETR model is built for object detection.

Benefits of technology

It improves the accuracy and robustness of object detection in nuclear medical imaging equipment, reduces training time, and significantly outperforms the traditional DETR model when identifying large targets.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118486029B_ABST
    Figure CN118486029B_ABST
Patent Text Reader

Abstract

The present invention discloses a target detection method for nuclear medical imaging equipment based on an improved DETR model, comprising the following steps: 1. Collecting images containing nuclear medical imaging equipment, annotating the nuclear medical imaging equipment in the image, constructing a data set, the data set including images and annotation information, and dividing the data set into a training set and a verification set; 2. Constructing a NMED-DETR model, and training the NMED-DETR model using the training set obtained in step one; 3. Inputting the image of the nuclear medical imaging equipment to be detected into the trained NMED-DETR model obtained in step two, and obtaining the detection result of the nuclear medical imaging equipment. The target detection method for nuclear medical imaging equipment based on the improved DETR model of the present invention adopts the NMED-DETR model, which not only improves the detection accuracy, but also reduces the training time. When identifying large targets of nuclear medical imaging equipment, it is significantly improved compared with the traditional DETR model.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention belongs to the field of visual perception of mobile working robots and relates to a real-time target detection method based on an improved DETR model. Background Art

[0002] As a cutting-edge diagnostic and treatment method in the medical field, nuclear medicine imaging technology plays an important role in the early screening of diseases such as tumors. Its accuracy and stability are crucial to ensuring medical quality and radiation safety. Usually, when inspectors perform quality control work, the radiation dose they receive is about 240 times the normal level. Excessive radiation exposure increases the risk of cancer.

[0003] When developing intelligent detection robots for nuclear medicine imaging equipment, it is crucial to accurately identify and locate objects in the environment. Designing a real-time intelligent target detection method has become an urgent challenge to be solved.

[0004] Traditional target detection methods are all based on convolutional neural networks, such as Faster R-CNN, SSD, and YOLO. These methods often use pre-defined anchor points to generate candidate boxes and use non-maximum suppression to filter the final detection results. The DETR (Detection Transformer) model is a target detection method based on the Transformer structure. Unlike most previous target detection methods, DETR abandons many traditional components in its design, such as anchor points and non-maximum suppression, providing new opportunities and methods for robot visual perception.

[0005] Although the DETR model has the potential to play a role in more target detection problems, there is currently no public and reliable solution for applying the DETR model to target detection in nuclear medicine imaging equipment.

[0006] Therefore, a nuclear medicine imaging equipment target detection method based on an improved DETR model is needed to solve the above-mentioned problems. Summary of the invention

[0007] The purpose of the present invention is to provide a target detection method for nuclear medicine imaging equipment based on an improved DETR model to solve the problems raised in the above background technology.

[0008] A target detection method for nuclear medicine imaging equipment based on an improved DETR model comprises the following steps:

[0009] 1. Collect images containing nuclear medicine imaging devices, annotate the nuclear medicine imaging devices in the images, construct a data set, the data set includes images and annotation information, and divide the data set into a training set and a validation set;

[0010] 2. Constructing the NMED-DETR model, using the training set obtained in step 1 to train the NMED-DETR model,

[0011] The NMED-DETR model includes a feature extraction network, a query generation module, an encoder module, a decoder module, a multi-scale global and local attention mechanism, and a classification and bounding box regression module, wherein the feature extraction network, the query generation module, the encoder module, the decoder module, the multi-scale global and local attention layer, and the classification and bounding box regression module are connected in sequence;

[0012] The feature extraction network includes a convolutional neural network and a dynamic geometric position encoding module.

[0013] The convolutional neural network is used to process the input image of the target to extract a multi-scale feature map;

[0014] The dynamic geometric position encoding module is used to add dynamic geometric position encoding to the feature map to obtain a dynamic geometric position encoding feature map;

[0015] The query generation module is used to generate a target query vector according to the dynamic geometric position encoding feature map;

[0016] The encoder module includes 6 layers of encoders, each layer of encoders includes a first multi-head self-attention layer and a first feedforward neural network, which is used to encode the input dynamic geometric position encoding feature map to obtain encoding features;

[0017] The decoder module includes 6 layers of decoders, each layer of the decoder includes a second multi-head self-attention layer, a third multi-head self-attention layer and a second feedforward neural network, which is used to decode the target query vector and the encoded feature to obtain a decoded feature;

[0018] The multi-scale global and local attention layers are used to perform multi-scale feature fusion on the encoding features and the decoding features to obtain fused features;

[0019] The classification and bounding box regression module is used to output the category and bounding box coordinates of the target according to the fusion features;

[0020] 3. Input the image of the nuclear medicine imaging device to be detected into the trained NMED-DETR model obtained in step 2 to obtain the detection result of the nuclear medicine imaging device.

[0021] Furthermore, the nuclear medicine imaging equipment in step 1 includes a positron emission tomography (PET), a single photon emission computed tomography (SPECT) and a PET / CT scanner.

[0022] Furthermore, the convolutional neural network in step 2 extracts multi-scale feature maps using the following formula:

[0023]

[0024] Where I is the input image, Represents the ResNet-152 network operation, F is the extracted feature map, and the dimension is (H, W, C), where H and W are the height and width of the feature map respectively, and C is the number of channels.

[0025] ResNet-152 is selected as the backbone network to extract feature maps F from the input image I. The extracted feature maps contain rich spatial and semantic information, providing high-quality input for subsequent query generation and Transformer structure parts.

[0026] Furthermore, the dynamic geometric position encoding module in step 2 adds dynamic geometric position encoding to the feature map using the following formula:

[0027] ,

[0028] in: are the pixel coordinates in the input image, For even indexes, For odd indexes, is the dimension of the position encoding, It is an adjustment coefficient used to dynamically adjust the position encoding according to image features.

[0029] Add position coding to the feature map to enhance the spatial geometric information of the feature map. Based on the position coding method of sine and cosine functions, the feature position of the image is encoded, thereby retaining the spatial geometric information of the image. The dimension d of the position coding in the present invention is 256, and the adjustment coefficient and is 0.5.

[0030] Furthermore, the query generation module in step 2 generates a single query vector using the following formula: :

[0031] ,

[0032] In the formula, is the regularized query vector, is the L2 norm of Q, Q is the query vector, and the dimension of the query vector is D. is a fully connected layer operation, G is a global feature vector, is the global average pooling operation, F is the extracted feature map, and the dimension of the global feature vector is C.

[0033] A query vector is generated based on the input features to capture the features of the target object. By generating a single query vector, the query calculation process is simplified and the computational complexity is reduced. The use of global average pooling and fully connected layers enables the generated query vector to capture the global information of the input image and improve the expressiveness of the features. Regularization ensures the scale and stability of the query vector, which helps with the subsequent processing of the Transformer structure.

[0034] Furthermore, the multi-scale global and local attention layers described in step 2 use the following formula to perform multi-scale feature fusion:

[0035] ,

[0036] In the formula, S is a multi-scale set, is the weight of scale S, , , are the query, key and value matrices of scale S; Q is the query matrix, K is the key matrix, and V is the value matrix. is the dimension of the key matrix; is the i-th position of the query matrix, is the jth position of the key matrix, is the j-th position of the value matrix.

[0037] Combine global attention and local attention for feature fusion to improve the accuracy and robustness of target detection. The input is the decoded feature Y and the encoded feature Z, and the output is the fused feature , for classification and bounding box regression.

[0038] Furthermore, the encoder also includes a first residual connection, a first layer normalization, a second residual connection and a second layer normalization, the first residual connection and the first layer normalization are arranged between the first multi-head self-attention layer and the first feedforward neural network, and the second residual connection and the second layer normalization are arranged after the first feedforward neural network.

[0039] Furthermore, the decoder also includes a third residual connection, a third layer normalization, a fourth residual connection, a fourth layer normalization, a fifth residual connection and a fifth layer normalization, the third layer normalization and the third residual connection are arranged between the second multi-head self-attention mechanism and the third multi-head self-attention layer, the fourth residual connection and the fourth layer normalization are arranged between the third multi-head self-attention layer and the second feedforward neural network, and the fifth residual connection and the fifth layer normalization are arranged after the second feedforward neural network.

[0040] The feature extraction network described in step 2 uses the following formula to perform batch normalization calculation on the extracted features:

[0041] ,

[0042] in: is the adjusted value of the ith sample in feature channel c, is the learnable scaling factor of feature channel c, is the learnable offset of feature channel c, is the value of the i-th sample in the feature channel c after standardization, is a constant, is the batch variance of feature channel c, is the batch mean of feature channel c, m is the batch size, that is, the number of samples in a small batch, is the value of the i-th sample in feature channel c.

[0043] Furthermore, the loss function of the NMED-DETR model in step 2 is expressed as follows:

[0044] ,

[0045] In the formula, represents the weight of classification loss, represents the weight of the bounding box regression loss, N is the number of samples, C is the number of categories, is the true category label of the i-th sample, is the predicted category probability of the i-th sample, is the predicted bounding box of the ith sample, is the true bounding box of the i-th sample.

[0046] When choosing the weights of classification loss and bounding box regression loss, we should not only look at the total loss, but also consider the balance between classification loss and bounding box regression loss. A combination with a low total loss value may not be the best choice because it may ignore the optimization of a certain part of the loss. Choosing the weights of classification loss and bounding box regression loss is a process that requires experimentation and empirical adjustment.

[0047] Beneficial effect: The nuclear medicine imaging equipment target detection method based on the improved DETR model of the present invention adopts the NMED-DETR model, which not only improves the detection accuracy, but also reduces the training time. When identifying large targets of nuclear medicine imaging equipment, it is significantly improved compared with the traditional DETR model. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 It is a flowchart of the target detection method of nuclear medicine imaging equipment based on the improved DETR model;

[0049] Figure 2It is a schematic diagram of the structure of the NMED-DETR model of the present invention;

[0050] Figure 3 Schematic diagram of the process of the NMED-DETR model of the present invention;

[0051] Figure 4 It is a structural schematic diagram of the feature extraction network of the present invention;

[0052] Figure 5 The nuclear medicine imaging equipment to be tested;

[0053] Figure 6 Nuclear medicine imaging equipment to complete target detection. DETAILED DESCRIPTION

[0054] The present invention is further explained below in conjunction with specific embodiments. It should be understood that these embodiments are only used to illustrate the present invention and are not used to limit the scope of the present invention. After reading the present invention, various equivalent forms of modifications to the present invention by those skilled in the art all fall within the scope defined by the claims attached to this application.

[0055] See also Figure 1 and Figure 2 As shown, the target detection method of nuclear medicine imaging equipment based on the improved DETR model of the present invention comprises the following steps:

[0056] 1. Obtain images containing nuclear medicine imaging equipment as data sets through public data sets, cooperation with medical institutions, self-collection of data, etc. The nuclear medicine imaging equipment includes but is not limited to positron emission tomography PET, single photon emission computed tomography SPECT, and PET / CT scanners;

[0057] 2. Before using the dataset of nuclear medicine imaging equipment for video analysis, data preprocessing is an important step to improve data quality and enhance the training effect and generalization ability of the model.

[0058] a) Data cleaning: delete or correct missing data and damaged image files, unify data formats and naming conventions, and remove duplicate images or video frames;

[0059] b) Data normalization: scaling the data to a standard range to reduce the impact of different data scales;

[0060] c) Data enhancement: Generate variant data by rotation, translation, noise addition, color jitter, etc. to increase the amount of data and improve the generalization ability of the model;

[0061] d) Data cropping and resampling: Through cropping and resampling, the image size and frame rate are unified to ensure the consistency of input data;

[0062] e) Data labeling: Use the labeling tool LabelImg to manually label the location and category of the target object and generate a labeling file corresponding to the image or video;

[0063] f) Image enhancement and filtering: Enhance image quality and highlight target features through image processing techniques such as denoising, sharpening, and histogram equalization;

[0064] g) Data segmentation: randomly divide the data set according to the proportion, such as 70% for training, 20% for validation, and 10% for testing.

[0065] 3. Build the NMED-DETR model and use the training set obtained in step 1 to train the NMED-DETR model:

[0066] See also Figure 2 As shown in the figure, the structure of the NMED-DETR model (Nuclear Medicine Equipment Detection using Detection Transformers) method includes the following key parts:

[0067] 1. Feature extraction part

[0068] a) Backbone network

[0069] ResNet-152 is selected as the backbone network to extract feature maps F from the input image I. The extracted feature maps contain rich spatial and semantic information, providing high-quality input for subsequent query generation and Transformer structure parts.

[0070] ,

[0071] Where I is the input image, Represents the ResNet-152 network operation, F is the extracted feature map, and the dimension is (H, W, C), where H and W are the height and width of the feature map respectively, and C is the number of channels.

[0072] b) Dynamic Geometric Encoding (DGE)

[0073] Add position encoding to the feature map to enhance the spatial geometric information of the feature map. The position encoding method based on sine and cosine functions encodes the feature position of the image, thereby retaining the spatial geometric information of the image. The dimension of position encoding is 256, adjust the coefficient and is 0.5.

[0074] ,

[0075] in: are the pixel coordinates in the input image, For even indexes, For odd indexes, is the dimension of the position encoding, It is an adjustment coefficient used to dynamically adjust the position encoding according to image features.

[0076] See also Figure 3 and Figure 4 As shown, for the calculation of feature extraction, the present invention adopts the scaling factor of batch normalization, and measures the variance of the channel through this factor to determine its importance.

[0077] The batch normalization process involves calculating the mean and variance of each feature channel, normalizing the input data, and adjusting it using learnable scaling factors and offsets. The channel attention module enhances important features and suppresses unimportant features by assigning different weights to each channel.

[0078] The following is the detailed formula and the specific meaning of each parameter:

[0079] 1. Batch mean Calculated by the following formula:

[0080] ,

[0081] in: is the batch mean of feature channel c, m is the batch size, that is, the number of samples in a small batch, is the value of the i-th sample in feature channel c;

[0082] 2. Batch variance Calculated by the following formula:

[0083] ,

[0084] in: is the batch variance of feature channel c, m is the batch size, that is, the number of samples in a small batch, is the value of the i-th sample in feature channel c, is the batch mean of feature channel c;

[0085] 3. Standardization Calculated by the following formula:

[0086] ,

[0087] in: is the value of the i-th sample in the feature channel c after standardization, is the value of the i-th sample in feature channel c, is the batch mean of feature channel c, is the batch variance of feature channel c, is a constant used to prevent division by zero. The empirical value 10 is used in this invention. -5 ;

[0088] 4. The scale factor and offset adjustment are expressed by the following formula:

[0089] ,

[0090] in: is the adjusted value of the ith sample in feature channel c, is the learnable scaling factor of feature channel c, is the value of the i-th sample in the feature channel c after standardization, is the learnable offset of feature channel c.

[0091] Input data: The input data of the batch normalization layer can be the output feature map of the convolutional layer or the output vector of the fully connected layer.

[0092] Output data: batch normalized and resized data with the same shape as the input data.

[0093] 5. Channel Attention Module

[0094] The channel attention module performs weighted processing on the normalized and activated feature maps, assigning different weights to each channel to enhance important features and suppress unimportant features. Figure 4 As shown, the formula is:

[0095] ,

[0096] in, and are the scaling factors for the ith and jth channels respectively, and is the weight.

[0097] 2. Query generation part

[0098] Single query generation: Generate a query vector based on the input features to capture the features of the target object. By generating a single query vector, the query calculation process is simplified and the computational complexity is reduced. The use of global average pooling and fully connected layers enables the generated query vector to capture the global information of the input image and improve the expressiveness of the features. Regularization ensures the scale and stability of the query vector, which helps with the subsequent processing of the Transformer structure.

[0099] a) Input feature preparation

[0100] The feature maps F of the input image are obtained from the feature extraction part. These feature maps contain rich spatial and semantic information of the input image.

[0101] ,

[0102] Where: I is the input image, F is the extracted feature map, the dimension is (H, W, C), where H and W are the height and width of the feature map respectively, and C is the number of channels.

[0103] b) Global average pooling

[0104] Perform global average pooling on the feature map F to obtain a global feature vector G. Global average pooling can summarize the global information in the feature map and generate a feature vector of a fixed size.

[0105] ,

[0106] Where: G is the global feature vector with dimension C;

[0107] c) Fully connected layer:

[0108] The global feature vector G is input into a fully connected layer to obtain a high-dimensional feature vector Q. The fully connected layer is used to further process and transform the features to meet the needs of query generation.

[0109] ,

[0110] Where Q is the query vector, dimension is D, and D is the dimension of the query vector;

[0111] d) Regularization of query vector

[0112] The generated query vector Q is regularized to ensure that the query vector has a suitable scale and stability.

[0113] ,

[0114] in, is the regularized query vector, is the L2 norm of Q.

[0115] e) Output of query vector

[0116] The normalized query vector As the final single query vector , which is used in the subsequent Transformer structure to capture and process target features.

[0117] 3. Transformer structure

[0118] The Transformer structure of the NMED-DETR model includes an encoder part, a decoder part, a multi-scale global and local attention mechanism, and a classification and bounding box regression part.

[0119] a) Encoder part: Encode the input features and generate high-dimensional feature representation to capture the global information in the image. The input is the feature map F from the feature extraction part, and the encoded features Z will be used in the decoder part.

[0120] Specifically, the encoder structure has 6 layers, each layer contains a multi-head self-attention mechanism and a feedforward neural network. Each sub-layer contains residual connections and layer normalization.

[0121] Implementation steps:

[0122] 1) Input feature map F calculates self-attention through multi-head self-attention mechanism;

[0123] 2) Perform nonlinear transformation through a feedforward neural network;

[0124] 3) Layer normalization and residual connections ensure stable gradient flow and model performance;

[0125] b) Decoder: decodes the encoded features and generates target detection results, combining the global and local information of the input image. The input is the target query vector And the encoded feature Z, the output feature Y, is used for multi-scale global and local attention mechanisms.

[0126] Specifically, the decoder structure has 6 layers, each of which contains a multi-head self-attention mechanism, an encoder-decoder attention mechanism, and a feedforward neural network. Similarly, each sub-layer contains residual connections and layer normalization. Among them, the multi-head self-attention mechanism is used inside the encoder and decoder, mainly processing the internal relationship of the same sequence. The encoder-decoder attention mechanism is used in the decoder to process the relationship between the decoder input and the encoder output.

[0127] The query, key, and value of the multi-head self-attention mechanism all come from the same input sequence. The query of the encoder-decoder attention mechanism comes from the decoder input, and the key and value come from the encoder output.

[0128] Implementation steps:

[0129] 1) Target query vector Self-attention is calculated through a multi-head self-attention mechanism;

[0130] 2) Combine the query vector and the encoded feature Z, and calculate the mutual attention through the encoder-decoder attention mechanism;

[0131] 3) Perform nonlinear transformation through feedforward neural network;

[0132] 4) Layer normalization and residual connections ensure stable gradient flow and model performance.

[0133] In the encoder and decoder structures of the NMED-DETR model, each sublayer contains layer normalization and residual connections, with the order of residual connection first and layer normalization next. These two work together to stabilize the training process, prevent gradient disappearance, accelerate model convergence, and retain the original input information. These mechanisms enable deep neural networks to be trained and inferred more efficiently, thereby improving the performance and reliability of the model.

[0134] The role of residual connection: In order to solve the problem of difficult training of multi-layer neural networks, by transferring part of the information of the previous layer to the next layer without difference, the model performance can be effectively improved. Because for some layers, it is not certain whether their effect is positive. After adding residual connection, it is equivalent to dividing the information of the previous layer into two paths, one part is transformed through the current layer, and the other part is directly passed to the next layer, and then the results of these two parts are added as the output of the next layer. After residual connection, the information of the previous layer can be retained, specifically:

[0135] 1) Preventing gradient disappearance: In deep networks, the gradient disappearance problem will hinder the training of the network. Residual connections help the gradient propagate back to the previous layer more easily by adding a short-circuit path when skipping one or more layers, alleviating the gradient disappearance problem.

[0136] 2) Preserving information: Residual connections allow the network to skip some nonlinear transformation layers and pass the input directly to the subsequent layers, which helps to preserve the original input information.

[0137] The role of layer normalization: By normalizing the activation values ​​of the layer, the training process of the model can be accelerated and converged faster. Specifically:

[0138] 1) Stabilize the training process: By standardizing the output of each layer, layer normalization can reduce internal covariate shift and thus stabilize the training process.

[0139] 2) Accelerate convergence: Layer normalization makes the input of each layer have a distribution with mean 0 and variance 1, which helps the gradient propagate more stably and efficiently, thereby accelerating the convergence of the model.

[0140] c) Multi-scale global and local attention mechanism (MSGA):

[0141] Combine global attention and local attention for feature fusion to improve the accuracy and robustness of target detection. The input is the decoded feature Y and the encoded feature Z, and the output is the fused feature , for classification and bounding box regression.

[0142] Implementation steps:

[0143] 1) Calculate multi-scale global attention to capture global features;

[0144] 2) Calculate multi-scale local attention to capture local features;

[0145] 3) Perform weighted fusion of multi-scale global and local attention outputs to generate fused features.

[0146] d) Classification and bounding box regression part: output the category and bounding box coordinates of the target. The input is the fused feature , the output is the target category and bounding box coordinates .

[0147] Implementation steps:

[0148] 1) For the fused features Perform classification and output the target category;

[0149] 2) The fused features Perform bounding box regression and output the bounding box coordinates of the target.

[0150] About loss function: In the NMED-DETR model, the loss function is mainly used to evaluate the difference between the model's prediction and the true label and guide the optimization of model parameters. The loss function of NMED-DETR consists of two main parts: classification loss and bounding box regression loss. These two parts are usually combined by weighted summation to obtain the total loss function.

[0151] 1. Classification loss: used to measure the difference between the category predicted by the model and the true category, calculated using cross entropy loss.

[0152] ,

[0153] Where: N: number of samples, C: number of categories, : The true category label of the i-th sample, : The predicted category probability of the i-th sample;

[0154] 2. Bounding box regression loss: used to measure the difference between the bounding box predicted by the model and the true bounding box, calculated using L1 loss.

[0155] ,

[0156] in: : The predicted bounding box of the i-th sample, : The true bounding box of the i-th sample;

[0157] 3. Total loss function: It is the weighted sum of classification loss and bounding box regression loss.

[0158] ,

[0159] in: : The weight of the classification loss, : Weight of the bounding box regression loss.

[0160] When choosing the weights of the classification loss and bounding box regression loss, we should not only look at the total loss, but also consider the balance between the classification loss and the bounding box regression loss. A combination with a low total loss value may not be the best choice because it may ignore the optimization of a certain part of the loss. Choosing the weights of the classification loss and the bounding box regression loss is a process that requires experimentation and empirical adjustment.

[0161] Verification results:

[0162] Table 1. Comparison of accuracy, speed and IoU between NMED-DETR and traditional DETR

[0163]

[0164] Table 1 Data analysis:

[0165] The impact of dynamic geometric position coding: before and after the introduction of dynamic geometric position coding, mAP increased from 0.80 to 0.82; IoU increased from 0.75 to 0.76. This shows that after the introduction of dynamic geometric position coding, the model can better capture the spatial geometric information in the image and improve the feature representation ability.

[0166] Impact of single query generation: After testing, the FPS increased by 15 -> 17 and the mAP increased by 0.82 -> 0.83 before and after the introduction of single query generation. This shows that the use of a single query vector simplifies the calculation process, reduces the calculation complexity, and improves the detection speed.

[0167] The impact of multi-scale global and local attention mechanisms: After testing, mAP increased by 0.83 -> 0.85 and IoU increased by 0.76 -> 0.78 before and after the introduction of multi-scale global and local attention mechanisms. The manual combines multi-scale features to improve the detection ability of objects of different scales and enhance the feature fusion effect.

[0168] Table 2. Average precision AP of NMED-DETR and traditional DETR for large targets l , AP50 and AP75 comparison table

[0169]

[0170] Table 2 Data analysis:

[0171] The experimental data show that as the number of GPUs increases, the performance of the two models also improves. When the number of GPUs increases from 1 to 4, the average precision AP of the large objects of the two models increases. l , AP50, and AP75 have all been improved. Specifically, NMED-DETR has improved by about 6.95%~8.01% in terms of large target average precision AP1; about 5.31%~6.51% in terms of AP50; and about 6.97%~7.53% in terms of AP75. The training time of the NMED-DETR model has been reduced under all GPU number configurations. For example, when using 1 GPU, the training time is reduced from 181.2 hours to 160.3 hours, a reduction of 11.53%; when using 2 GPUs, the training time is reduced by 12.65%; and when using 4 GPUs, the training time is reduced by 8.38%. This shows that as the number of GPUs increases, the training time is significantly reduced, which improves the training efficiency. The number of parameters of the NMED-DETR model has increased slightly, from 41M to 43M, an increase of 4.87%. Despite the increase in the number of parameters, the improvement in performance shows that these additional parameters are effective.

[0172] As the number of GPUs increases, the average precision AP of large targets of the two models l , AP50, and AP75 have all been improved. This is because increasing the number of GPUs can enhance the generalization performance of the parameters, allowing the model to perform better when processing complex tasks. The training time of the NMED-DETR model has been reduced under all GPU configurations, showing higher training efficiency. Especially in multi-GPU configurations, NMED-DETR can complete training tasks faster, improving overall efficiency. Although the number of parameters of NMED-DETR has increased, this increase is reasonable and effective relative to the significant improvement in performance. The additional parameters are mainly used to enhance the model's feature extraction and attention mechanism, making it perform better when identifying large targets.

[0173] See also Figure 5 and Figure 6 As shown, the nuclear medicine imaging equipment target detection method based on the improved DETR model of the present invention can quickly and accurately identify nuclear medicine imaging equipment.

[0174] In summary, the NMED-DETR model of the present invention has significantly improved the traditional DETR model in identifying large targets of nuclear medicine imaging equipment through multiple structural optimizations and parameter adjustments. The NMED-DETR model not only improves the detection accuracy, but also reduces the training time, proving its effectiveness and superiority in practical applications.

Claims

1. A target detection method for nuclear medicine imaging equipment based on an improved DETR model. It is characterized in that The following steps are involved: Step 1: Acquire images containing nuclear medicine imaging devices, annotate the nuclear medicine imaging devices in the images, construct a data set, the data set includes images and annotation information, and divide the data set into a training set and a validation set; Step 2: construct a NMED-DETR model, and train the NMED-DETR model using the training set obtained in step 1. The NMED-DETR model includes a feature extraction network, a query generation module, an encoder module, a decoder module, a multi-scale global and local attention mechanism, and a classification and bounding box regression module, wherein the feature extraction network, the encoder module, the decoder module, the multi-scale global and local attention layer, and the classification and bounding box regression module are connected in sequence; The feature extraction network includes a convolutional neural network and a dynamic geometric position encoding module. The convolutional neural network is used to process the input image of the target to extract a multi-scale feature map; The dynamic geometric position encoding module is used to add dynamic geometric position encoding to the feature map to obtain a dynamic geometric position encoding feature map; The query generation module is used to generate a target query vector according to the feature map; the query generation module and the dynamic geometric position encoding module are added and connected to the decoder module; The query generation module in step 2 generates the target query vector using the following formula: : , In the formula, is the regularized query vector, is the L2 norm of Q, Q is the query vector, and the dimension of the query vector is D. is a fully connected layer operation, G is a global feature vector, is the global average pooling operation, F is the extracted feature map, and the dimension of the global feature vector is C; The encoder module includes 6 layers of encoders, each layer of encoders includes a first multi-head self-attention layer and a first feedforward neural network, which is used to encode the input dynamic geometric position encoding feature map to obtain encoding features; The decoder module includes 6 layers of decoders, each layer of the decoder includes a second multi-head self-attention layer, a third multi-head self-attention layer and a second feedforward neural network, which is used to decode the target query vector and the encoded feature to obtain a decoded feature; The multi-scale global and local attention layers are used to perform multi-scale feature fusion on the encoding features and the decoding features to obtain fused features; The multi-scale global and local attention layers described in step 2 use the following formula to perform multi-scale feature fusion: , In the formula, S is a multi-scale set, is the weight of scale S, , , are the query, key and value matrices of scale S; Q is the query matrix, K is the key matrix, and V is the value matrix. is the dimension of the key matrix; is the i-th position of the query matrix, is the jth position of the key matrix, is the j-th position of the value matrix; The classification and bounding box regression module is used to output the category and bounding box coordinates of the target according to the fusion features; Step 3: Input the image of the nuclear medicine imaging device to be detected into the trained NMED-DETR model obtained in step 2 to obtain the detection result of the nuclear medicine imaging device.

2. The target detection method for nuclear medicine imaging equipment based on the improved DETR model as claimed in claim 1, It is characterized in that The nuclear medicine imaging equipment described in step 1 includes a positron emission tomography (PET), a single photon emission computed tomography (SPECT) and a PET / CT scanner.

3. The target detection method for nuclear medicine imaging equipment based on the improved DETR model as claimed in claim 1, It is characterized in that The convolutional neural network described in step 2 uses the following formula to extract multi-scale feature maps: , Where I is the input image, Represents the ResNet-152 network operation, F is the extracted feature map, and the dimension is (H, W, C), where H and W are the height and width of the feature map respectively, and C is the number of channels.

4. The target detection method for nuclear medicine imaging equipment based on the improved DETR model as claimed in claim 1, It is characterized in that The dynamic geometric position encoding module in step 2 adds dynamic geometric position encoding to the feature map using the following formula: , in: are the pixel coordinates in the input image, For even indexes, For odd indexes, is the dimension of the position encoding, It is an adjustment coefficient used to dynamically adjust the position encoding according to image features.

5. The target detection method for nuclear medicine imaging equipment based on the improved DETR model as claimed in claim 1, It is characterized in that The encoder also includes a first residual connection, a first layer normalization, a second residual connection and a second layer normalization, wherein the first residual connection and the first layer normalization are arranged between the first multi-head self-attention layer and the first feedforward neural network, and the second residual connection and the second layer normalization are arranged after the first feedforward neural network.

6. The target detection method for nuclear medicine imaging equipment based on the improved DETR model as claimed in claim 1, It is characterized in that The decoder also includes a third residual connection, a third layer normalization, a fourth residual connection, a fourth layer normalization, a fifth residual connection and a fifth layer normalization, the third layer normalization and the third residual connection are arranged between the second multi-head self-attention layer and the third multi-head self-attention layer, the fourth residual connection and the fourth layer normalization are arranged between the third multi-head self-attention layer and the second feedforward neural network, and the fifth residual connection and the fifth layer normalization are arranged after the second feedforward neural network.

7. The target detection method for nuclear medicine imaging equipment based on the improved DETR model as claimed in claim 1, It is characterized in that The feature extraction network described in step 2 uses the following formula to perform batch normalization calculation on the extracted features: , in: is the adjusted value of the ith sample in feature channel c, is the learnable scaling factor of feature channel c, is the learnable offset of feature channel c, is the value of the i-th sample in the feature channel c after standardization, is a constant, is the batch variance of feature channel c, is the batch mean of feature channel c, m is the batch size, that is, the number of samples in a small batch, is the value of the i-th sample in feature channel c.

8. The target detection method for nuclear medicine imaging equipment based on the improved DETR model as claimed in claim 1, It is characterized in that The loss function of the NMED-DETR model in step 2 is expressed as follows: , In the formula, represents the weight of classification loss, represents the weight of the bounding box regression loss, N is the number of samples, C is the number of categories, is the true category label of the i-th sample, is the predicted category probability of the i-th sample, is the predicted bounding box of the i-th sample, is the true bounding box of the i-th sample.