Lymph node detection method, system, device and medium based on anatomical environment perception

Abstract

The application discloses a lymph node detection method, system, device and medium based on anatomical environment perception, and belongs to the technical field of artificial intelligence medicine, and aims to solve the technical problem of low lymph node detection accuracy. The method comprises the following steps: acquiring sample and label data, constructing and training a lymph node detection model, and performing real-time lymph node detection. The lymph node detection model comprises a feature extraction network, a center slice guided cross-slice attention submodule, a three-dimensional channel and spatial saliency enhancement submodule, a hybrid encoder and a Transformer decoder submodule, and a detection head submodule. The prototype consistency and difficult negative samples are constrained by using an auxiliary anatomical environment perception module. The auxiliary anatomical environment perception module comprises an ROI-level double-view data enhancement submodule, a prototype consistency constraint submodule and a difficult negative sample comparison submodule. The method can enhance the model's ability to distinguish lymph nodes from complex anatomical environments, and the lymph node detection accuracy is higher.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of artificial intelligence medical technology, and relates to the detection of lymph nodes, in particular to a lymph node detection method and system based on anatomical environment perception, a device and a medium. BACKGROUND

[0002] In malignant tumors such as colorectal cancer, abdominal pelvic lymph node detection has important clinical significance in medical image analysis, and its results are directly related to preoperative staging of malignant tumors, development of radiotherapy and chemotherapy plans, and postoperative recurrence monitoring. Computed tomography (CT) as a commonly used non-invasive medical imaging examination method provides a preliminary basis for doctors to evaluate the status of lymph nodes. However, due to the complex anatomical structure of the abdominal pelvic region, lymph nodes are usually small in volume, varied in shape, and often adjacent to blood vessels, intestinal tubes, nerve ganglions and other tissues in CT images, which makes it difficult for doctors to accurately locate and identify each lymph node. Many adjacent structures have similar appearances on images, often leading to false positives and false negatives of lymph nodes. At the same time, manually interpreting a large number of continuous CT images is time-consuming and labor-intensive, and the experience level and fatigue level of doctors can also affect the interpretation results, further increasing the risk of misjudgment and missed detection. In order to achieve automatic detection of small volume lymph nodes in the abdominal pelvic region under complex anatomical environment, researchers have proposed a variety of detection methods based on deep learning, including two-dimensional, three-dimensional and hybrid models combining two-dimensional and three-dimensional, to improve the model's ability to identify lymph nodes in complex anatomical environments and improve the accuracy and efficiency of detection.

[0003] In two-dimensional detection methods, researchers usually use a single CT slice or stack three consecutive grayscale slices into an image similar to an RGB three-channel image as model input. For example, Xu et al. used an improved Faster R-CNN with multi-scale training, path enhancement, online hard sample mining, etc. to identify lung lymph nodes; Wang et al. improved 2D Mask R-CNN and proposed a global-local attention module and multi-task uncertainty loss to detect lymph nodes in abdominal MR images; Zhou et al. proposed the LN-DETR model to improve the detection efficiency of 2D level lung nodules through various module optimization.

[0004] In three-dimensional convolution detection methods, 3D U-Net, VNet or 3D ResNet, etc. are used to directly model three-dimensional volume data at the voxel level. For example, Zhang et al. proposed a 1.5-stage 3D framework for pelvic lymph node detection in colorectal cancer.

[0005] The application patent application with the application number 202211092052.7 discloses a lesion detection method, system, device and storage medium, which comprises the following steps: obtaining a cannula electron microscope lesion picture; obtaining a capsule endoscope lesion picture; enhancing the capsule endoscope lesion picture based on a style transfer model; training and generating a cannula electron microscope picture lesion detection model; fixing the feature extraction layer parameters of the detection model; training the network parameters of the cannula electron microscope picture lesion detection model using the enhanced capsule endoscope picture; generating a capsule endoscope enhanced picture lesion detection model; and performing real-time lesion detection on the capsule endoscope picture.

[0006] The application patent application with the application number 202411833173.1 discloses an abdominal lymph node metastasis prediction method, system and device based on contrast learning, which comprises the following steps: step 1, obtaining sample and label data; step 2, the constructed lymph node metastasis prediction model comprises a feature extraction module, an attention module and a feature fusion module; the feature extraction module comprises a first convolutional block, a first 2.5D convolutional layer, a second convolutional block, a second 2.5D convolutional layer, a third convolutional block and a linear layer arranged in sequence; step 3, pre-training the lymph node metastasis prediction model by using a supervised contrast learning method; step 4, constructing a feature extractor with the same structure as the feature extraction module, and migrating the parameters in the pre-trained feature extraction module to the feature extractor; constructing and training a simple metastasis prediction network; step 5, training the lymph node metastasis prediction model; step 6, real-time metastasis prediction; obtaining an abdominal CT image to be predicted and inputting the trained lymph node metastasis prediction model of step 5, and the lymph node metastasis prediction model outputs a prediction result that the corresponding lymph node is positive.

[0007] As in the above-mentioned application patent, the prior art mostly applies contrast constraints at the query feature level, which lacks semantic alignment and feature distribution constraints at the anatomical environment level, and it is difficult to form robust feature separation between high-similarity structures. In addition, it excludes lymph node samples with a diameter less than 5mm from true positives and false positives in the validation phase, resulting in low resolution of small and low-contrast lymph node features, and low accuracy of lymph node recognition and detection. SUMMARY

[0008] The purpose of the present application is to solve the technical problems of low resolution of small and low-contrast lymph node features and low accuracy of lymph node recognition and detection due to the failure to fully utilize the anatomical environment information around the lymph node, and to provide a lymph node detection method, system, device and medium based on anatomical environment perception.

[0009] In order to achieve the above-mentioned purpose, the present application specifically adopts the following technical solutions:

[0010] A lymph node detection method based on anatomical environment perception, comprising the following steps:

[0011] Step 1, obtaining sample and label data;

[0012] Obtaining CT sample images, and labeling lymph nodes in the CT sample images to obtain label data;

[0013] Step 2, constructing a lymph node detection model;

[0014] The lymph node detection model comprises a feature extraction network, a center slice guided cross-slice attention sub-module, a three-dimensional channel and spatial saliency enhancement sub-module, a hybrid encoder and Transformer decoder sub-module, and a detection head sub-module arranged in sequence; the CT image is input into the feature extraction network and a feature map set is output; the feature map set is input into the center slice guided cross-slice attention sub-module and a context enhanced feature map is output The context enhanced feature map The context enhanced feature map is input into the three-dimensional channel and spatial saliency enhancement sub-module and a final enhanced feature is output The final enhanced feature The final enhanced feature is input into the hybrid encoder and Transformer decoder sub-module and an object query is output; the object query is input into the detection head sub-module and a predicted class confidence and a bounding box position are output;

[0015] Step 3, training the lymph node detection model;

[0016] The sample data and label data obtained in step 1 are used to train the lymph node detection model constructed in step 2; during the training, an auxiliary anatomical environment perception module is used to constrain the prototype consistency and difficult negative samples; the auxiliary anatomical environment perception module comprises an ROI-level dual-view data enhancement sub-module, a prototype consistency constraint sub-module, and a difficult negative sample comparison sub-module; the feature map output at a scale with a down-sampling step length of 8 of the feature extraction network is input into the ROI-level dual-view data enhancement sub-module and a positive sample view is output; the prototype consistency constraint sub-module is used to constrain the output positive sample view, and the difficult negative sample comparison sub-module is used to constrain the difficult negative samples output by the detection head sub-module;

[0017] Step 4, real-time lymph node detection;

[0018] Obtaining a CT image to be detected and inputting it into the lymph node detection model, and the lymph node detection model outputs a prediction result.

[0019] Further, in step 2, the center slice guided cross-slice attention sub-module comprises a projection layer, a cross-attention layer, and a 1*1 convolution layer;

[0020] The feature map set is input into the center slice guided cross-slice attention sub-module, and the feature maps of corresponding levels in the feature map set are projected by the projection layer Pre-slice feature map Current slice feature map and subsequent slice feature maps The mapping is performed using three vectors: query, key, and value. An attention weight matrix A is obtained through a cross-attention layer and a softmax function, and contextual features are obtained through a 1x1 convolutional layer. Contextual features With the current slice feature map The feature maps are fused to obtain context-enhanced feature maps. .

[0021] Furthermore, in step 2, the three-dimensional channel and spatial saliency enhancement submodule includes three processing branches with the same structure, as well as an averaging operation unit and a spatial saliency calculation unit. Each processing branch includes a convolutional layer, a double pooling layer, and a depthwise separable convolutional layer.

[0022] Context-enhanced feature maps The data is fed into three processing branches in parallel; in each processing branch, context-enhanced feature maps are processed. First, a convolutional layer is used to perform dimensionality permutation to obtain the feature perspective. Then, a double pooling layer is used for global average pooling and global max pooling. Finally, attention weights are generated through a depthwise separable convolutional layer and a sigmoid function. Attention weights of the outputs of the three processing branches , , After element-wise multiplication, the input is fed into the averaging unit for averaging to obtain the channel attention weights. Channel attention weights Applied to context-enhanced feature maps To obtain intermediate feature maps Intermediate feature map The data is fed into the spatial saliency calculation unit to obtain the final enhanced features. .

[0023] Furthermore, intermediate feature maps The data is fed into the spatial saliency calculation unit and the final enhanced features are obtained. The specific process is as follows:

[0024] Calculate intermediate feature maps Energy value at each spatial location t The energy diagram E is obtained;

[0025] Then, invert the energy map E and apply the sigmoid function to generate the spatial attention map. ;

[0026] Using spatial attention maps intermediate feature map modulation to obtain final enhanced features .

[0027] Further, in step 3, for each view in the pair of positive sample views (x, x+) output by the ROI-level dual-view data augmentation sub-module, the prototype consistency constraint sub-module calculates the assignment probability of the view to all prototypes and forms an assignment code , and respectively uses each view to predict the optimal assignment code of the other view;

[0028] When the prototype consistency constraint sub-module is used to constrain the output positive sample views, the prototype consistency loss Lconsis is expressed as:

[0029] ;

[0030] ;

[0031] ;

[0032] wherein, represents the prototype assignment of view x using the features of view x, Figure 1 Figure 2 , represents the prototype assignment of view x+ using the features of view x+, Figure 2 Figure 1 , represents the soft assignment weight of the target view on the kth prototype, represents the probability of view z being assigned to the kth prototype, represents the temperature coefficient for controlling the smoothness of the similarity probability distribution, represents the transpose, represents the kth learnable prototype vector, represents the jth prototype vector in the same group as the current sample in the summation operation.

[0033] Further, in step 3, the difficult negative sample contrast sub-module is used to constrain the difficult negative samples output by the detection head sub-module, specifically:

[0034] From the predicted class confidence of the detection head sub-module output, select the candidate box whose predicted class confidence is higher than 0.5 and does not match any real label as the difficult negative sample;

[0035] ​​​​​​​​For each ROI-level dual-view data augmentation submodule, a pair of positive sample views are output. , ), and a set of feature vectors containing R difficult negative samples mined in this batch. Calculate the contrast loss based on infoNCE; contrast loss Represented as:

[0036] ;

[0037] in, Represents cosine similarity. This represents the temperature coefficient used in comparative learning.

[0038] Furthermore, in step 3, when training the lymph node detection model, the global loss... for:

[0039] ;

[0040] ;

[0041] ;

[0042] ;

[0043] ;

[0044] ;

[0045] ;

[0046] in, This indicates the loss incurred by those responsible for overseeing the location and classification of targets. Indicates the weight of the auxiliary total loss. Indicates the total auxiliary loss; Represents the classification loss weights. Represents classification loss, Indicates the regression loss weights. Indicates regression loss; This represents the IoU value of the predicted bounding box that matches the ground truth bounding box. This indicates the confidence level at which the candidate region is identified as a lymph node. This represents the focusing factor that controls the weights of high-scoring negative samples; Indicates the prediction box. Represents the true bounding box. Table used for balancing The weight of the loss This indicates the relationship between the predicted bounding box and the ground truth bounding box in normalized coordinates. distance, represents a weight for balancing the GIoU loss, represents a loss based on the intersection over union; represents an abscissa of a center point of a prediction box, represents an abscissa of a center point of a real box, represents an ordinate of a center point of a prediction box, represents an abscissa of a center point of a real box, represents a width of a prediction box, represents a width of a real box, represents a height of a prediction box, represents a height of a real box; represents a prototype consistency loss, represents a weight of the contrastive loss, represents the contrastive loss.

[0047] A lymph node detection system based on anatomical environment perception, comprising:

[0048] A sample and label data acquisition module is configured to acquire a CT sample image and label lymph nodes in the CT sample image to obtain label data.

[0049] A lymph node detection model construction module is configured to construct a lymph node detection model. The lymph node detection model comprises a feature extraction network, a center slice guided cross-slice attention sub-module, a three-dimensional channel and spatial saliency enhancement sub-module, a hybrid encoder and Transformer decoder sub-module, and a detection head sub-module arranged in sequence. The CT image is input into the feature extraction network and a feature map set is output. The feature map set is input into the center slice guided cross-slice attention sub-module and context enhanced feature maps are output. The context enhanced feature maps are input into the three-dimensional channel and spatial saliency enhancement sub-module and final enhanced features are output. The final enhanced features are input into the hybrid encoder and Transformer decoder sub-module and an object query is output. The object query is input into the detection head sub-module and a predicted class confidence and a bounding box position are output.

[0050] The lymph node detection model training module is configured to train the lymph node detection model constructed by the lymph node detection model construction module by using the sample data and the label data obtained by the sample and label data acquisition module; during the training, the lymph node detection model is constrained by the auxiliary anatomical environment perception module; the auxiliary anatomical environment perception module comprises an ROI-level double-view data enhancement submodule, a prototype consistency constraint submodule and a difficult negative sample comparison submodule; the feature map output by the feature extraction network at a scale of a down-sampling step of 8 is input into the ROI-level double-view data enhancement submodule and outputs a positive sample view, the prototype consistency constraint submodule is used to constrain the prototype consistency of the output positive sample view, and the difficult negative sample comparison submodule is used to constrain the difficult negative sample output by the detection head submodule;

[0051] The lymph node real-time detection module is configured to acquire a CT image to be detected and input the CT image into the lymph node detection model, and the lymph node detection model outputs a prediction result.

[0052] A computer device comprises a memory and a processor, the memory stores a computer program, and the computer program is executed by the processor to enable the processor to execute the steps of the above method.

[0053] A computer readable storage medium stores a computer program, and the computer program is executed by a processor to enable the processor to execute the steps of the above method.

[0054] The beneficial effects of the present application are as follows:

[0055] 1、In the present application, compared with the main detection branch of the lymph node detection model, the auxiliary anatomical environment perception module is arranged in a matched manner, the parallel training of the two branches can be performed without additional labeling, the distinguishing ability of the model for lymph nodes and the complex anatomical environment in which the lymph nodes are located is enhanced, the semantic alignment and feature distribution constraint of the anatomical environment level can be performed, the stable feature separation between high-similarity structures can be formed, the distinguishing ability of the model for small and low-contrast lymph node features is improved, and the recognition and detection accuracy of the lymph nodes is higher.

[0056] 2、In the present application, the cross-slice attention mechanism of the center slice query driven cross-slice attention submodule guided by the center slice realizes adaptive weight allocation, the inter-slice correlation is explicitly modeled while maintaining the detail resolution, and the detection accuracy of small targets in a complex anatomical region is significantly improved.

[0057] 3、In the present application, the prototype consistency constraint submodule only performs prototype alignment on positive samples, in-silico semantic preservation is realized by dynamic prototype memory updating, and the feature aggregation stability under small sample conditions is significantly improved.

[0058] 4、In the present application, the high confidence false positive candidate predicted by the detection head is used as the source of explicit negative samples, and the difficult negative sample comparison submodule is used to constrain the difficult negative samples output by the detection head submodule in the training process, so that the model can effectively identify the "highly similar non-target" structure and reduce the false positive detection in the pelvic CT. BRIEF DESCRIPTION OF DRAWINGS

[0059] Figure 1 is a flowchart of the present application;

[0060] Figure 2 is a structural diagram of the lymph node detection model and the auxiliary anatomical environment perception module in the present application;

[0061] Figure 3 is a structural diagram of the feature extraction network in the present application;

[0062] Figure 4 is a structural diagram of the center slice guided cross-slice attention submodule in the present application;

[0063] Figure 5 is a structural diagram of the three-dimensional channel and spatial saliency enhancement submodule in the present application;

[0064] Figure 6 is an interactive prediction diagram of the prototype consistency constraint submodule in the present application. DETAILED DESCRIPTION

[0065] In order to make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all the embodiments.

[0066] Therefore, based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application.

[0067] Embodiment 1

[0068] The present embodiment provides a lymph node detection method based on anatomical environment perception, which is used for identifying and detecting lymph nodes in chest cavity, abdominal cavity and the like. As shown in the figure, it includes the following steps: Figure 1

[0069] Step 1, obtaining sample and label data;

[0070] Obtaining CT sample images and labeling lymph nodes in the CT sample images to obtain label data.

[0071] ​A number of three-dimensional CT sample images are obtained from the West China Hospital of Sichuan University, and lymph nodes in the CT sample images are labeled by experienced professional doctors to obtain label data.

[0072] The obtained CT sample images are preprocessed and labeled, and the specific method is as follows:

[0073] 1. Normalization processing;

[0074] The obtained three-dimensional CT sample images are subjected to gray scale normalization processing. A window width of 300HU and a window level of 40HU are set, and Hounsfield Unit (HU) is linearly mapped to a gray scale interval of 0-255 to enhance the contrast of the target tissue and the surrounding structure; then spatial standardization is performed, and all body data are unified to a thickness of 2.0mm and a plane resolution of 0.75mm*0.75mm through three-dimensional resampling of voxel interpolation, to ensure the consistency of the spatial scale among samples.

[0075] 2. Noise removal processing;

[0076] Human contour segmentation algorithm is used to remove background noise such as scanning bed, and the body data is cropped according to key frame or anatomical positioning information to concentrate computing resources in the effective area where the lesion is located. In order to preserve the three-dimensional context information, the cropped data block is sliced along the Z axis, each slice is stacked with its adjacent two frames before and after it, forming a three-channel pseudo-color image, and fusing local spatial features.

[0077] The lymph nodes in the CT sample images after noise removal processing are labeled. During labeling, a labeling software is used for labeling, and experienced professional doctors find out the 2D image (i.e. two-dimensional slice) with the largest lymph node slice by sliding the mouse, and label the lymph node in the 2D image. During labeling, the data labeled is the lymph node center point coordinates (x, y, z) and diameter, and it is processed into a 2D box representing the lymph node.

[0078] Finally, a structured data set conforming to the COCO format is output, including uniform multi-channel two-dimensional CT image samples and their corresponding bounding box labels, providing high-quality training samples for subsequent detection models.

[0079] 3. Data augmentation processing;

[0080] A multi-channel data loading and enhancement mechanism is introduced to perform data enhancement processing on the two-dimensional CT image after noise processing, so as to enhance the spatial continuity of the features in the CT image and the model generalization ability. Specifically, through an extended CT CocoDetection dataset class, the class establishes a correspondence between the slices and the original three-dimensional sequence according to the CT image mapping file on the basis of loading the two-dimensional slices and the boundary box labels; during data loading, the neighboring multiple frames of images before and after each center slice are dynamically extracted to form a multi-slice input group containing 9 frames of CT information, so that more rich local three-dimensional context features are introduced.

[0081] Step 2, constructing a lymph node detection model;

[0082] The lymph node detection model comprises a feature extraction network, a center slice guided cross-slice attention submodule, a three-dimensional channel and spatial saliency enhancement submodule, a mixed encoder and Transformer decoder submodule, and a detection head submodule arranged in sequence; the CT image is input into the feature extraction network and a feature map set is output, the feature map set is input into the center slice guided cross-slice attention submodule and a context enhanced feature map is output , the context enhanced feature map is input into the three-dimensional channel and spatial saliency enhancement submodule and a final enhanced feature is output , the final enhanced feature is input into the mixed encoder and Transformer decoder submodule and an object query is output, the object query is input into the detection head submodule and a predicted class confidence and a bounding box position are output.

[0083] The feature extraction network, the center slice guided cross-slice attention submodule, the three-dimensional channel and spatial saliency enhancement submodule, the mixed encoder and Transformer decoder submodule, and the detection head submodule jointly constitute a main detection branch (corresponding to a subsequent auxiliary anatomical environment perception branch), as shown in Figure 2 .

[0084] For the feature extraction network, the feature extraction network comprises three backbone networks arranged side by side, each backbone network is a Resnet50 network, and the three Resnet50 networks share parameters. As shown in Figure 3 , the multi-slice input group containing 9 frames of CT information is divided into three groups (the first group is Z-4, Z-3, Z-2; the second group is Z-1, Z, Z+1; the third group is Z+2, Z+3, Z+4) in the order of adjacency, and is input into the corresponding backbone network for hierarchical feature extraction (the multi-slice input of the first group is input into the first backbone network, the multi-slice input of the second group is input into the second backbone network, and the multi-slice input of the third group is input into the third backbone network), and the three backbone networks output previous feature maps , current feature maps and the subsequent feature maps . The process aims to transform the input pixel-level information into a set of feature maps with rich semantics and multi-scale , where represents the feature map of the i-th level, denotes the batch size, denotes the channel dimension, , denotes the input image size. The feature map set provides a comprehensive feature representation from low-level texture to high-level semantics for subsequent.

[0085] For the center slice guided cross-slice attention submodule, the center slice guided cross-slice attention submodule includes a projection layer, a cross-attention layer and a 1*1 convolution layer, as shown in Figure 4 . The feature map set inputs the center slice guided cross-slice attention submodule, and the feature maps of the corresponding levels in the feature map set are mapped into three vectors of query, key and value through the projection layer; , the current slice feature map and the subsequent slice feature map ;

[0086] ;

[0087] ;

[0088] ;

[0089] wherein, , , denotes a learnable weight matrix, denotes a concatenation operation along the channel dimension.

[0090] The attention weight matrix A is obtained through the cross-attention layer and the Softmax function. The attention weight matrix A is obtained by calculating the dot product of the query and the key, and is normalized by the softmax function to quantify the degree of dependence of each pixel position in the center slice on each position in the adjacent slice. The attention weight matrix A is represented as:

[0091] ;

[0092] wherein, denotes the dimension of the key vector, which is used for scaling to prevent gradient vanishing.

[0093] Then, through the 1*1 convolution layer, the attention weight matrix A is applied to the value vector to obtain the context feature after weighted aggregation; the context feature is represented as:

[0094] ;

[0095] Finally, the aggregated context features are fused with the current slice feature map through a residual connection or the like to generate and obtain the final context-enhanced feature map :

[0096] .

[0097] The center slice-guided cross-slice attention sub-module can adaptively focus on the most information-rich areas in adjacent slices through the above mechanism, thereby effectively enhancing the three-dimensional spatial continuity expression of features.

[0098] For the three-dimensional channel and spatial saliency enhancement sub-module, the three-dimensional channel and spatial saliency enhancement sub-module includes three structurally identical processing branches, an average operation unit (Avg), and a spatial saliency calculation unit, and each processing branch includes a convolution layer, a double pooling layer, and a depth separable convolution layer, as shown in Figure 5 .

[0099] After the center slice-guided cross-slice attention sub-module obtains the fused context-enhanced feature map , in order to further improve its representation ability, a cascaded double attention enhancement strategy is adopted in the three-dimensional channel and spatial saliency enhancement sub-module. Specifically:

[0100] First, a three-channel attention fusion mechanism is used to learn the complex dependency relationship between channels. The context-enhanced feature map is fed into three structurally identical processing branches in parallel, and each branch learns from three orthogonal dimensional perspectives; in each processing branch, the context-enhanced feature map is first dimensionally transformed by a 1*1 convolution layer to generate and obtain three different feature perspectives. Specifically, it is represented as:

[0101] .

[0102] Subsequently, parallel global average pooling and global maximum pooling are performed through a double pooling layer to capture the average and peak responses of the channels. Among them, the global average pooling is represented as:

[0103] ;

[0104] Then, a depth separable convolution layer and a Sigmoid function are used to generate attention weights in the range of [0, 1]; that is:

[0105] ;

[0106] The attention weights output by the three processing branches 、 、 After element-by-element multiplication, the average operation unit is input for average operation processing, and is fused into a unified channel attention weight ; that is,

[0107] ;

[0108] The channel attention weight is applied to the context-enhanced feature map through broadcast multiplication , to obtain a channel-refined intermediate feature map ; that is,

[0109] ;

[0110] After completing the semantic enhancement of the channel dimension, the intermediate feature map is serially sent to the spatial saliency calculation unit and is modulated by the spatial attention map to obtain the final enhanced feature .

[0111] The intermediate feature map is sent to the spatial saliency calculation unit and the specific process of obtaining the final enhanced feature is as follows:

[0112] An energy function is defined for each position t in the feature map, and the importance of the position is evaluated by the energy function. The closed-form solution of the energy function can be efficiently calculated as:

[0113] ;

[0114] wherein 、 respectively represent the mean and variance of all other neurons in the channel except neuron t, denotes a regularization hyperparameter. The lower the energy value, the higher the degree of differentiation of the position from the surrounding area, and the higher the importance of the position.

[0115] The energy value of each spatial position t in the intermediate feature map is calculated to obtain an energy map . Wherein, essentially depicts the deviation degree of position t relative to the mean square deviation normalization of other positions in the same channel, that is, the "prominence" of the position in the channel.

[0116] Take the energy map E again and apply the Sigmoid function, that is , to generate a spatial attention map with a dimension of ;

[0117] Finally, the spatial attention map is used to modulate the intermediate feature map to obtain the final enhanced feature ; denoted as:

[0118] .

[0119] For the hybrid encoder and Transformer decoder sub-modules, the module adopts a Transformer-based detector architecture to process the enhanced multi-scale features. The hybrid encoder receives attention-enhanced enhanced features from multiple levels as input. Inside the encoder, the local and global context information of different scale feature maps are further integrated through the alternating action of convolution operation and self-attention mechanism, generating a set of refined memory features containing rich semantic information. The Transformer decoder receives the memory features output by the encoder and a set of learnable, fixed number of object queries as input. In each decoder layer, the object queries first model the relationship with each other through self-attention mechanism, and then interact with the memory features output by the encoder through cross-attention mechanism. This process is iteratively performed, so that each object query can gradually focus on a potential target in the image and absorb its feature information.

[0120] Based on the above description, the technical personnel in the art can implement the hybrid encoder and Transformer decoder sub-modules described above without creative labor based on the conventional technical means in the art and related prior art.

[0121] For the detection head sub-module, the detection head sub-module processes the optimized object query through a feedforward neural network shared by two parameters: the classification branch predicts the target class confidence, and the regression branch predicts the bounding box position information. In particular, for the predicted samples with high confidence but proved to be false positives, their features and position information are extracted and fed back to the auxiliary contrastive learning branch to guide the model to more accurately distinguish real lymph nodes from high-similarity artifacts or normal tissues, thereby reducing the false detection rate.

[0122] Based on the above description, the technical personnel in the art can implement the detection head sub-module described above without creative labor based on the conventional technical means in the art and related prior art.

[0123] Step 3, train the lymph node detection model;

[0124] The sample and label data obtained in step 1 are used to train the lymph node detection model constructed in step 2; during the training, the original prototype consistency and difficult negative samples are constrained by using an auxiliary dissection environment perception module; the auxiliary dissection environment perception module includes an ROI-level double-view data enhancement sub-module, an original prototype consistency constraint sub-module, and a difficult negative sample comparison sub-module; the feature map output by the feature extraction network at a scale of a down-sampling step of 8 is input into the ROI-level double-view data enhancement sub-module and outputs a positive sample view, the original prototype consistency constraint sub-module is used to constrain the output positive sample view, and the difficult negative sample comparison sub-module is used to constrain the difficult negative sample output by the detection head sub-module.

[0125] In this embodiment, without additional labeling, an auxiliary dissection environment perception module is specially set to enhance the model's ability to distinguish lymph nodes and the complex dissection environment in which they are located. The auxiliary dissection environment perception module is trained in parallel with the main detection branch, and its loss function is integrated into the global total loss to guide the main detection branch to generate discriminative feature representations.

[0126] The auxiliary dissection environment perception module reuses the feature extraction network in the main detection branch, which directly uses the feature map output by the feature extraction network at a scale of a down-sampling step of 8, and the channel dimension of the feature map is 512. Therefore, the auxiliary branch can optimize the features that have undergone preliminary semantic extraction without increasing the computational overhead of the additional backbone network (i.e., the feature extraction network).

[0127] For the ROI-level double-view data enhancement sub-module, in order to construct a positive sample pair for contrastive learning, the module performs double-view data enhancement on each real lymph node region of interest (ROI) in the training batch. The process first locates the center point of the lymph node on the feature map and dynamically extracts a rectangular feature block with a side length ranging from 4 to 8 pixels according to its original size; then, by applying two independent random data augmentations to the feature block, a pair of positive sample views that are both related and different are generated 、 ). The specific enhancement operations include applying a random translation perturbation of up to 1.5 pixels to the center position of the feature block and performing scale scaling within a range of ± 20%. Through this series of operations, each lymph node instance is finally converted into a pair of slightly perturbed positive sample views in the feature space, providing key inputs for subsequent contrastive learning tasks.

[0128] For the original prototype consistency constraint sub-module, the module uses a commutative assignment multi-view mechanism, as shown in Figure 6 , to apply original prototype consistency constraints to the positive sample pair generated by the ROI-level double-view data enhancement sub-module. Specifically:

[0129] First, a projection head consisting of two layers of multilayer perceptron (MLP), layer normalization (LayerNorm), and rectified linear unit (ReLU) maps the 512-dimensional view features z to a 128-dimensional unit hypersphere space, maintaining a set of 64 learnable prototype vectors. Each prototype It represents a cluster center in the feature space.

[0130] For each view feature z, this module calculates its assignment probability to all prototypes, forming an assignment code. Here This does not refer to a rigid optimal allocation, but rather to the soft allocation weights of sample features on the k-th prototype. This allocation encoding is iteratively optimized three times using the Sinkhorn-Knopp algorithm to ensure that all samples are distributed as evenly as possible across prototypes, preventing the model from collapsing to use only a few prototypes.

[0131] The core idea of ​​this module is "exchange," which utilizes the features of the first view. To predict the optimal allocation encoding for the second view. Conversely, utilize the features of the second view. To predict the optimal allocation encoding for the first view (Optimal allocation encoding, i.e., prototype allocation). When applying prototype consistency constraints to the output positive sample view using the prototype consistency constraint submodule, the prototype consistency loss is defined as the sum of the cross-entropy losses in both directions. This prototype consistency loss... Represented as:

[0132] ;

[0133] ;

[0134] ;

[0135] in, Indicates the use of vision Figure 1 Features Defitting the view Figure 2 Prototype allocation , Indicates the use of vision Figure 2 Features Defitting the view Figure 1 Prototype allocation ; This represents the k-th element of the allocation code q, i.e., the target view (e.g.) Soft-assignment weight on the k-th prototype, and the "target assignment" predicted in the swap prediction; to predict this assignment, the model computes , represents the probability that view z (such as ) is assigned to the k-th prototype; this probability is derived by a softmax function with temperature ; represents the temperature coefficient that controls the smoothness of the similarity probability distribution, which is set to 0.25 in this embodiment; represents the transpose; the core of this module is , represents the k-th learnable prototype vector, which serves as a clustering center in the feature space to compute similarity with feature z; finally, to ensure is a valid probability distribution, the denominator of the softmax function is normalized by summing over all prototype vectors , represents the j-th prototype vector that belongs to the same group as the current sample in the summation operation.

[0136] For the difficult negative sample comparison submodule, to further improve the model's ability to distinguish high-similarity organizations, this module introduces an explicit contrastive learning mechanism, which uses the difficult negative samples generated by the main detection branch for constraint. Specifically:

[0137] This module selects those candidate boxes from the output of the detection head whose prediction confidence is higher than 0.5 but do not match any real label as difficult negative samples.

[0138] For each pair of positive samples views output by the ROI-level dual-view data augmentation submodule , ), for each positive sample feature vector , the sample feature vector of the other augmented view corresponding to it , and a set of R difficult negative sample feature vectors mined in this batch, the contrastive loss based on infoNCE is calculated. This loss aims to pull the distance between the positive sample pair , ) in the feature space closer, while pushing the distance between the sample and all negative samples farther. Therefore, the contrastive loss is expressed as:

[0139] ;

[0140] where represents the cosine similarity; represents the temperature coefficient of contrastive learning, and is set to 0.07; R represents the number of difficult negative samples selected in each training batch, and is set to 16.

[0141] When training the lymph node detection model, the weights of the entire model are synchronously updated by constructing and minimizing a unified, multi-task global loss. The global loss is:

[0142] ;

[0143] ;

[0144] ;

[0145] ;

[0146] ;

[0147] ;

[0148] ;

[0149] wherein, represents a loss responsible for supervision of positioning and classification of targets, represents an auxiliary total loss weight, represents an auxiliary total loss; represents a classification loss weight; represents a classification loss, which adopts a variable focus loss, which can better handle the problems of imbalance between positive and negative samples and different difficulties of samples; represents a regression loss weight, represents a regression loss; represents an IoU value of a prediction box matched with a real box, represents a confidence of the candidate region being identified as a lymph node, specifically, for a positive sample successfully matched with a certain real annotation box, the value is set to the intersection over union with the corresponding real annotation box, and thus is a floating point number between (0, 1), and for a negative sample not matched with any real annotation box, the value is 0; represents a weight for adjusting the contribution of positive and negative samples to the total loss, and is set to 0.75; represents a focus factor for controlling the weight of high-score negative samples, which is used to reduce the loss weight of easy-to-classify samples, so that the model training focuses more on learning difficult negative samples, and is set to 2.0; represents a prediction box, represents a real box; represents a table for balancing a weight of the loss, taking a value of 5; denotes a distance between the predicted box and the real box in the normalized coordinates denotes a distance between the predicted box and the real box in the normalized coordinates denotes a weight for balancing the GIoU loss, taking a value of 2; denotes a loss based on the intersection over union; denotes an abscissa of a center point of the predicted box, denotes an abscissa of a center point of the real box, denotes an ordinate of a center point of the predicted box, denotes an ordinate of a center point of the real box, denotes a width of the predicted box, denotes a width of the real box, denotes a height of the predicted box, denotes a height of the real box; denotes a prototype consistency loss, denotes a weight of the contrastive loss, denotes the contrastive loss.

[0150] Step 4, real-time lymph node detection;

[0151] The CT image to be detected is input into the lymph node detection model, and the CT image to be detected is sequentially processed by the feature extraction network, the center slice guided cross-slice attention submodule, the three-dimensional channel and spatial saliency enhancement submodule, the hybrid encoder and Transformer decoder submodule, and the detection head submodule of the lymph node detection model, and finally the prediction result of the lymph node is output by the detection head submodule of the lymph node detection model.

[0152] Embodiment 2

[0153] The embodiment provides a lymph node detection system based on anatomical environment perception, comprising:

[0154] A sample and label data acquisition module is configured to acquire a CT sample image and label lymph nodes in the CT sample image to obtain label data.

[0155] A lymph node detection model construction module is configured to construct a lymph node detection model. The lymph node detection model comprises, in sequence, a feature extraction network, a center slice guided cross-slice attention submodule, a three-dimensional channel and spatial saliency enhancement submodule, a hybrid encoder and Transformer decoder submodule, and a detection head submodule. The CT image is input into the feature extraction network and outputs a feature map set. The feature map set is input into the center slice guided cross-slice attention submodule and outputs a context enhanced feature map The context enhanced feature map is input into the three-dimensional channel and spatial saliency enhancement submodule and outputs a final enhanced feature ​Final enhanced feature The input mixed encoder and the Transformer decoder submodule output an object query, and the object query is input into the detection head submodule to output a predicted category confidence and a bounding box position.

[0156] The lymph node detection model training module is configured to train the lymph node detection model constructed by the lymph node detection model construction module using the sample data and the label data obtained by the sample and label data acquisition module. During the training, the prototype consistency and the difficult negative sample are constrained by using the auxiliary dissection environment perception module. The auxiliary dissection environment perception module includes an ROI-level dual-view data enhancement submodule, a prototype consistency constraint submodule, and a difficult negative sample comparison submodule. The feature map output at a scale of a down-sampling step of 8 of the feature extraction network is input into the ROI-level dual-view data enhancement submodule and outputs a positive sample view. The prototype consistency constraint submodule is used to constrain the output positive sample view, and the difficult negative sample comparison submodule is used to constrain the difficult negative sample output by the detection head submodule.

[0157] The lymph node real-time detection module is configured to acquire a CT image to be detected and input the CT image into the lymph node detection model, and the lymph node detection model outputs a prediction result.

[0158] Embodiment 3

[0159] A computer device includes a memory and a processor, the memory stores a computer program, and the computer program is executed by the processor to make the processor execute the steps of the lymph node detection method based on dissection environment perception.

[0160] The computer device can be a desktop computer, a notebook computer, a palm computer, a cloud server, and the like. The computer device can interact with a user through a keyboard, a mouse, a remote controller, a touchpad, a voice control device, and the like.

[0161] The memory includes at least one type of readable storage medium, including flash memory, hard disk, multimedia card, card-type memory (e.g., SD or D interface display memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory can be an internal storage unit of the computer device, such as a hard disk or memory of the computer device. In other embodiments, the memory can also be an external storage device of the computer device, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc. equipped on the computer device. Of course, the memory can also include both the internal storage unit and the external storage device of the computer device. In this embodiment, the memory is commonly used to store the operating system and various application software installed on the computer device, such as the program code of the lymph node detection method based on anatomical environment perception, etc. In addition, the memory can also be used to temporarily store various data that have been output or will be output.

[0162] The processor in some embodiments can be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip. The processor is generally used to control the overall operation of the computer device. In this embodiment, the processor is used to run the program code or process data stored in the memory, such as running the program code of the lymph node detection method based on anatomical environment perception.

[0163] Embodiment 4

[0164] A computer readable storage medium storing a computer program, the computer program being executed by a processor to cause the processor to perform the steps of the lymph node detection method based on anatomical environment perception.

[0165] The computer readable storage medium stores an interface display program, which can be executed by at least one processor to cause the at least one processor to perform the steps of the lymph node detection method based on anatomical environment perception as described above.

[0166] Through the description of the above embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment method can be realized by means of software and the necessary general hardware platform, of course, it can also be realized by hardware, but in many cases, the former is a better embodiment. Based on such understanding, the technical solutions of the present application can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a plurality of instructions for making a terminal device (which can be a mobile phone, computer, server or network device, etc.) execute the lymph node detection method based on anatomical environment perception described in the embodiments of the present application.

Claims

1. A lymph node detection method based on anatomical environment perception, characterized in that, Includes the following steps: Step 1: Obtain sample and label data; Acquire CT sample images and label the lymph nodes in the CT sample images to obtain label data; Step 2: Construct a lymph node detection model; The lymph node detection model comprises, in sequence, a feature extraction network, a central slice-guided cross-slice attention submodule, a 3D channel and spatial saliency enhancement submodule, a hybrid encoder and Transformer decoder submodule, and a detection head submodule. CT images are input to the feature extraction network, which outputs a set of feature maps. These feature map sets are then input to the central slice-guided cross-slice attention submodule, which outputs context-enhanced feature maps. Context-enhanced feature maps Input the 3D channel and spatial saliency enhancement submodule and output the final enhanced features. Ultimately enhance features The input is the hybrid encoder and Transformer decoder submodule and the output is the object query. The object query is input to the detection head submodule and the output is the predicted class confidence and bounding box position. Step 3: Train the lymph node detection model; The lymph node detection model constructed in step 2 is trained using the sample data and label data obtained in step 1. During training, an auxiliary anatomical environment perception module is used to constrain prototype consistency and difficult negative samples. The auxiliary anatomical environment perception module includes a ROI-level dual-view data augmentation submodule, a prototype consistency constraint submodule, and a difficult negative sample comparison submodule. The feature map output by the feature extraction network at a downsampling step size of 8 is input into the ROI-level dual-view data augmentation submodule and outputs a positive sample view. The prototype consistency constraint submodule is used to constrain the prototype consistency of the output positive sample view, and the difficult negative sample comparison submodule is used to constrain the difficult negative samples output by the detection head submodule. Step 4, real-time lymph node detection; The CT image to be detected is acquired and input into the lymph node detection model, which then outputs the prediction results.

2. The lymph node detection method based on anatomical environment perception as described in claim 1, characterized in that, In step 2, the cross-slice attention submodule guided by the central slice includes a projection layer, a cross attention layer, and a 1*1 convolutional layer; The cross-slice attention submodule, guided by the input center slice of the feature map set, projects the feature maps of the corresponding levels in the feature map set through a projection layer. Pre-slice feature map Current slice feature map and subsequent slice feature maps The mapping is performed using three vectors: query, key, and value. An attention weight matrix A is obtained through a cross-attention layer and a softmax function, and contextual features are obtained through a 1x1 convolutional layer. ; Contextual features With the current slice feature map The feature maps are fused to obtain context-enhanced feature maps. .

3. The lymph node detection method based on anatomical environment perception as described in claim 1, characterized in that, In step 2, the 3D channel and spatial saliency enhancement submodule includes three structurally identical processing branches, as well as an averaging operation unit and a spatial saliency calculation unit. Each processing branch includes a convolutional layer, a double pooling layer, and a depthwise separable convolutional layer. Context-enhanced feature maps Three processing branches are fed in parallel; In each processing branch, context-enhanced feature maps First, a convolutional layer is used to perform dimensionality permutation to obtain the feature perspective. Then, a double pooling layer is used for global average pooling and global max pooling. Finally, attention weights are generated through a depthwise separable convolutional layer and a sigmoid function. ; Attention weights of the outputs of the three processing branches , , After element-wise multiplication, the input is fed into the averaging unit for averaging to obtain the channel attention weights. Channel attention weights Applied to context-enhanced feature maps To obtain intermediate feature maps Intermediate feature map The data is fed into the spatial saliency calculation unit to obtain the final enhanced features. .

4. The lymph node detection method based on anatomical environment perception as described in claim 3, characterized in that, intermediate feature map The data is fed into the spatial saliency calculation unit and the final enhanced features are obtained. The specific process is as follows: Calculate intermediate feature maps Energy value at each spatial location t The energy diagram E is obtained; Then, invert the energy map E and apply the sigmoid function to generate the spatial attention map. ; Using spatial attention maps For intermediate feature maps Modulation is performed to obtain the final enhanced features. .

5. The lymph node detection method based on anatomical environment perception as described in claim 1, characterized in that, In step 3, for the pair of positive sample views output by the ROI-level dual-view data augmentation submodule ( , For each view in the model, the prototype consistency constraint submodule calculates the assignment probability of that view to all prototypes and forms an assignment code. And each view is used to predict the optimal allocation code for the other view; When applying prototype consistency constraints to the output positive sample view using the prototype consistency constraint submodule, the prototype consistency loss is... Represented as: ； ； ； in, Indicates the features of view 1 To fit the prototype assignment of view 2 , Indicates the features of view 2 To fit the prototype assignment of view 1 , This represents the soft-assigned weight of the target view on the k-th prototype. This represents the probability that view z is assigned to the k-th prototype. The temperature coefficient represents the smoothness of the similarity probability distribution. Indicates transpose. Let represent the k-th learnable prototype vector. This represents the j-th prototype vector in the same group as the current sample in the summation operation.

6. The lymph node detection method based on anatomical environment perception as described in claim 1, characterized in that, In step 3, the difficult negative samples output by the detection head submodule are constrained using the difficult negative sample comparison submodule, specifically as follows: From the predicted class confidence scores output by the detection head submodule, candidate boxes with predicted class confidence scores higher than 0.5 and that do not match any true labels are selected as hard negative samples. For each ROI-level dual-view data augmentation submodule, a pair of positive sample views are output. , ), and a set of feature vectors containing R difficult negative samples mined in this batch. Calculate the contrast loss based on infoNCE; contrast loss Represented as: ； in, Represents cosine similarity. This represents the temperature coefficient used in comparative learning.

7. The lymph node detection method based on anatomical environment perception as described in claim 1, characterized in that, In step 3, when training the lymph node detection model, the global loss... for: ； ； ； ； ； ； ； in, This indicates the loss incurred by those responsible for overseeing the location and classification of targets. Indicates the weight of the auxiliary total loss. Indicates the total auxiliary loss; Represents the classification loss weights. Represents classification loss, Indicates the regression loss weights. Indicates regression loss; This represents the IoU value of the predicted bounding box that matches the ground truth bounding box. This indicates the confidence level at which the candidate region is identified as a lymph node. This indicates the weights used to adjust the contribution of positive and negative samples to the total loss. This represents the focusing factor that controls the weights of high-scoring negative samples; Indicates the prediction box. Represents the true bounding box. Tables are used for balancing The weight of the loss This indicates the relationship between the predicted bounding box and the ground truth bounding box in normalized coordinates. distance, This represents the weights used to balance the GIoU loss. This represents the loss based on the intersection-union ratio; This represents the x-coordinate of the center point of the prediction box. This represents the x-coordinate of the center point of the true bounding box. This represents the ordinate of the center point of the prediction box. This represents the x-coordinate of the center point of the true bounding box. Indicates the width of the prediction box. This represents the width of the actual bounding box. Indicates the height of the predicted bounding box. Indicates the height of the actual bounding box; This represents the prototype consistency loss. Indicates the weights of the comparative loss. This indicates a comparative loss.

8. A lymph node detection system based on anatomical environment perception, characterized in that, include: The sample and label data acquisition module is used to acquire CT sample images and label the lymph nodes in the CT sample images to obtain label data. The lymph node detection model construction module is used to build a lymph node detection model. The lymph node detection model includes, in sequence, a feature extraction network, a central slice-guided cross-slice attention submodule, a 3D channel and spatial saliency enhancement submodule, a hybrid encoder and Transformer decoder submodule, and a detection head submodule. CT images are input to the feature extraction network and output a set of feature maps. The feature map set is then input to the central slice-guided cross-slice attention submodule and outputs a context-enhanced feature map. Context-enhanced feature maps Input the 3D channel and spatial saliency enhancement submodule and output the final enhanced features. Ultimately enhance features The input is the hybrid encoder and Transformer decoder submodule and the output is the object query. The object query is input to the detection head submodule and the output is the predicted class confidence and bounding box position. The lymph node detection model training module is used to train the lymph node detection model constructed by the lymph node detection model construction module using sample and label data acquired by the sample and label data acquisition module. During training, the auxiliary anatomical environment perception module is used to constrain prototype consistency and difficult negative samples. The auxiliary anatomical environment perception module includes a ROI-level dual-view data augmentation submodule, a prototype consistency constraint submodule, and a difficult negative sample comparison submodule. The feature map output by the feature extraction network at a downsampling step size of 8 is input into the ROI-level dual-view data augmentation submodule and outputs a positive sample view. The prototype consistency constraint submodule is used to constrain the prototype consistency of the output positive sample view, and the difficult negative sample comparison submodule is used to constrain the difficult negative samples output by the detection head submodule. The real-time lymph node detection module is used to acquire the CT image to be detected and input it into the lymph node detection model. The lymph node detection model outputs the prediction results.

9. A computer device, characterized in that: It includes a memory and a processor, the memory storing a computer program that, when executed by the processor, causes the processor to perform the steps of the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The device stores a computer program that, when executed by a processor, causes the processor to perform the steps of the method as described in any one of claims 1 to 7.

Images