Lymph node detection method, system and equipment based on anatomical environment perception and medium

Abstract

The invention discloses a lymph node detection method, system and equipment based on anatomical environment perception and a medium, belongs to the technical field of artificial intelligence medicine, and aims to solve the technical problem of low accuracy of lymph node detection. Comprising the steps of acquiring samples and label data, constructing and training a lymph node detection model, and performing real-time lymph node detection. The lymph node detection model comprises a feature extraction network, a cross-slice attention sub-module guided by a central slice, a three-dimensional channel and space significance enhancement sub-module, a hybrid encoder and Transform decoder sub-module and a detection head sub-module; an auxiliary anatomy environment sensing module is adopted to constrain prototype consistency and difficult negative samples; the auxiliary anatomy environment perception module comprises an ROI-level double-view data enhancement sub-module, a prototype consistency constraint sub-module and a difficult negative sample comparison sub-module. The method can enhance the distinguishing ability of the model to the lymph node and the complex anatomical environment where the model is located, and the lymph node detection accuracy is higher.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence medical technology, and relates to the detection of lymph nodes, and more particularly to a method, system, device and medium for lymph node detection based on anatomical environment perception. Background Technology

[0002] In malignant tumors such as colorectal cancer, the detection of abdominopelvic lymph nodes plays a crucial clinical role in medical image analysis. The results directly impact preoperative staging, radiotherapy and chemotherapy regimen development, and postoperative recurrence monitoring. Computed tomography (CT), a commonly used non-invasive medical imaging technique, provides physicians with preliminary information to assess lymph node status. However, due to the complex anatomical structure of the abdominopelvic region, lymph nodes in CT images are typically small, varied in shape, and often located close to blood vessels, intestines, nerve ganglia, and other tissues, making it difficult for physicians to accurately locate and identify each lymph node. Many adjacent structures appear very similar to lymph nodes on imaging, often leading to misdiagnosis and missed detection. Furthermore, manually interpreting large batches of continuous CT images is time-consuming and laborious; factors such as the physician's experience level and fatigue can also affect the interpretation results, further increasing the risk of misdiagnosis and missed detection. To enable automated detection of small-volume lymph nodes in complex anatomical environments, researchers have proposed various deep learning-based detection methods, including two-dimensional, three-dimensional, and hybrid models combining two-dimensional and three-dimensional approaches, to improve the model's ability to identify lymph nodes in complex anatomical environments and enhance the accuracy and efficiency of detection.

[0003] In two-dimensional detection methods, researchers typically use a single CT slice or three consecutive grayscale slices stacked into an RGB-like three-channel image as model input. For example, Xu et al. used an improved Faster R-CNN that incorporated multi-scale training, path enhancement, and online hard sample mining to identify pulmonary lymph nodes; Wang et al. improved 2D Mask R-CNN, proposing a global-local attention module and multi-task uncertainty loss to detect lymph nodes in abdominal MR images; and Zhou et al.'s LN-DETR model improved the detection efficiency of pulmonary nodules in 2D planes through various module optimizations.

[0004] In 3D convolutional detection methods, 3D volumetric data is directly modeled using voxel-level networks such as 3D U-Net, VNet, or 3D ResNet. For example, Zhang et al. proposed a 1.5-stage 3D framework for pelvic lymph node detection in colorectal cancer.

[0005] Patent application No. 202211092052.7 discloses a lesion detection method, system, device, and storage medium, including the following steps: acquiring lesion images from an electron microscope (EM) with a cannula; acquiring lesion images from a capsule endoscope; enhancing the capsule endoscope lesion images based on a style transfer model; training and generating a lesion detection model from the EEM images; fixing the feature extraction layer parameters of the detection model; training the network parameters of the lesion detection model from the EEM images using the enhanced capsule endoscope images; generating a lesion detection model from the enhanced capsule endoscope images; and performing real-time lesion detection on the capsule endoscope images.

[0006] Patent application No. 202411833173.1 discloses a method, system, and device for predicting abdominal lymph node metastasis based on contrastive learning, which includes the following steps: Step 1, acquiring sample and label data; Step 2, constructing a lymph node metastasis prediction model including a feature extraction module, an attention module, and a feature fusion module; the feature extraction module includes a first convolutional block, a first 2.5D convolutional layer, a second convolutional block, a second 2.5D convolutional layer, a third convolutional block, and a linear layer arranged sequentially; Step 3, pre-training the lymph node metastasis prediction model using supervised contrastive learning; Step 4, constructing a feature extractor with the same structure as the feature extraction module, transferring the parameters from the pre-trained feature extraction module to the feature extractor; constructing and training a simplified metastasis prediction network; Step 5, training the lymph node metastasis prediction model; Step 6, real-time metastasis prediction; acquiring the abdominal CT image to be predicted and inputting it into the lymph node metastasis prediction model trained in Step 5, the lymph node metastasis prediction model outputs a prediction result indicating that the corresponding lymph node is positive.

[0007] Similar to the aforementioned invention patents, most existing technologies impose contrast constraints at the query feature level, lacking semantic alignment and feature distribution constraints at the anatomical environment level, making it difficult to form robust feature separation between highly similar structures. Furthermore, during the verification phase, they exclude lymph node samples with a diameter less than 5mm from true positives and false positives, resulting in low model resolution for small, low-contrast lymph node features and low accuracy in lymph node identification and detection. Summary of the Invention

[0008] The purpose of this invention is to address the technical problems in the prior art, namely, the low ability to distinguish small and low-contrast lymph node features and the failure to fully utilize the anatomical environment information around the lymph nodes, which leads to low accuracy in lymph node identification and detection. This invention provides a lymph node detection method, system, device, and medium based on anatomical environment perception.

[0009] To achieve the above objectives, the present invention specifically adopts the following technical solution: A lymph node detection method based on anatomical environment awareness includes the following steps: Step 1: Obtain sample and label data; Acquire CT sample images and label the lymph nodes in the CT sample images to obtain label data; Step 2: Construct a lymph node detection model; The lymph node detection model comprises, in sequence, a feature extraction network, a central slice-guided cross-slice attention submodule, a 3D channel and spatial saliency enhancement submodule, a hybrid encoder and Transformer decoder submodule, and a detection head submodule. CT images are input to the feature extraction network, which outputs a set of feature maps. These feature map sets are then input to the central slice-guided cross-slice attention submodule, which outputs context-enhanced feature maps. Context-enhanced feature maps Input the 3D channel and spatial saliency enhancement submodule and output the final enhanced features. Ultimately enhance features The input is the hybrid encoder and Transformer decoder submodule and the output is the object query. The object query is input to the detection head submodule and the output is the predicted class confidence and bounding box position. Step 3: Train the lymph node detection model; The lymph node detection model constructed in step 2 is trained using the sample data and label data obtained in step 1. During training, an auxiliary anatomical environment perception module is used to constrain prototype consistency and difficult negative samples. The auxiliary anatomical environment perception module includes a ROI-level dual-view data augmentation submodule, a prototype consistency constraint submodule, and a difficult negative sample comparison submodule. The feature map output by the feature extraction network at a downsampling step size of 8 is input into the ROI-level dual-view data augmentation submodule and outputs a positive sample view. The prototype consistency constraint submodule is used to constrain the prototype consistency of the output positive sample view, and the difficult negative sample comparison submodule is used to constrain the difficult negative samples output by the detection head submodule. Step 4, real-time lymph node detection; The CT image to be detected is acquired and input into the lymph node detection model, which then outputs the prediction results.

[0010] Furthermore, in step 2, the cross-slice attention submodule guided by the central slice includes a projection layer, a cross attention layer, and a 1*1 convolutional layer; The cross-slice attention submodule, guided by the input center slice of the feature map set, projects the feature maps of the corresponding levels in the feature map set through a projection layer. Pre-slice feature map Current slice feature map and subsequent slice feature maps The mapping is performed using three vectors: query, key, and value. An attention weight matrix A is obtained through a cross-attention layer and a softmax function, and contextual features are obtained through a 1x1 convolutional layer. Contextual features With the current slice feature map The feature maps are fused to obtain context-enhanced feature maps. .

[0011] Furthermore, in step 2, the three-dimensional channel and spatial saliency enhancement submodule includes three processing branches with the same structure, as well as an averaging operation unit and a spatial saliency calculation unit. Each processing branch includes a convolutional layer, a double pooling layer and a depthwise separable convolutional layer. Context-enhanced feature maps The data is fed into three processing branches in parallel; in each processing branch, context-enhanced feature maps are processed. First, a convolutional layer is used to perform dimensionality permutation to obtain the feature perspective. Then, a double pooling layer is used for global average pooling and global max pooling. Finally, attention weights are generated through a depthwise separable convolutional layer and a sigmoid function. Attention weights of the outputs of the three processing branches , , After element-wise multiplication, the input is fed into the averaging unit for averaging to obtain the channel attention weights. Channel attention weights Applied to context-enhanced feature maps To obtain intermediate feature maps Intermediate feature map The data is fed into the spatial saliency calculation unit to obtain the final enhanced features. .

[0012] Furthermore, intermediate feature maps The data is fed into the spatial saliency calculation unit and the final enhanced features are obtained. The specific process is as follows: Calculate intermediate feature maps Energy value at each spatial location t The energy diagram E is obtained; Then, invert the energy map E and apply the sigmoid function to generate the spatial attention map. ; Using spatial attention maps For intermediate feature maps Modulation is performed to obtain the final enhanced features. .

[0013] Furthermore, in step 3, for the pair of positive sample views output by the ROI-level dual-view data augmentation submodule ( , For each view in the model, the prototype consistency constraint submodule calculates the assignment probability of that view to all prototypes and forms an assignment code. And each view is used to predict the optimal allocation code for the other view; When applying prototype consistency constraints to the output positive sample view using the prototype consistency constraint submodule, the prototype consistency loss is... Represented as: ; ; ; in, Indicates the use of vision Figure 1 Features Defitting the view Figure 2 Prototype allocation , Indicates the use of vision Figure 2 Features Defitting the view Figure 1 Prototype allocation , This represents the soft-assigned weight of the target view on the k-th prototype. This represents the probability that view z is assigned to the k-th prototype. The temperature coefficient represents the smoothness of the similarity probability distribution. Indicates transpose. Let represent the k-th learnable prototype vector. This represents the j-th prototype vector in the same group as the current sample in the summation operation.

[0014] Furthermore, in step 3, the difficult negative samples output by the detection head submodule are constrained using the difficult negative sample comparison submodule, specifically as follows: From the predicted class confidence scores output by the detection head submodule, candidate boxes with predicted class confidence scores higher than 0.5 and that do not match any true labels are selected as hard negative samples. For each ROI-level dual-view data augmentation submodule, a pair of positive sample views are output. , ), and a set of feature vectors containing R difficult negative samples mined in this batch. Calculate the contrast loss based on infoNCE; contrast loss Represented as: ; in, Represents cosine similarity. This represents the temperature coefficient used in comparative learning.

[0015] Furthermore, in step 3, when training the lymph node detection model, the global loss... for: ; ; ; ; ; ; ; in, This indicates the loss incurred by those responsible for overseeing the location and classification of targets. Indicates the weight of the auxiliary total loss. Indicates the total auxiliary loss; Represents the classification loss weights. Represents classification loss, Indicates the regression loss weights. Indicates regression loss; This represents the IoU value of the predicted bounding box that matches the ground truth bounding box. This indicates the confidence level at which the candidate region is identified as a lymph node. This represents the focusing factor that controls the weights of high-scoring negative samples; Indicates the prediction box. Represents the true bounding box. Tables are used for balancing The weight of the loss This indicates the relationship between the predicted bounding box and the ground truth bounding box in normalized coordinates. distance, This represents the weights used to balance the GIoU loss. This represents the loss based on the intersection-union ratio; This represents the x-coordinate of the center point of the prediction box. This represents the x-coordinate of the center point of the true bounding box. This represents the ordinate of the center point of the prediction box. This represents the x-coordinate of the center point of the true bounding box. Indicates the width of the prediction box. This represents the width of the actual bounding box. Indicates the height of the predicted bounding box. Indicates the height of the actual bounding box; This represents the prototype consistency loss. Indicates the weights of the comparative loss. This indicates a comparative loss.

[0016] A lymph node detection system based on anatomical environment awareness, comprising: The sample and label data acquisition module is used to acquire CT sample images and label the lymph nodes in the CT sample images to obtain label data. The lymph node detection model construction module is used to build a lymph node detection model. The lymph node detection model includes, in sequence, a feature extraction network, a central slice-guided cross-slice attention submodule, a 3D channel and spatial saliency enhancement submodule, a hybrid encoder and Transformer decoder submodule, and a detection head submodule. CT images are input to the feature extraction network and output a set of feature maps. The feature map set is then input to the central slice-guided cross-slice attention submodule and outputs a context-enhanced feature map. Context-enhanced feature maps Input the 3D channel and spatial saliency enhancement submodule and output the final enhanced features. Ultimately enhance features The input is the hybrid encoder and Transformer decoder submodule and the output is the object query. The object query is input to the detection head submodule and the output is the predicted class confidence and bounding box position. The lymph node detection model training module is used to train the lymph node detection model constructed by the lymph node detection model construction module using sample and label data acquired by the sample and label data acquisition module. During training, the auxiliary anatomical environment perception module is used to constrain prototype consistency and difficult negative samples. The auxiliary anatomical environment perception module includes a ROI-level dual-view data augmentation submodule, a prototype consistency constraint submodule, and a difficult negative sample comparison submodule. The feature map output by the feature extraction network at a downsampling step size of 8 is input into the ROI-level dual-view data augmentation submodule and outputs a positive sample view. The prototype consistency constraint submodule is used to constrain the prototype consistency of the output positive sample view, and the difficult negative sample comparison submodule is used to constrain the difficult negative samples output by the detection head submodule. The real-time lymph node detection module is used to acquire the CT image to be detected and input it into the lymph node detection model. The lymph node detection model outputs the prediction results.

[0017] A computer device includes a memory and a processor, the memory storing a computer program that, when executed by the processor, causes the processor to perform the steps of the method described above.

[0018] A computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the steps of the above-described method.

[0019] The beneficial effects of this invention are as follows: 1. In this invention, an auxiliary anatomical environment perception module is set up in addition to the main detection branch of the lymph node detection model. The two branches can be trained in parallel without additional annotation, which enhances the model's ability to distinguish lymph nodes from their complex anatomical environment. It can perform semantic alignment and feature distribution constraints at the anatomical environment level, form robust feature separation between highly similar structures, improve the model's ability to distinguish small, low-contrast lymph node features, and make the lymph node identification and detection more accurate.

[0020] 2. In this invention, the cross-slice attention mechanism driven by the center slice query of the cross-slice attention submodule guided by the center slice achieves adaptive weight allocation, which can explicitly model the correlation between slices while maintaining the resolution of details, and significantly improve the detection accuracy of small targets in complex anatomical regions.

[0021] 3. In this invention, the prototype consistency constraint submodule performs prototype alignment only for positive samples, and achieves intra-class semantic preservation through dynamic prototype memory update, which significantly improves the stability of feature aggregation under small sample conditions.

[0022] 4. In this invention, high-confidence false positive candidates predicted by the detection head are used as explicit negative sample sources. During the training process, the difficult negative sample comparison submodule is used to constrain the difficult negative samples output by the detection head submodule, so that the model can effectively identify "highly similar non-target" structures and reduce false positive detections in pelvic CT. Attached Figure Description

[0023] Figure 1 This is a flowchart illustrating the present invention; Figure 2 This is a schematic diagram of the lymph node detection model and the auxiliary anatomical environment perception module in this invention; Figure 3 This is a schematic diagram of the feature extraction network structure in this invention; Figure 4 This is a schematic diagram of the structure of the cross-slice attention submodule guided by the central slice in this invention; Figure 5 This is a schematic diagram of the structure of the three-dimensional channel and spatial saliency enhancement submodule in this invention; Figure 6 This is a schematic diagram of the interactive prediction of the prototype consistency constraint submodule in this invention. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are some embodiments of the present invention, but not all embodiments.

[0025] Therefore, all other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0026] Example 1 This embodiment provides a lymph node detection method based on anatomical environment awareness, used to identify and detect lymph nodes in locations such as the thoracic cavity and abdominal cavity. Figure 1 As shown, it includes the following steps: Step 1: Obtain sample and label data; Acquire CT sample images and label the lymph nodes in the CT sample images to obtain label data.

[0027] Several 3D CT sample images were obtained from West China Hospital of Sichuan University, and experienced professional physicians labeled the lymph nodes in the CT sample images to obtain label data.

[0028] The acquired CT sample images are preprocessed and labeled, and the specific methods are as follows: 1. Normalization process; The acquired 3D CT sample images were subjected to grayscale normalization. A window width of 300 HU and a window level of 40 HU were set to linearly map the Hounsfield Unit to a grayscale range of 0-255 to enhance the contrast between the target tissue and the surrounding structure. Subsequently, spatial normalization was performed. Through 3D resampling with voxel interpolation, all volume data were unified to a thickness of 2.0 mm and a planar resolution of 0.75 mm * 0.75 mm to ensure consistent spatial scale between samples.

[0029] 2. Noise removal treatment; A human contour segmentation algorithm is used to remove background noise from the scanning bed, and the volume data is cropped based on keyframes or anatomical localization information, concentrating computational resources on the effective area where the lesion is located. To preserve 3D contextual information, the cropped data blocks are sliced ​​layer by layer along the Z-axis, and each slice is stacked with its two adjacent frames to form a three-channel pseudo-color image, which is then fused with local spatial features.

[0030] Lymph node annotation was performed on CT sample images after noise removal. Annotation software was used, and experienced physicians navigated by moving the mouse to display a continuous 2D image along the Z-axis (i.e., a two-dimensional slice). They then identified the frame with the largest lymph node slice and annotated the lymph node within that frame. The specific data used for annotation included the coordinates (x, y, z) of the lymph node's center point and its diameter, which were then processed into a 2D bounding box representing the lymph node.

[0031] The final output is a structured dataset conforming to the COCO format, containing unified multi-channel 2D CT image samples and their corresponding bounding box annotations, providing high-quality training samples for subsequent detection models.

[0032] 3. Data augmentation processing; A multi-channel data loading and enhancement mechanism is introduced to perform data enhancement processing on noise-processed 2D CT images, thereby enhancing the spatial continuity of features in CT images and the model's generalization ability. Specifically, this is achieved through an extended CTCocoDetection dataset class. This class, based on the loaded 2D slices and bounding box annotations, establishes the correspondence between slices and the original 3D sequence according to the CT image mapping file. During data loading, multiple adjacent frames of images before and after each central slice are dynamically extracted to form a multi-slice input group containing 9 frames of CT information, thus introducing richer local 3D contextual features.

[0033] Step 2: Construct a lymph node detection model; The lymph node detection model comprises, in sequence, a feature extraction network, a central slice-guided cross-slice attention submodule, a 3D channel and spatial saliency enhancement submodule, a hybrid encoder and Transformer decoder submodule, and a detection head submodule. CT images are input to the feature extraction network, which outputs a set of feature maps. These feature map sets are then input to the central slice-guided cross-slice attention submodule, which outputs context-enhanced feature maps. Context-enhanced feature maps Input the 3D channel and spatial saliency enhancement submodule and output the final enhanced features. Ultimately enhance features The input is the hybrid encoder and Transformer decoder submodules, and the output is the object query. The object query is input to the detection head submodule and the output is the predicted class confidence and bounding box position.

[0034] The main detection branch (corresponding to the subsequent auxiliary anatomical environment perception branch) is composed of a feature extraction network, a central slice-guided cross-slice attention submodule, a 3D channel and spatial saliency enhancement submodule, a hybrid encoder and Transformer decoder submodule, and a detection head submodule. Figure 2 As shown.

[0035] The feature extraction network consists of three backbone networks arranged in parallel, each a ResNet50 network, and the three ResNet50 networks share parameters. For example... Figure 3As shown, the multi-slice input containing 9 frames of CT information is divided into three groups according to adjacent order (the first group is Z-4, Z-3, Z-2; the second group is Z-1, Z, Z+1; the third group is Z+2, Z+3, Z+4), and each group is input into the corresponding backbone network for hierarchical feature extraction (the multi-slice input of the first group is input into the first backbone network, the multi-slice input of the second group is input into the second backbone network, and the multi-slice input of the third group is input into the third backbone network). The three backbone networks output the preceding feature maps respectively. Current feature map and subsequent feature maps This process aims to transform the input pixel-level information into a set of multi-scale feature maps with rich semantics. ,in The feature map representing the i-th level. Indicates batch size. Indicates channel dimension, , This represents the size of the input image. This set of feature maps provides a comprehensive feature representation for subsequent processing, from low-level texture to high-level semantics.

[0036] For the center-slice-guided cross-slice attention submodule, this submodule includes a projection layer, a cross-attention layer, and a 1*1 convolutional layer, such as... Figure 4 As shown. The cross-slice attention submodule, guided by the input center slice, uses a projection layer to process the feature maps of the corresponding levels in the feature map set. Pre-slice feature map Current slice feature map and subsequent slice feature maps The mapping is represented by three vectors: query, key, and value. ; ; ; in, , , This represents the learnable weight matrix. This indicates a splicing operation along the channel dimension.

[0037] The attention weight matrix A is obtained through a cross-attention layer and a softmax function. This matrix A is calculated by multiplying the query and the key, and then normalized using the softmax function to quantify the dependence of each pixel position in the central slice on positions in neighboring slices. The attention weight matrix A is expressed as: ; in, This represents the dimension of the key vector, used for scaling to prevent gradient vanishing.

[0038] Then, a 1x1 convolutional layer is used to apply the attention weight matrix A to the value vector, resulting in weighted aggregated context features. Contextual features Represented as: ; Finally, the aggregated context features By using residual connections and other methods, it is integrated with the current slice feature map. The features are fused to generate the final context-enhanced feature map. : .

[0039] The cross-slice attention submodule guided by the central slice, through the aforementioned mechanism, can adaptively focus on the most informative regions in adjacent slices, thereby effectively enhancing the three-dimensional spatial continuity of the features.

[0040] For the 3D channel and spatial saliency enhancement submodule, it includes three structurally identical processing branches, an averaging operation unit (Avg), and a spatial saliency calculation unit. Each processing branch includes a convolutional layer, a double pooling layer, and a depthwise separable convolutional layer, such as... Figure 5 As shown.

[0041] The fused context-enhanced feature map is obtained from the cross-slice attention submodule guided by the central slice. Subsequently, to further enhance its representational capabilities, a cascaded dual attention enhancement strategy was adopted in the 3D channel and spatial saliency enhancement submodules. Specifically: First, a three-channel attention fusion mechanism is used to learn the complex dependencies between channels. Context-enhanced feature maps. The data are fed in parallel into three structurally identical processing branches, each learning from three orthogonal dimensional perspectives; within each processing branch, context-enhanced feature maps are generated. First, a 1x1 convolutional layer is used to perform dimension permutation, generating and obtaining three different feature perspectives. Specifically, this is represented as follows: .

[0042] Subsequently, global average pooling and global max pooling are performed in parallel through a double pooling layer to capture the average and peak responses of the channel. The global average pooling is expressed as follows: ; Next, attention weights in the range [0,1] are generated using depthwise separable convolutional layers and a sigmoid function. ;Right now: ; The attention weights of the outputs from the three processing branches , , After element-wise multiplication, the weights are input into an averaging unit for averaging, resulting in a unified channel attention weight. ;Right now: ; Channel attention weights Broadcast multiplication is applied to context-enhanced feature maps. The intermediate feature map obtained after channel refinement is obtained. ;Right now: ; After completing the semantic enhancement at the channel dimension, the intermediate feature map The data are sequentially fed into the spatial saliency calculation unit and fed into the spatial attention map. Modulation is performed to obtain the final enhanced features. .

[0043] intermediate feature map The data is fed into the spatial saliency calculation unit and the final enhanced features are obtained. The specific process is as follows: An energy function is defined for each position t in the feature map, and its importance is evaluated using this energy function. The closed-form solution of this energy function can be efficiently computed as follows: ; in, , Let represent the mean and variance of all neurons in this channel except for neuron t, respectively. This represents a regularization hyperparameter. The lower the energy value, the higher the distinction between the location and the surrounding area, and the greater its importance.

[0044] Calculate intermediate feature maps Energy value at each spatial location t To obtain the energy map .in, Essentially, it characterizes the degree of deviation of position t relative to other positions in the same channel after the mean square error is normalized, that is, the "prominence" of the position in the channel.

[0045] Then invert the energy diagram E and apply the Sigmoid function, i.e. Generate a dimension of Spatial attention map ; Finally, spatial attention maps are used. For intermediate feature maps Modulation is performed to obtain the final enhanced features. ; indicates as: .

[0046] The hybrid encoder and Transformer decoder submodule employs a Transformer-based detector architecture to process the enhanced multi-scale features. The hybrid encoder receives attention-enhanced features from multiple levels. As input, the encoder internally integrates local and global contextual information from feature maps at different scales through alternating convolutional operations and self-attention mechanisms, generating a refined set of semantically rich in-memory features. The Transformer decoder receives the in-memory features output by the encoder and a set of learnable, fixed-number object queries as input. In each decoder layer, the object queries first model their relationships through a self-attention mechanism, and then interact with the in-memory features output by the encoder through a cross-attention mechanism. This process is iterative, allowing each object query to gradually focus on a potential target in the image and absorb its feature information.

[0047] Based on the above description, conventional technical means in the field, and related existing technologies, those skilled in the art can implement the above-described hybrid encoder and Transformer decoder submodules without any creative effort.

[0048] For the detection head submodule, the optimized object query is processed through two parameter-sharing feedforward neural networks: a classification branch predicts the target category confidence, and a regression branch predicts the bounding box location information. Specifically, for predicted samples with high confidence but confirmed as false positives, their features and location information are extracted and fed back to an auxiliary contrastive learning branch. This guides the model to more accurately distinguish between real lymph nodes and highly similar artifacts or normal tissue, thereby reducing the false detection rate.

[0049] Based on the above description, conventional technical means in the field, and related existing technologies, those skilled in the art can implement the above-described detection head sub-module without any creative effort.

[0050] Step 3: Train the lymph node detection model; The lymph node detection model constructed in step 2 is trained using the sample and label data obtained in step 1. During training, an auxiliary anatomical environment perception module is used to constrain prototype consistency and difficult negative samples. The auxiliary anatomical environment perception module includes a ROI-level dual-view data augmentation submodule, a prototype consistency constraint submodule, and a difficult negative sample comparison submodule. The feature map output by the feature extraction network at a downsampling step size of 8 is input into the ROI-level dual-view data augmentation submodule and outputs a positive sample view. The prototype consistency constraint submodule is used to constrain the prototype consistency of the output positive sample view, and the difficult negative sample comparison submodule is used to constrain the difficult negative samples output by the detection head submodule.

[0051] In this embodiment, to enhance the model's ability to distinguish lymph nodes and their complex anatomical environments without requiring additional annotation, an auxiliary anatomical environment perception module is set up. This auxiliary anatomical environment perception module is trained in parallel with the main detection branch, and its loss function is integrated into the global total loss to guide the main detection branch in generating discriminative feature representations.

[0052] This auxiliary anatomical environment perception module reuses the feature extraction network from the main detection branch. It directly utilizes the feature map output by the feature extraction network at a downsampling step size of 8, and the channel dimension of this feature map is 512. Therefore, this auxiliary branch can perform deep optimization on the features that have already undergone preliminary semantic extraction without increasing the computational overhead of the additional backbone network (i.e., the feature extraction network).

[0053] For the ROI-level dual-view data augmentation submodule, to construct positive sample pairs for contrastive learning, this module performs dual-view data augmentation on each real lymph node region of interest (ROI) in the training batch. This process first locates the center point of the lymph node on the feature map and dynamically extracts a rectangular feature block with sides ranging from 4 to 8 pixels based on its original size; then, by applying two sets of independent random data augmentations to this feature block, a pair of related yet different positive sample views are generated. , Specific enhancement operations include applying a random translation perturbation of up to 1.5 pixels to the center of the feature block and scaling within ±20%. Through this series of operations, each lymph node instance is ultimately transformed into a pair of slightly perturbed positive sample views in the feature space, providing crucial input for subsequent contrastive learning tasks.

[0054] For the prototype consistency constraint submodule, this module employs an exchangeable and assignable multi-view mechanism, such as... Figure 6 As shown, a prototype consistency constraint is applied to the positive sample pairs generated by the ROI-level dual-view data augmentation submodule. Specifically: First, a projection head consisting of two layers of multilayer perceptron (MLP), layer normalization (LayerNorm), and rectified linear unit (ReLU) maps the 512-dimensional view features z to a 128-dimensional unit hypersphere space, maintaining a set of 64 learnable prototype vectors. Each prototype It represents a cluster center in the feature space.

[0055] For each view feature z, this module calculates its assignment probability to all prototypes, forming an assignment code. Here This does not refer to a rigid optimal allocation, but rather to the soft allocation weights of sample features on the k-th prototype. This allocation encoding is iteratively optimized three times using the Sinkhorn-Knopp algorithm to ensure that all samples are distributed as evenly as possible across prototypes, preventing the model from collapsing to use only a few prototypes.

[0056] The core idea of ​​this module is "exchange," which utilizes the features of the first view. To predict the optimal allocation encoding for the second view. Conversely, utilize the features of the second view. To predict the optimal allocation encoding for the first view (Optimal allocation encoding, i.e., prototype allocation). When applying prototype consistency constraints to the output positive sample view using the prototype consistency constraint submodule, the prototype consistency loss is defined as the sum of the cross-entropy losses in both directions. This prototype consistency loss... Represented as: ; ; ; in, Indicates the use of vision Figure 1 Features Defitting the view Figure 2 Prototype allocation , Indicates the use of vision Figure 2 Features Defitting the view Figure 1 Prototype allocation ; This represents the k-th element of the allocation code q, i.e., the target view (e.g.) The model soft-assigns weights on the k-th prototype and predicts the "target assignment" in the exchange prediction; to predict this distribution, the model calculates... , Represents the view z (e.g.) The probability assigned to the k-th prototype; this probability is expressed by a temperature coefficient. The result is obtained from the softmax function; The temperature coefficient represents the smoothness of the similarity probability distribution, and in this embodiment, it is set to 0.25. This indicates transpose; the core calculation of this module is... , Let represent the k-th learnable prototype vector, which serves as the cluster center in the feature space and is used to calculate the similarity with feature z; finally, to ensure It is a valid probability distribution. The denominator of the softmax function iterates through all prototype vectors by summing the index j. Normalize, This represents the j-th prototype vector in the same group as the current sample in the summation operation.

[0057] For the difficult negative sample contrast submodule, to further improve the model's ability to distinguish highly similar tissues, this module introduces an explicit contrastive learning mechanism, using difficult negative samples generated by the main detection branch as constraints. Specifically: This module selects candidate boxes with a prediction confidence higher than 0.5 but which do not match any real labels from the output of the detection head as hard negative samples.

[0058] For each ROI-level dual-view data augmentation submodule, a pair of positive sample views are output ( , For each positive sample feature vector The corresponding sample feature vector of another enhanced view And a set of feature vectors containing R difficult negative samples mined in this batch. Calculate the contrastive loss based on infoNCE. This loss aims to bring positive sample pairs closer together. , The distance in the feature space, while pushing the sample further away. With all negative samples The distance between them. Therefore, the contrast loss. Represented as: ; in, Indicates cosine similarity; The temperature coefficient for contrastive learning is 0.07; R represents the number of difficult negative samples selected in each training batch, with a value of 16.

[0059] When training the lymph node detection model, the weights of the entire model are updated synchronously by constructing and minimizing a unified, multi-task global loss. Global Loss for: ; ; ; ; ; ; ; in, This indicates the loss incurred by those responsible for overseeing the location and classification of targets. Indicates the weight of the auxiliary total loss. Indicates the total auxiliary loss; Indicates the classification loss weights; This represents the classification loss, which employs variable focusing loss and is better able to handle the problems of imbalance between positive and negative samples and varying sample difficulty. Indicates the regression loss weights. Indicates regression loss; This represents the IoU value of the predicted bounding box that matches the ground truth bounding box. This indicates the confidence level that the candidate region is identified as a lymph node. Specifically, for a positive sample that successfully matches a ground truth bounding box, its... The value is set to the intersection-union ratio with the corresponding ground truth bounding box, and is therefore a floating-point number between (0,1). For negative samples that do not match any ground truth bounding box, its... The value is then 0; This indicates the weight used to adjust the contribution of positive and negative samples to the total loss, with a value of 0.75. This represents the focus factor that controls the weight of high-scoring negative samples. It is used to reduce the loss weight of easily classified samples, so that the model training focuses more on learning difficult negative samples that are hard to distinguish. The value is 2.0. Indicates the prediction box. Represents a real bounding box; Tables are used for balancing The weight of the loss is set to 5. This indicates the relationship between the predicted bounding box and the ground truth bounding box in normalized coordinates. distance; This represents the weight used to balance the GIoU loss, with a value of 2; This represents the loss based on the intersection-union ratio; This represents the x-coordinate of the center point of the prediction box. This represents the x-coordinate of the center point of the true bounding box. This represents the ordinate of the center point of the prediction box. This represents the x-coordinate of the center point of the true bounding box. Indicates the width of the prediction box. This represents the width of the actual bounding box. Indicates the height of the predicted bounding box. Indicates the height of the actual bounding box; This represents the prototype consistency loss. Indicates the weights of the comparative loss. This indicates a comparative loss.

[0060] Step 4, real-time lymph node detection; The CT image to be detected is acquired and input into the lymph node detection model. The CT image to be detected is processed sequentially through the feature extraction network of the lymph node detection model, the cross-slice attention submodule guided by the central slice, the three-dimensional channel and spatial saliency enhancement submodule, the hybrid encoder and Transformer decoder submodule, and the detection head submodule. Finally, the detection head submodule of the lymph node detection model outputs the prediction result of the lymph node.

[0061] Example 2 This embodiment provides a lymph node detection system based on anatomical environment awareness, including: The sample and label data acquisition module is used to acquire CT sample images and label the lymph nodes in the CT sample images to obtain label data. The lymph node detection model construction module is used to build a lymph node detection model. The lymph node detection model includes, in sequence, a feature extraction network, a central slice-guided cross-slice attention submodule, a 3D channel and spatial saliency enhancement submodule, a hybrid encoder and Transformer decoder submodule, and a detection head submodule. CT images are input to the feature extraction network and output a set of feature maps. The feature map set is then input to the central slice-guided cross-slice attention submodule and outputs a context-enhanced feature map. Context-enhanced feature maps Input the 3D channel and spatial saliency enhancement submodule and output the final enhanced features. Ultimately enhance features The input is the hybrid encoder and Transformer decoder submodule and the output is the object query. The object query is input to the detection head submodule and the output is the predicted class confidence and bounding box position. The lymph node detection model training module is used to train the lymph node detection model constructed by the lymph node detection model construction module using sample and label data acquired by the sample and label data acquisition module. During training, the auxiliary anatomical environment perception module is used to constrain prototype consistency and difficult negative samples. The auxiliary anatomical environment perception module includes a ROI-level dual-view data augmentation submodule, a prototype consistency constraint submodule, and a difficult negative sample comparison submodule. The feature map output by the feature extraction network at a downsampling step size of 8 is input into the ROI-level dual-view data augmentation submodule and outputs a positive sample view. The prototype consistency constraint submodule is used to constrain the prototype consistency of the output positive sample view, and the difficult negative sample comparison submodule is used to constrain the difficult negative samples output by the detection head submodule. The real-time lymph node detection module is used to acquire the CT image to be detected and input it into the lymph node detection model. The lymph node detection model outputs the prediction results.

[0062] Example 3 A computer device includes a memory and a processor, the memory storing a computer program that, when executed by the processor, causes the processor to perform steps of a lymph node detection method based on anatomical environment awareness.

[0063] The computer device can be a desktop computer, laptop, handheld computer, or cloud server, etc. The computer device can interact with the user via a keyboard, mouse, remote control, touchpad, or voice control.

[0064] The memory includes at least one type of readable storage medium, including flash memory, hard disk, multimedia card, card-type memory (e.g., SD or D-interface display memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the memory may be an internal storage unit of the computer device, such as the hard disk or memory of the computer device. In other embodiments, the memory may also be an external storage device of the computer device, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the computer device. Of course, the memory may include both internal storage units and external storage devices of the computer device. In this embodiment, the memory is often used to store the operating system and various application software installed on the computer device, such as the program code of the lymph node detection method based on anatomical environment awareness. In addition, the memory can also be used to temporarily store various types of data that have been output or will be output.

[0065] In some embodiments, the processor may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip. The processor is typically used to control the overall operation of the computer device. In this embodiment, the processor is used to run program code stored in the memory or process data, for example, to run the program code of the anatomically aware lymph node detection method.

[0066] Example 4 A computer-readable storage medium storing a computer program that, when executed by a processor, causes the processor to perform steps of a lymph node detection method based on anatomical environment awareness.

[0067] The computer-readable storage medium stores an interface display program that can be executed by at least one processor to cause the at least one processor to perform the steps of the anatomical environment-aware lymph node detection method described above.

[0068] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the lymph node detection method based on anatomical environment perception described in the embodiments of this application.

Claims

1. A lymph node detection method based on anatomical environment perception, characterized in that, Includes the following steps: Step 1: Obtain sample and label data; Acquire CT sample images and label the lymph nodes in the CT sample images to obtain label data; Step 2: Construct a lymph node detection model; The lymph node detection model comprises, in sequence, a feature extraction network, a central slice-guided cross-slice attention submodule, a 3D channel and spatial saliency enhancement submodule, a hybrid encoder and Transformer decoder submodule, and a detection head submodule. CT images are input to the feature extraction network, which outputs a set of feature maps. These feature map sets are then input to the central slice-guided cross-slice attention submodule, which outputs context-enhanced feature maps. Context-enhanced feature maps Input the 3D channel and spatial saliency enhancement submodule and output the final enhanced features. Ultimately enhance features The input is the hybrid encoder and Transformer decoder submodule and the output is the object query. The object query is input to the detection head submodule and the output is the predicted class confidence and bounding box position. Step 3: Train the lymph node detection model; The lymph node detection model constructed in step 2 is trained using the sample data and label data obtained in step 1. During training, an auxiliary anatomical environment perception module is used to constrain prototype consistency and difficult negative samples. The auxiliary anatomical environment perception module includes a ROI-level dual-view data augmentation submodule, a prototype consistency constraint submodule, and a difficult negative sample comparison submodule. The feature map output by the feature extraction network at a downsampling step size of 8 is input into the ROI-level dual-view data augmentation submodule and outputs a positive sample view. The prototype consistency constraint submodule is used to constrain the prototype consistency of the output positive sample view, and the difficult negative sample comparison submodule is used to constrain the difficult negative samples output by the detection head submodule. Step 4, real-time lymph node detection; The CT image to be detected is acquired and input into the lymph node detection model, which then outputs the prediction results.

2. The lymph node detection method based on anatomical environment perception as described in claim 1, characterized in that, In step 2, the cross-slice attention submodule guided by the central slice includes a projection layer, a cross attention layer, and a 1*1 convolutional layer; The cross-slice attention submodule, guided by the input center slice of the feature map set, projects the feature maps of the corresponding levels in the feature map set through a projection layer. Pre-slice feature map Current slice feature map and subsequent slice feature maps The mapping is performed using three vectors: query, key, and value. An attention weight matrix A is obtained through a cross-attention layer and a softmax function, and contextual features are obtained through a 1x1 convolutional layer. ; Contextual features With the current slice feature map The feature maps are fused to obtain context-enhanced feature maps. .

3. The lymph node detection method based on anatomical environment perception as described in claim 1, characterized in that, In step 2, the 3D channel and spatial saliency enhancement submodule includes three structurally identical processing branches, as well as an averaging operation unit and a spatial saliency calculation unit. Each processing branch includes a convolutional layer, a double pooling layer, and a depthwise separable convolutional layer. Context-enhanced feature maps Three processing branches are fed in parallel; In each processing branch, context-enhanced feature maps First, a convolutional layer is used to perform dimensionality permutation to obtain the feature perspective. Then, a double pooling layer is used for global average pooling and global max pooling. Finally, attention weights are generated through a depthwise separable convolutional layer and a sigmoid function. ; Attention weights of the outputs of the three processing branches , , After element-wise multiplication, the input is fed into the averaging unit for averaging to obtain the channel attention weights. Channel attention weights Applied to context-enhanced feature maps To obtain intermediate feature maps Intermediate feature map The data is fed into the spatial saliency calculation unit to obtain the final enhanced features. .

4. The lymph node detection method based on anatomical environment perception as described in claim 3, characterized in that, intermediate feature map The data is fed into the spatial saliency calculation unit and the final enhanced features are obtained. The specific process is as follows: Calculate intermediate feature maps Energy value at each spatial location t The energy diagram E is obtained; Then, invert the energy map E and apply the sigmoid function to generate the spatial attention map. ; Using spatial attention maps For intermediate feature maps Modulation is performed to obtain the final enhanced features. .

5. The lymph node detection method based on anatomical environment perception as described in claim 1, characterized in that, In step 3, for the pair of positive sample views output by the ROI-level dual-view data augmentation submodule ( , For each view in the model, the prototype consistency constraint submodule calculates the assignment probability of that view to all prototypes and forms an assignment code. And each view is used to predict the optimal allocation code for the other view; When applying prototype consistency constraints to the output positive sample view using the prototype consistency constraint submodule, the prototype consistency loss is... Represented as: ； ； ； in, Indicates the features of view 1 To fit the prototype assignment of view 2 , Indicates the features of view 2 To fit the prototype assignment of view 1 , This represents the soft-assigned weight of the target view on the k-th prototype. This represents the probability that view z is assigned to the k-th prototype. The temperature coefficient represents the smoothness of the similarity probability distribution. Indicates transpose. Let represent the k-th learnable prototype vector. This represents the j-th prototype vector in the same group as the current sample in the summation operation.

6. The lymph node detection method based on anatomical environment perception as described in claim 1, characterized in that, In step 3, the difficult negative samples output by the detection head submodule are constrained using the difficult negative sample comparison submodule, specifically as follows: From the predicted class confidence scores output by the detection head submodule, candidate boxes with predicted class confidence scores higher than 0.5 and that do not match any true labels are selected as hard negative samples. For each ROI-level dual-view data augmentation submodule, a pair of positive sample views are output. , ), and a set of feature vectors containing R difficult negative samples mined in this batch. Calculate the contrast loss based on infoNCE; contrast loss Represented as: ； in, Represents cosine similarity. This represents the temperature coefficient used in comparative learning.

7. The lymph node detection method based on anatomical environment perception as described in claim 1, characterized in that, In step 3, when training the lymph node detection model, the global loss... for: ； ； ； ； ； ； ； in, This indicates the loss incurred by those responsible for overseeing the location and classification of targets. Indicates the weight of the auxiliary total loss. Indicates the total auxiliary loss; Represents the classification loss weights. Represents classification loss, Indicates the regression loss weights. Indicates regression loss; This represents the IoU value of the predicted bounding box that matches the ground truth bounding box. This indicates the confidence level at which the candidate region is identified as a lymph node. This indicates the weights used to adjust the contribution of positive and negative samples to the total loss. This represents the focusing factor that controls the weights of high-scoring negative samples; Indicates the prediction box. Represents the true bounding box. Tables are used for balancing The weight of the loss This indicates the relationship between the predicted bounding box and the ground truth bounding box in normalized coordinates. distance, This represents the weights used to balance the GIoU loss. This represents the loss based on the intersection-union ratio; This represents the x-coordinate of the center point of the prediction box. This represents the x-coordinate of the center point of the true bounding box. This represents the ordinate of the center point of the prediction box. This represents the x-coordinate of the center point of the true bounding box. Indicates the width of the prediction box. This represents the width of the actual bounding box. Indicates the height of the predicted bounding box. Indicates the height of the actual bounding box; This represents the prototype consistency loss. Indicates the weights of the comparative loss. This indicates a comparative loss.

8. A lymph node detection system based on anatomical environment perception, characterized in that, include: The sample and label data acquisition module is used to acquire CT sample images and label the lymph nodes in the CT sample images to obtain label data. The lymph node detection model construction module is used to build a lymph node detection model. The lymph node detection model includes, in sequence, a feature extraction network, a central slice-guided cross-slice attention submodule, a 3D channel and spatial saliency enhancement submodule, a hybrid encoder and Transformer decoder submodule, and a detection head submodule. CT images are input to the feature extraction network and output a set of feature maps. The feature map set is then input to the central slice-guided cross-slice attention submodule and outputs a context-enhanced feature map. Context-enhanced feature maps Input the 3D channel and spatial saliency enhancement submodule and output the final enhanced features. Ultimately enhance features The input is the hybrid encoder and Transformer decoder submodule and the output is the object query. The object query is input to the detection head submodule and the output is the predicted class confidence and bounding box position. The lymph node detection model training module is used to train the lymph node detection model constructed by the lymph node detection model construction module using sample and label data acquired by the sample and label data acquisition module. During training, the auxiliary anatomical environment perception module is used to constrain prototype consistency and difficult negative samples. The auxiliary anatomical environment perception module includes a ROI-level dual-view data augmentation submodule, a prototype consistency constraint submodule, and a difficult negative sample comparison submodule. The feature map output by the feature extraction network at a downsampling step size of 8 is input into the ROI-level dual-view data augmentation submodule and outputs a positive sample view. The prototype consistency constraint submodule is used to constrain the prototype consistency of the output positive sample view, and the difficult negative sample comparison submodule is used to constrain the difficult negative samples output by the detection head submodule. The real-time lymph node detection module is used to acquire the CT image to be detected and input it into the lymph node detection model. The lymph node detection model outputs the prediction results.

9. A computer device, characterized in that: It includes a memory and a processor, the memory storing a computer program that, when executed by the processor, causes the processor to perform the steps of the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The device stores a computer program that, when executed by a processor, causes the processor to perform the steps of the method as described in any one of claims 1 to 7.

Images