Deep learning-based endoscopic anatomical structure auxiliary detection and identification system and method

By utilizing the multi-scale axial attention mechanism of deep learning models and the YOLO11 architecture, the stability and cross-device adaptability issues of anatomical structure recognition in spinal endoscopic surgery are resolved. This enables high-precision recognition and real-time assistance for multiple types of anatomical structures, making it suitable for virtual teaching and intraoperative navigation in spinal endoscopic surgery.

CN121685496APending Publication Date: 2026-03-17JINING NO 1 PEOPLES HOSPITAL (JINING ACAD OF MEDICAL SCI) +1
View PDF 6 Cites 0 Cited by

Patent Information

Application Number
CN202511901193.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2025-12-16
Publication Date
2026-03-17

AI Technical Summary

Technical Problem

Existing technologies in spinal endoscopic surgery suffer from insufficient stability in anatomical structure recognition, poor cross-device generalization ability, and insufficient ability to identify multiple tissues simultaneously, making it difficult to achieve high-precision anatomical structure recognition, especially under complex conditions.

Method used

An endoscopic anatomical structure-assisted detection system based on deep learning was adopted. Through a deep learning model with a multi-scale axial attention mechanism, combined with the YOLO11 architecture, a dual-channel neck feature fusion layer and detection head were designed to achieve feature extraction, dynamic memorization and precise localization of multiple anatomical structures. Cosine annealing learning rate scheduling and early stopping strategy were used for model training. Multiple spinal endoscope devices were integrated for data acquisition and annotation.

Benefits of technology

It improves the accuracy and stability of anatomical structure recognition, maintains efficient recognition across different devices and patients, provides virtual teaching tools and real-time intraoperative assistance, and meets the needs of multi-tissue synchronous identification in spinal endoscopic surgery.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN121685496A_ABST
    Figure CN121685496A_ABST
Patent Text Reader

Abstract

The invention discloses an endoscopic anatomical structure auxiliary detection and recognition system and method based on deep learning, and relates to the technical field of medical image processing and deep learning cross. YOLO11 is used as a basis, a multi-scale axial attention mechanism is introduced, key anatomical structure feature capture is enhanced, background interference is suppressed, the recognition precision of various anatomical structures is improved, and the recognition efficiency is improved. The process of'feature extraction-dynamic memory-precise positioning 'is combined with a dual-channel fusion layer, continuous frame feature dynamic association can be realized, recognition continuity can be kept, over-fitting is avoided through a cosine annealing learning rate and an early stop strategy, the model generalization ability is guaranteed, and finally, precise positioning of a multi-scale structure is realized through a decoupling detection head, so that the accuracy of the multi-scale structure is improved. The system can be used as a virtual teaching tool to simulate anatomy identification training in a complex operation scene, and can also be integrated to endoscope equipment to realize real-time assistance in an operation, so that visual anatomy navigation is provided for an operator.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention belongs to the interdisciplinary field of medical image processing and deep learning, specifically relating to a deep learning-based endoscopic anatomical structure auxiliary detection and recognition system and method. Background Technology

[0002] Lumbar disc herniation, a common degenerative spinal disease, is showing an increasing incidence rate globally, particularly among young and middle-aged adults. Epidemiological surveys indicate that this disease can cause severe low back and leg pain, significantly impacting patients' quality of life, and in some cases, even leading to motor dysfunction due to nerve compression. Percutaneous endoscopic lumbar discectomy, with its advantages of minimal trauma, rapid recovery, and fewer complications, has become the preferred surgical procedure for treating single-segment lumbar disc herniation. However, this technique demands extremely high operator precision, as the complex and highly variable anatomical structures under endoscopic visualization result in a steep learning curve—studies show that beginners need to complete at least 50 surgeries to reach a proficient level, and the incidence of iatrogenic nerve injury due to unclear anatomical identification during this process is as high as 3.2%.

[0003] In recent years, deep learning technology has made breakthrough progress in the application of medical imaging, such as achieving clinical-grade performance in static image analysis tasks such as lung CT nodule detection and fundus image lesion recognition.

[0004] However, in the context of spinal endoscopic surgery, existing technologies still have significant limitations: On the one hand, traditional convolutional neural networks have difficulty capturing dynamically changing anatomical features in the surgical field of view, especially under complex conditions such as instrument obstruction and blurred bleeding, resulting in insufficient recognition stability. On the other hand, existing models mostly rely on data collected from a single device for training, which makes them less adaptable to imaging differences between different brands of endoscopic devices and limits their cross-device generalization ability.

[0005] In addition, most studies focus on the identification of single anatomical structures (such as nerve roots) and lack a systematic solution for the simultaneous identification of multiple tissues required for spinal endoscopic surgery, which is difficult to meet the clinical needs for comprehensive intraoperative assistance. Summary of the Invention

[0006] To address the problems mentioned in the background art, the present invention provides a deep learning-based endoscopic anatomical structure-assisted detection and recognition system and method to solve the problems of insufficient technical recognition stability, limited cross-device generalization ability, and lack of multi-tissue simultaneous identification capability.

[0007] To achieve the above objectives, the present invention provides the following technical solution: A deep learning-based endoscopic anatomical structure-assisted detection and recognition system includes: Data acquisition module; The data acquisition module includes multiple spinal endoscope devices. The data acquisition module is used to acquire video data of spinal endoscopic surgery and extract key frames from the video data at preset time intervals to obtain multiple valid images. Data preprocessing module; The data preprocessing module is connected to the data acquisition module. The data preprocessing module includes annotation tools. The data preprocessing module is used to perform anatomical structure annotation, quality control and image enhancement on valid images, and output standardized images to obtain a dataset. The model processing module is connected to the data preprocessing module. The model processing module contains a deep learning model with a multi-scale attention mechanism. The deep learning model is based on the YOLO11 architecture and includes a backbone network, a neck feature fusion layer, and a detection head. The backbone network adopts a multi-scale axial attention structure, and the neck feature fusion layer is designed as a dual-channel structure. The downsampling path and upsampling path of the dual-channel structure are equipped with multi-scale axial attention modules. The model processing module is used to train the deep learning model using the dataset to obtain the trained deep learning model. The detection and recognition module is connected to the model processing module. The detection and recognition module is used to input the image to be recognized into the trained deep learning model to realize the detection and recognition of multiple anatomical structures.

[0008] A deep learning-based method for assisted detection and identification of endoscopic anatomical structures includes the following steps: S1: Data collection and preprocessing; valid images are acquired through the data acquisition module, and after the valid images are input into the data preprocessing module, a dataset is obtained. The dataset is then divided into a training set and a validation set according to the proportions. S2: Model Training Phase; The model training phase includes feature extraction, dynamic memory, and precise localization. The feature extraction phase uses a backbone network and a multi-scale axial attention module to capture local edges, textures, and key region features of anatomical structures in the image, suppressing background interference. The dynamic memory phase relies on the dual-channel structure of the neck feature fusion layer and the multi-scale axial attention module to achieve feature fusion at different scales of the anatomical structure and dynamic correlation of features between consecutive frames, maintaining spatiotemporal consistency of features. The precise localization phase uses a decoupled detection head to complete the localization of bounding boxes, category determination, and mask generation of anatomical structures at different scales. During the model training phase, a cosine annealing learning rate scheduling strategy is adopted. The model is trained using the training set and preset learning rate, weight decay coefficient, and momentum. An early stopping strategy is also adopted. Finally, the deep learning model with the best performance on the validation set is selected as the final recognition model for testing. S3: Input the image to be identified into the final recognition model through the detection and recognition module to realize the detection and recognition of multiple anatomical structures.

[0009] Compared with the prior art, the beneficial effects of the present invention are: Based on YOLO11, this application introduces a multi-scale axial attention mechanism to enhance the capture of key anatomical features, suppress background interference, and improve the recognition accuracy of various anatomical structures. The "feature extraction-dynamic memory-precise localization" process of this application, combined with a dual-channel fusion layer, can realize the dynamic association of features in continuous frames, maintain the continuity of recognition, avoid overfitting through cosine annealing learning rate and early stopping strategy, and ensure the generalization ability of the model. Finally, the precise localization of multi-scale structures is achieved through decoupling the detection head. This application can be used as a virtual teaching tool to simulate anatomical identification training in complex surgical scenarios, and can also be integrated into endoscopic equipment to realize real-time intraoperative assistance and provide surgeons with visual anatomical navigation. Attached Figure Description

[0010] Figure 1 This is a flowchart illustrating the process of this application. Detailed Implementation

[0011] To facilitate understanding of the technical content of this invention by those skilled in the art, the invention will be further described in detail below with reference to the accompanying drawings and specific examples. It should be understood that the specific examples described herein are merely illustrative and not intended to limit the scope of the invention.

[0012] A deep learning-based endoscopic anatomical structure-assisted detection and recognition system includes: Data acquisition module; The data acquisition module includes multiple spinal endoscope devices. The data acquisition module is used to acquire video data of spinal endoscopic surgery and extract key frames from the video data at preset time intervals to obtain multiple valid images. Data preprocessing module; The data preprocessing module is connected to the data acquisition module. The data preprocessing module includes annotation tools. The data preprocessing module is used to perform anatomical structure annotation, quality control and image enhancement processing on valid images, and output standardized images to obtain a dataset. Anatomical structures include muscle structures, ligament structures, bone structures, nerve structures, intervertebral disc structures and herniated nucleus pulposus structures. The model processing module is connected to the data preprocessing module. It contains a deep learning model with a multi-scale attention mechanism, based on the YOLO11 architecture. The model includes a backbone network, a neck feature fusion layer, and a detection head. The backbone network employs a multi-scale axial attention structure, and the neck feature fusion layer is designed as a dual-channel structure. Multi-scale axial attention modules are added to both the downsampling and upsampling paths of the dual-channel structure. Furthermore, to address the issue of blurred tissue boundaries in spinal endoscopy images, the loss function of the Mask branch in the detection head is optimized, using a weighted hybrid loss of Dice coefficient and cross-entropy to improve segmentation accuracy. The model processing module is used to train the deep learning model using the dataset, resulting in a trained deep learning model. The detection and recognition module is connected to the model processing module. The detection and recognition module is used to input the image to be recognized into the trained deep learning model to realize the detection and recognition of multiple anatomical structures.

[0013] Deep learning-based methods for assisted detection and identification of endoscopic anatomical structures, such as Figure 1 As shown, it includes the following steps: Data collection and preprocessing; The data for this study came from 91 patients with lumbar disc herniation who underwent percutaneous endoscopic spinal surgery at a tertiary hospital. All cases were confirmed by preoperative MRI to be single-segment disc herniation, and patients with severe spinal stenosis, spinal deformities, or a history of previous surgery were excluded. Surgical videos were acquired using two different models of spinal endoscopes: each set consisted of three domestically produced high-definition endoscope systems (1920×1080 resolution, 60fps) from different manufacturers. Keyframes were extracted from the original surgical videos at 1-second intervals using a professional medical imaging workstation, resulting in a total of 58,667 valid images. Image annotation was performed using the Labelme open-source annotation tool, with two chief physicians with over 10 years of experience in spinal endoscopic surgery completing pixel-level annotations of six anatomical structures: muscles, ligaments, bones, nerves, intervertebral discs, and herniated nucleus pulposus.

[0014] The annotation process implements a three-level quality control: after the junior annotator completes the first draft, the senior physician conducts cross-checking, and confirms the inconsistent areas (such as the boundary between the intervertebral disc and the herniated nucleus pulposus) by reviewing the surgical video. Finally, the annotated dataset is reviewed and approved by a third-party imaging expert.

[0015] The data preprocessing stage employs a multi-dimensional enhancement strategy: spatial transformation (random rotation ±15°, scaling 0.8-1.2 times, horizontal flip), illumination adjustment (brightness ±20%, contrast ±15%), noise injection (Gaussian noise σ=0.01), and elastic deformation (mesh distortion coefficient 0.05) to improve the model's robustness to interference factors such as illumination changes and instrument occlusion in the surgical field. Simultaneously, the CLAHE algorithm is used to enhance local image contrast, and median filtering is employed to remove salt-and-pepper noise from endoscopic images. Finally, all images are uniformly adjusted to a size of 640×640 pixels, with pixel values ​​normalized to the [0,1] interval.

[0016] Model building and training; The model is based on YOLO11 and innovatively introduces a multi-scale attention mechanism to form a three-level processing flow of "feature extraction - dynamic memory - precise localization". The backbone network adopts an improved multi-scale axial attention structure, and the neck feature fusion layer is designed as a dual-channel structure: multi-scale axial attention modules are added to the downsampling path and the upsampling path to model the feature maps of different scales in the time dimension and realize the dynamic association of anatomical features between consecutive surgical frames.

[0017] The improvement in the attention mechanism is reflected in two aspects: the multi-scale axial attention module generates the spatial weight distribution of feature maps through convolutional layers, highlighting the regional response of key structures such as intervertebral discs and nerves; the detection head adopts a decoupled design to improve the localization accuracy of bounding boxes and masks.

[0018] Model training was conducted using the PyTorch 2.0+ deep learning framework, with a hardware environment consisting of two NVIDIA RTX 3060 GPUs (12GB VRAM). A cosine annealing learning rate scheduling strategy was employed, with an initial learning rate of 0.001, a weight decay factor of 1e-4, and a momentum of 0.937. The dataset was divided into training and validation sets at an 80%:20% ratio, and an early stopping strategy was used (training was terminated if the validation set loss did not decrease for 20 consecutive epochs). The model with the best performance on the validation set was selected as the test model. The specific training process is as follows: Feature extraction stage: The input end receives the preprocessed endoscope image (size 640×640×3), performs low-level feature extraction through the initial convolutional layer group, and generates the first layer feature map (containing local edge and texture information). The input multi-scale convolutional module (C3 and SPPF) captures texture features in small, medium, and large receptive fields, respectively. This stage outputs feature maps (containing coarse-grained spatial distribution information of bones, muscles, ligaments, etc.). The input is the "Multi-Scale Axial Attention Module" (MSA Block), which consists of three dilated convolutional branches with dilation rates of 1, 2, and 4, used to model structural relationships at different scales. The outputs of each branch are concatenated to generate a fused feature map. The MSA module then calculates attention weights along the channel and spatial directions, highlighting the response intensity of target regions (such as neural boundaries and intervertebral disc contours) and suppressing background interference. The output then proceeds to the next stage.

[0019] Dynamic memory stage: The feature fusion stage employs a parallel structure of upsampling and downsampling paths. The downsampling path is responsible for extracting deep semantic features, while the upsampling path achieves multi-scale information backflow through interpolation and skip connections (Concat). A multi-scale axial attention module is embedded at each feature fusion node to dynamically weight feature maps from different levels, forming a spatiotemporally consistent fused feature map. This process can be understood as "cross-frame memory": in sequential frame inference, the network uses the feature map of the previous frame to update the weight distribution of the current frame, thereby maintaining the temporal continuity of structural recognition; Through multi-layer fusion and weighting, the model generates a unified feature representation, which includes complete target space layout and dynamic semantic information.

[0020] Precise positioning process: The input detection head consists of three branches, each corresponding to the prediction of targets at different scales: Small target detection branch: Identifies fine-grained structures such as nerve roots; Mid-scale target detection branch: Identifies mid-scale tissues such as ligaments and intervertebral discs; Large target detection branch: Identifying macroscopic structures such as bones and muscles; Each branch contains an attention module to refine the feature response, and the output includes a prediction result containing a bounding box (BBox), confidence score, and class mask.

[0021] When the validation set loss does not decrease for 20 consecutive epochs, the early stopping strategy is triggered to terminate training, and the model with the best performance on the validation set (minimum loss and highest accuracy) is selected as the final test model.

[0022] This study employed a two-dimensional testing approach to evaluate model performance: a random sampling test to verify basic recognition capabilities and a generalization test to assess clinical applicability. The random sampling test used stratified sampling to divide 58,667 images into a training set (46,934 images, 80%) and a test set (11,733 images, 20%), ensuring consistent distribution of anatomical structures in both groups (e.g., a protruding nucleus pulposus percentage of 18.7% in both groups). The generalization test set, independent of the training process, consisted of data from 3 patients (1 L4 / 5, 2 L5 / S1) from device A, 3 patients (1 L4 / 5, 2 L5 / S1) from device B, and 1 patient from device 3. This set included 1,842 images not used in training and was used to simulate different clinical application scenarios involving different patients and devices.

[0023] The evaluation metrics used are Mask Precision (Mask P) and Mask Recall (Mask R), which are commonly used in the field of medical image segmentation: Mask P: The ratio of the number of correctly identified anatomical structure pixels to the total number of pixels predicted by the model, reflecting the accuracy of the prediction results; Mask R: The ratio of the number of correctly identified anatomical structure pixels to the total number of pixels labeled with the gold standard, reflecting the model's ability to cover the real structure.

[0024] All tests were run independently three times under the same hardware environment, and the average value was taken as the final result to reduce the impact of random errors. Table 1 shows the data distribution.

[0025] Table 1

[0026] The model demonstrated excellent anatomical structure recognition performance in random sampling tests, with Mask Precision (P) and Mask Recall (R) for all six target tissue categories reaching a high level of approximately 0.9. Skeletal structures showed the best recognition performance, with Mask P of 0.93±0.02 and Mask R of 0.92±0.03; followed by protruding nucleus pulposus (Mask P=0.91±0.02, Mask R=0.90±0.03) and intervertebral discs (Mask P=0.90±0.03, Mask R=0.89±0.02). The recognition accuracy for neural tissues was relatively lower, but Mask P and Mask R still reached 0.88±0.03 and 0.87±0.04, respectively.

[0027] The high recognition rate is mainly due to the multi-scale axial attention mechanism's ability to continuously track features. It can effectively capture the spatiotemporal correlation of relatively stable structures such as bones and intervertebral discs in the surgical field of view, enhance the feature weight of small tissues such as nerves and ligaments, and improve the feature discrimination of blurred boundary areas through the synergistic effect of spatial attention and channel attention. Table 2 is a table of Mask P / R values ​​(Mean±SD) of each anatomical structure in the sampling test.

[0028] Table 2

[0029] In the generalization test, the model performance showed a certain decline compared to the sampling test, but maintained good overall stability. Except for intervertebral discs (Mask R=0.68±0.04), bones (Mask R=0.69±0.03), and herniated nucleus pulposus (Mask R=0.67±0.05), the Mask P and Mask R values ​​for other structures remained above 0.7. In the device A dataset, ligaments showed the highest Mask P (0.82±0.03), while nerves showed the lowest Mask R (0.71±0.04). In the device B dataset, muscles showed the best recognition accuracy (Mask P=0.80±0.04, Mask R=0.79±0.03), while the Mask R value for herniated nucleus pulposus decreased to 0.65±0.05.

[0030] Performance degradation is mainly affected by two factors: first, the differences in optical characteristics between different devices. The 4K ultra-high-definition imaging of device B causes some tissue texture features to deviate from the distribution of the training set (mainly device A); second, individual anatomical variations in patients, such as the increased muscle layer thickness in obese patients, may interfere with the model's recognition of deep ligaments. It is worth noting that the Mask R attenuation is the largest for bones and protruding nucleus pulposus (>0.2), indicating that the model is more sensitive to morphological variations in high-density tissues. Table 3 shows the performance comparison (Mean±SD) between sampling tests and generalization tests.

[0031] Table 3

[0032] Robustness tests across devices and patients showed that the model significantly outperformed the traditional YOLO11 baseline model: in the generalization test on device B, the model's average Mask R was 0.72±0.05, an improvement of 18.0% compared to YOLO11 (0.61±0.06); on new patient data, the average Mask P reached 0.76±0.04, superior to the comparative model's 0.65±0.05 (p<0.01). This result validates the role of the multi-scale axial attention module in improving the stability of feature extraction—by storing anatomical features of historical frames through dynamic memory units, the model can maintain feature consistency even with significant differences in device imaging, especially in the recognition of rigid structures such as bones and intervertebral discs.

[0033] Further analysis revealed that the model's adaptability to data from device A was better than that from device B, possibly due to the higher proportion of device A data in the training set (74.2%). This suggests that future research could employ a multi-center, multi-device data balancing training strategy to reduce device dependence. Furthermore, the computational cost increased by only 12% (single-frame inference time increased from 18ms to 20ms), still meeting the latency requirements for intraoperative real-time assistance (<50ms), indicating that the model achieved a good balance between accuracy and efficiency.

[0034] In summary, the model demonstrates both high accuracy and strong generalization in the spinal endoscopic anatomical structure identification task, and its performance degradation mode provides a clear direction for subsequent optimization: for bone and protruding nucleus pulposus structures with high degradation rates, 3D convolutional modules can be introduced to enhance spatial depth information; for the problem of device differences, domain adaptive training technology may be a potential solution.

[0035] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A deep learning-based endoscopic anatomy assisted detection recognition system, characterized in that, The application relates to a multi-class anatomical structure detection and recognition method based on a multi-scale axial attention mechanism. The data acquisition module comprises multiple endoscopic spine devices, and is used for acquiring video data of endoscopic spine surgery and extracting key frames from the video data at a preset time interval to obtain multiple effective images. The data preprocessing module is connected with the data acquisition module, and comprises an annotation tool. The data preprocessing module is used for anatomical structure annotation, quality control and image enhancement processing of the effective images, and outputs standardized images to obtain a data set. The model processing module is connected with the data preprocessing module, and comprises a deep learning model with a multi-scale attention mechanism. The deep learning model takes YOLO11 as a basic architecture, and comprises a backbone network, a neck feature fusion layer and a detection head. The backbone network adopts a multi-scale axial attention structure. The neck feature fusion layer is designed as a double-channel structure. The double-channel structure is additionally provided with a multi-scale axial attention module in a down-sampling path and an up-sampling path.

2. The deep learning-based endoscopic anatomy aided detection recognition system according to claim 1, wherein, The model processing module is used for training the deep learning model through the data set to obtain a trained deep learning model.

3. The deep learning-based endoscopic anatomy aided detection recognition system of claim 1, wherein, The detection and recognition module is connected with the model processing module.

4. The method of claim 1, wherein the method is applied to the system of any one of claims 1-3. The detection and recognition module is used for inputting a to-be-recognized image into the trained deep learning model to realize detection and recognition of multiple types of anatomical structures. The anatomical structures include muscle structures, ligament structures, skeletal structures, nerve structures, intervertebral disc structures and protruding nucleus pulposus structures. The multi-scale axial attention module of the deep learning model generates a spatial weight distribution of a feature map through a convolution layer. The detection head part of the deep learning model adopts a decoupling design. The method comprises the following steps: S1: data collection and preprocessing; effective images are obtained through the data acquisition module, and the effective images are input into the data preprocessing module to obtain a data set. S2: model training stage; 5. The deep learning-based endoscopic anatomy aided detection and recognition method according to claim 4, characterized in that, The model training stage comprises a feature extraction link, a dynamic memory link and a precise positioning link.

6. The deep learning-based endoscopic anatomy aided detection and recognition method according to claim 4, characterized in that, The feature extraction link cooperates the backbone network with the multi-scale axial attention module to capture local edge, texture and key area features of anatomical structures in an image and suppress background interference. The dynamic memory link relies on the double-channel structure of the neck feature fusion layer and the multi-scale axial attention module to realize different scale feature fusion and continuous frame feature dynamic association of anatomical structures and maintain feature spatio-temporal consistency. The precise positioning link completes boundary box positioning, category determination and mask generation of different scale anatomical structures through the decoupling designed detection head. The model training stage adopts a cosine annealing learning rate scheduling strategy, trains through the training set and a preset learning rate, a weight decay coefficient and a momentum, adopts an early stopping strategy, and finally selects a deep learning model with optimal performance of the validation set as a final recognition model for testing. S3: a to-be-recognized image is input into the final recognition model through the detection and recognition module to realize detection and recognition of multiple types of anatomical structures. In S2, the model training stage is based on a PyTorch 2.0+ deep learning framework, the initial learning rate is 0.001, the weight decay coefficient is 1e-4, and the momentum is 0.

937. The feature extraction link specifically comprises: The input end of the deep learning model receives images of the training set, and then low-level feature extraction is performed through the initial convolutional layer group of the backbone network to generate a first layer feature map containing local edge and texture information; The first layer feature map is input into the multi-scale convolution module of the backbone network to capture texture features under small, medium and large receptive fields respectively, and output a second feature map containing coarse-grained spatial distribution information of anatomical structures; The second feature map is input into the multi-scale axial attention module of the backbone network, which is composed of three dilated convolution branches inside to model structure associations of different scales. After channel splicing, each branch outputs a fusion feature map. The multi-scale axial attention module further calculates attention weights in the channel and spatial direction to highlight the response strength of the target region and suppress background interference, and outputs the fusion feature map to the dynamic memory link.

7. The deep learning-based endoscopic anatomy aided detection and recognition method according to claim 6, characterized in that, The dynamic memory link specifically includes: The fusion feature map is input into the neck feature fusion layer, which adopts a parallel structure of up-sampling and down-sampling paths. The down-sampling path is responsible for extracting deep semantic features, and the up-sampling path realizes multi-scale information backflow through interpolation and jump connection; A multi-scale axial attention module is embedded at each feature fusion node of the neck feature fusion layer to dynamically weight the feature maps from different levels in the fusion feature map through the multi-scale axial attention module, forming a spatio-temporally consistent fusion feature map; After multi-layer fusion and weighting processing of the neck feature fusion layer, a unified feature representation containing complete target spatial layout and dynamic semantic information is generated. This unified feature representation is output as the output of the dynamic memory link and input into the precise positioning link.

8. The deep learning-based endoscopic anatomy aided detection and recognition method according to claim 7, characterized in that, The precise positioning link specifically includes: The unified feature is input into the detection head of the recognition model, which contains three branches corresponding to the prediction of targets of different scales. The small target detection branch is responsible for identifying fine-grained structures including nerve roots, the medium target detection branch is responsible for identifying medium-scale tissues including ligaments and intervertebral discs, and the large target detection branch is responsible for identifying macroscopic structures including bones and muscles. Each branch of the detection head contains an attention module inside to refine the feature response of the corresponding branch, and finally outputs the bounding box, confidence and class mask of the corresponding target to form the recognition result of multiple types of anatomical structures.

Citation Information

Patent Citations

  • Lightweight pulmonary nodule detection method based on multi-dimensional collaborative attention mechanism

    CN120339256A

  • Tomato image real-time detection method, tomato image real-time detection system, tomato picking method and tomato picking system

    CN120388367A

  • Multi-mode teenager idiopathic scoliosis screening method based on back RGB-D image

    CN120543912A

  • Medical image computer-aided analysis method based on deep learning

    CN120807509A

  • Road surface scattering detection method and device based on deep learning, electronic equipment and program product

    CN121121507A