A nerve root lightweight segmentation method and system for intraoperative images of PELD

Abstract

The present application belongs to the technical field of medical endoscopic image semantic segmentation, and proposes a neural root lightweight segmentation method and system for PELD intraoperative images. Obtain the intraoperative endoscopic original video frame approved by ethics to construct image training samples; build and initialize a lightweight hybrid segmentation network; use image training samples to iteratively train the network; each layer of global iteration queries a refined decoder to complete three-stage multi-scale feature iterative refinement of coarse alignment, mask purification and fine correction; the iterative region boundary loss calculation unit performs layered region and edge joint supervision on the output of the multi-layer decoder; all network parameters are updated in reverse; continue training until the preset maximum iteration number is reached or the early stopping condition is met; load the optimal model to perform inference on the intraoperative real-time endoscopic video frame, and output the neural root segmentation visualization image. The present application solves the problems of segmentation missed detection, fracture, boundary offset, large parameter quantity, high inference delay and insufficient real-time performance of existing models.

Description

Technical Field

[0001] This invention belongs to the field of semantic segmentation technology of medical endoscopic images, and particularly relates to a lightweight segmentation method and system for nerve roots in PELD intraoperative images. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] Percutaneous endoscopic discectomy (PELD) of the lumbar spine is currently the mainstream minimally invasive spinal surgery. Real-time and precise segmentation of nerve roots during the operation is crucial to avoid iatrogenic injury and ensure surgical safety. The core solution for intraoperative intelligent navigation is a medical endoscopic segmentation model based on deep learning. Existing models include pure convolutional networks, pure Transformer networks, and CNN-Transformer hybrid networks.

[0004] Traditional pure convolutional networks, such as U-Net and DeepLab series, rely on local convolution to extract features. With a fixed receptive field, they struggle to model long-range global dependencies. When dealing with thin, strip-like nerve roots in PELD surgery, where grayscale values ​​highly overlap with fat / ligamentum flavum, they are prone to missed detections and segmentation breaks. Pure Transformer networks, such as Swin-Unet, while possessing global modeling capabilities, have a large number of parameters and slow inference speed, making them unsuitable for real-time intraoperative processing and hindering clinical equipment deployment.

[0005] Hybrid CNN-Transformer architectures such as SegFormer, FeedFormer, and U-MixFormer balance local detail extraction with global context modeling, but still have significant drawbacks: the decoder only uses unidirectional feature transfer and lacks cross-level feature backpropagation and iterative optimization mechanisms, making it unable to purify features for thin, low-contrast weak neural roots; it cannot cope with complex interferences such as intraoperative occlusion, metallic reflection, and bleeding, resulting in insufficient weak feature mining, leading to segmentation contour drift, jagged edges, and boundary defects; the loss function only constrains the region overlap and cannot simultaneously optimize region integrity and contour smoothness.

[0006] Furthermore, existing lightweight solutions generally adopt a design approach of encoder optimization and simple upsampling of decoder, neglecting fine boundary reconstruction. This results in insufficient segmentation accuracy in real-world scenarios with multiple interferences, weak targets, and limited labeled samples, which can easily mislead operators and increase the risk of neural injury. At the same time, boundary optimization methods often remain at a superficial level of correction and cannot fundamentally solve the segmentation distortion caused by occlusion. Summary of the Invention

[0007] To overcome the shortcomings of the prior art, this invention provides a lightweight segmentation method and system for nerve roots in PELD intraoperative images. It aims to solve the problems of existing models in scenarios with scarce clinical annotations in spinal endoscopy, instrument occlusion, grayscale overlap and reflection interference, such as missed segmentation, breakage, boundary offset, large number of parameters, high inference delay and insufficient real-time performance, which can easily mislead surgical operations.

[0008] To achieve the above objectives, one or more embodiments of the present invention provide the following technical solutions: The first aspect of this invention provides a lightweight segmentation method for nerve roots in intraoperative PELD images, comprising: Obtain ethically approved raw video frames of intraoperative endoscopy in PELD clinical procedures and perform standardized preprocessing to construct image training samples; A lightweight hybrid segmentation network was built and initialized, including a MiT-B0 lightweight hierarchical encoder, a multi-layer global iterative query refinement decoder, and an iterative region boundary loss calculation unit. Training hyperparameters are pre-configured, and the lightweight hybrid segmentation network is iteratively trained using the image training samples. Each layer of the global iterative query refinement decoder completes three stages of multi-scale feature iterative refinement: coarse alignment, mask purification, and fine correction. The iterative region boundary loss calculation unit performs hierarchical region and edge joint supervision on the output of the multi-layer global iterative query refinement decoder. After each round of training, all network parameters are updated in reverse after forward inference. Continue training until the preset maximum number of iterations is reached or the early stopping condition is met. Save the network weights with the best validation set metrics, load the optimal model to perform inference on real-time intraoperative endoscopic video frames, and output a visualized image of nerve root segmentation.

[0009] As a further technical solution, the standardization preprocessing includes: The original video frames are uniformly scaled to adjust the resolution; the images are normalized; online augmentation is performed using random scaling, random cropping, horizontal flipping, rotation, brightness shifting, and zero-filling of edges; and the dataset is divided into training, validation, and test sets according to the specified proportions.

[0010] As a further technical solution, the MiT-B0 lightweight hierarchical encoder downsamples the image training samples four times in succession and outputs four-layer multi-scale encoder features.

[0011] As a further technical solution, each layer of global iterative query refinement decoder independently builds a global feature pool, constructing a bidirectional multi-scale feature iterative interaction mechanism in which high-level semantics are transmitted from top to bottom and shallow refinement features are fed back from bottom to top.

[0012] As a further technical solution, the pre-configured training hyperparameters include: AdamW optimizer initial learning rate, weight decay coefficient, mask truncation threshold, IRB loss region balancing weight, IRB loss boundary balancing weight, validation interval iteration rounds, early stopping tolerance iteration rounds, and global maximum training iterations.

[0013] As a further technical solution, in the coarse alignment stage of the global iterative query refinement decoder, the features of the four-layer multi-scale encoder and the upper-layer refinement features are integrated to construct a global feature pool. Attention queries, key and value features are generated through scale uniform transformation and linear projection. Coarse localization of the neural root is completed by relying on scaling dot product attention. The mask purification stage of the global iterative query refines the decoder, and parallel multi-scale deep separable convolution extracts contextual features. After concatenation, pixel confidence maps are generated and threshold truncation is performed to filter out fat, instrument background noise and retain weak neural pixels. The global iterative query refines the fine correction stage of the decoder. Based on the mask-purified features, edge enhancement branches are constructed to generate update queries. The secondary cross-attention iteratively corrects the neural distortion contours, and the refined features are fed back into the global feature pool for use by the lower-level decoder.

[0014] As a further technical solution, the iterative region boundary loss includes a region loss term and a boundary gradient loss term; the region loss term integrates Dice loss and Focal loss to enhance the weak neural recognition capability; the boundary gradient loss term relies on the Sobel operator to extract the ground truth and predicted gradient, and calculates the gradient error only within the effective neural mask; the weighted summation of the coarse and fine prediction results of multi-layer decoding constitutes the overall training loss.

[0015] A second aspect of the present invention provides a lightweight nerve root segmentation system for intraoperative PELD images, comprising: The data acquisition module is configured to: acquire ethically approved raw video frames of PELD clinical intraoperative endoscopy and perform standardized preprocessing to construct image training samples; The network construction initialization module is configured to: build and initialize a lightweight hybrid segmentation network, including a MiT-B0 lightweight hierarchical encoder, a multi-layer global iterative query refinement decoder, and an iterative region boundary loss calculation unit; The network training module is configured to: pre-configure training hyperparameters and use the image training samples to iteratively train the lightweight hybrid segmentation network; wherein, each layer of the global iterative query refinement decoder completes three stages of multi-scale feature iterative refinement: coarse alignment, mask purification, and fine correction; the iterative region boundary loss calculation unit performs hierarchical region and edge joint supervision on the output of the multi-layer global iterative query refinement decoder; after each round of training, all network parameters are updated in reverse after forward inference; The intraoperative real-time inference module is configured to: continuously train until the preset maximum number of iterations is reached or the early stopping condition is met, save the network weights with the best validation set metrics, load the optimal model to perform inference on the intraoperative real-time endoscopic video frames, and output a visual image of nerve root segmentation.

[0016] A third aspect of the present invention provides a computer-readable storage medium having a program stored thereon that, when executed by a processor, implements the steps of a method for lightweight segmentation of nerve roots in PELD intraoperative images as described in the first aspect of the present invention.

[0017] A fourth aspect of the present invention provides an electronic device including a memory, a processor, and a program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of a lightweight segmentation method for nerve roots in PELD intraoperative images as described in the first aspect of the present invention.

[0018] The above one or more technical solutions have the following beneficial effects: This invention designs a GIQR global iterative query refinement module, constructs a global feature pool to achieve multi-layer feature bidirectional iterative backpropagation optimization, and relies on multi-scale confidence masks to adaptively filter various background interferences during surgery. It specifically enhances the feature expression of slender and weak nerve roots, improving nerve omission and contour breakage defects from the feature interaction level. Compared to traditional decoding schemes that only use unidirectional feature fusion, it can better handle complex intraoperative visual fields with instrument occlusion and low contrast, providing spinal surgeons with stable and clear anatomical references and reducing the risk of operational errors.

[0019] This invention constructs an IRB iterative region boundary joint loss mechanism, which hierarchically and synchronously constrains region integrity and edge smoothness features, while taking into account both coarse segmentation localization and fine contour correction, thus overcoming the shortcomings of existing losses that only focus on region overlap and lack edge gradient supervision. By having a feature purification module work in conjunction with the boundary loss, the neural segmentation topology is doubly optimized, improving the inherent defects of jagged segmentation contours and boundary offsets.

[0020] This invention adopts a lightweight CNN-Transformer hybrid architecture, using a lightweight backbone paired with simplified decoding units. While ensuring segmentation performance, it significantly reduces the number of model parameters and inference overhead, overcoming the limitations of heavy Transformer networks that rely on high-end hardware and lack real-time performance. It can be adapted for embedded deployment in operating room endoscopic terminals, meeting the needs of real-time navigation of dynamic intraoperative images.

[0021] This invention provides a dedicated data augmentation solution for spinal endoscopy, optimizing training robustness for real-world scenarios where clinically labeled samples are scarce. At the same time, the model has general endoscopy adaptability, allowing it to be transferred to other endoscopic segmentation tasks such as gastrointestinal polyps without backbone fine-tuning. The algorithm has a wider range of applicable scenarios and stronger versatility for deployment.

[0022] This invention is developed for real-world PELD minimally invasive surgical scenarios. It builds a dedicated labeled dataset based on clinical surgical images. The overall network is designed around the needs of identifying nerve roots with partial occlusion and weak features during surgery. In particular, it can help inexperienced young surgeons quickly identify nerve structures, reduce iatrogenic nerve injury-related complications, and improve the overall safety level of minimally invasive spinal endoscopic surgery.

[0023] Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0024] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0025] Figure 1 This is a schematic diagram of the overall architecture of the BRI-MixFormer lightweight segmentation network in the first embodiment; Figure 2 This is a detailed structural diagram of the GIQR module in the first embodiment; Figure 3 The image shows a comparison of the segmentation performance of the existing mainstream segmentation algorithms in the first embodiment with that of the present method in intraoperative endoscopic scenarios with instrument occlusion and low contrast. Figure 4 This is a thermal visualization diagram of the feature evolution of the GIQR module in the first embodiment. Detailed Implementation

[0026] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0027] It should be noted that the terminology used herein is for the purpose of describing particular implementations only and is not intended to limit the exemplary implementations of the present invention.

[0028] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0029] Example 1 This embodiment discloses a lightweight nerve root segmentation method for intraoperative images of PELD (Penetrating Endoscopic Surgery), which is particularly suitable for high-precision segmentation of delicate and weak nerve roots in PELD lumbar spine minimally invasive surgery. It can also be extended to lightweight CNN-Transformer hybrid segmentation methods and systems for segmenting various types of endoscopic lesions. The overall process involves the following steps: Step S100: Obtain ethically approved raw video frames of PELD clinical intraoperative endoscopy and perform standardized preprocessing to construct image training samples.

[0030] In this embodiment, retrospective PELD minimally invasive surgery videos approved by the ethics committee of a tertiary hospital were collected. After extracting the original endoscopic images frame by frame, the nerve roots were annotated at the pixel level by a spinal specialist. In the offline preprocessing stage, all images were uniformly scaled to a resolution of 512×512, and the mean and variance of ImageNet were used to complete the pixel standardization.

[0031] The online augmentation during the training phase includes random scaling from 0.8 to 1.2x, random cropping with foreground proportion constraints, horizontal flipping with a 50% probability, random rotation of ±15 degrees, brightness offset of ±30 luminance distortion, and zero-fill edge operations. It is divided into training, validation, and test sets in an 8:1:1 ratio, and training samples that can be read by the model are constructed in batches, with the training batch size set to 4 and the validation and test batch sizes set to 8.

[0032] Step S200: Build and initialize the lightweight hybrid segmentation network, such as... Figure 1 As shown, it includes a MiT-B0 lightweight hierarchical encoder, a multi-layer GIQR (Global Iterative Query Refinement) decoder, and an IRB (Iterative Region Boundary) loss calculation unit.

[0033] Specifically, the encoder uses the MiT-B0 lightweight backbone and outputs four-layer multi-scale encoder features after four consecutive downsampling operations. , Each layer of the global iterative query refinement decoder is independently configured with a GIQR module, and a global feature pool is built to realize a two-way feature iteration mechanism that guides high-level semantics from top to bottom and provides feedback on shallow details from bottom to top; the IRB loss calculation unit provides hierarchical joint supervision of region segmentation accuracy and contour smoothness during the network training phase.

[0034] Step S300: Pre-configure training hyperparameters and iteratively train the lightweight hybrid segmentation network using image training samples; wherein, each layer of the global iterative query refinement decoder completes three stages of multi-scale feature iterative refinement: coarse alignment, mask purification, and fine correction; the iterative region boundary loss calculation unit performs hierarchical region and edge joint supervision on the output of the multi-layer global iterative query refinement decoder; after each round of training, all model parameters are updated in reverse after forward inference. The specific steps are as follows: S310: Training Hyperparameter Settings: Initial learning rate set to... The weight decay coefficient is set to 0.05, the global maximum number of training iterations is set to 25,000, and a validation evaluation is conducted every 50 rounds; the optimal threshold for GIQR confidence mask lower bound is set. ; Set the IRB loss region balance weight Boundary gradient balancing weights The early stopping tolerance iteration count is set to 200. Training will stop if there is no improvement in mDice for several consecutive iterations.

[0035] S320: The MiT-B0 lightweight layered encoder receives the original video frames after PELD intraoperative preprocessing and completes layered downsampling and multi-scale feature extraction using a multi-layer overlapping patch merging module. The MiT-B0 lightweight layered encoder has four feature levels. Each level achieves size compression through a patch merging operation with overlapping windows and captures the contextual features of each layer through efficient local self-attention. After the image is compressed at four scales step by step, it outputs four sets of layered multi-scale encoder features with progressively smaller spatial dimensions and progressively larger channel dimensions, fully preserving multi-layered contextual information from shallow texture details to deep global anatomical semantics.

[0036] S330: GIQR global iterative query refinement module for phased feature calculation.

[0037] like Figure 2 As shown, the GIQR module consists of three stages: coarse alignment, mask refinement, and fine correction. The complete execution flow is as follows: S331: Coarse Alignment Stage: A global feature pool is constructed by fusing features from a four-layer multi-scale encoder with refined features from the upper layer. Attention queries, key-value features, are generated through scale unification transformation and linear projection. Coarse localization of neural roots is completed using scaling dot product attention. Specifically: First, a global feature matrix is ​​constructed by fusing the features of all four multi-scale encoder layers and the optimized refined features of the upper layers for the i-th decoding layer. , as a generation key ,value The original data source, and the corresponding calculation formula is:

[0038] In the formula: This represents the global feature matrix corresponding to the i-th layer decoder; Concat is a concatenation operation along the channel dimension. Represents the scale alignment transformation function; The input features for filtering the j-th layer are (single-level features after performing a two-choice filtering for a single level j), where j is 1, 2, 3, or 4, which are the feature level numbers.

[0039] The formula for segmented feature selection is as follows:

[0040] In the formula, j is the feature level number, and i is the current decoding layer number. This refers to the refined features output by the GIQR module in the previous training round (i.e., the refined features that have been optimized in the upper layer). This represents the j-th layer multi-scale encoder feature from the original output of the MiT-B0 encoder. When the feature layer number j is greater than the current decoding layer number i, the optimized and refined features from the upper layer are selected to participate in the global pooling, enabling the downward propagation of high-level semantics. When the feature layer number j is less than or equal to the current decoding layer number i, the multi-scale encoder feature is directly used to preserve the shallow details of the original image, ultimately achieving bidirectional feature feedback of high-level semantic guidance and shallow detail preservation.

[0041] The following is a scale alignment transformation, as shown in the formula:

[0042] In the formula, For the j-th single-level features to be unified in scale (i.e. , (It is the existence of a variable that can be substituted). The pooling scaling factor corresponding to the feature of the j-th layer; This is a two-dimensional average pooling operation, the purpose of which is to unify the size of the feature space. This represents a 1×1 pointwise convolution, and its function is to unify the number of feature channels at each level.

[0043] The formula for calculating pooling scaling factor is:

[0044] In the formula, To maximize the operation, the base of the exponent, 2, represents the downsampling factor, j is the feature level number, and 4 represents the deepest feature layer number of the encoder. If j is less than or equal to 4, The calculation result is 0, corresponding to No pooling is needed. If j is less than 4, the corresponding power of 2 is calculated to perform size compression, ensuring that all features are unified to the deepest layer. Resolution.

[0045] After constructing the global feature matrix, two sets of independent learnable linear projection layers are used to map and generate the key matrix. Value matrix The corresponding formula is: ,

[0046] in , As a learnable linear weight matrix, it can uniformly project global features onto the attention dimension. , and It is used to carry global anatomical information and various background interference contextual information of the entire endoscopic image.

[0047] The query vector is taken from the multi-scale encoder features of the current decoding layer. That is, GIQR input features Dimensional mapping is achieved through a dedicated learnable linear projection, resulting in the following formula:

[0048] in, To query specific learnable linear weights, This represents the local neural features to be retrieved at the current pixel location.

[0049] Obtain the original , , After generating the tensor, layer normalization needs to be performed separately to eliminate feature distribution shift. The general formula for layer normalization is:

[0050] In the formula, The characteristic mean, For characteristic variance, To prevent extremely small constants with a denominator of 0, , To learn the scaling and offset parameters, substituting them into the three sets of tensors yields:

[0051]

[0052]

[0053] The three sets of normalized features are input into the scaling dot product attention calculation, and the calculation formula is as follows:

[0054] In the formula, For the attention head dimension, Calculate the semantic similarity between the local query and the global key. Scaling suppresses numerical explosion caused by high-dimensional inner products, Softmax normalizes the output attention weights, and then weights them with the value matrix to fuse global context features. Finally, residual connections are used to fuse the original query features to obtain coarse-aligned features. The calculation formula is:

[0055] In the formula, LN represents the layer normalization operation. This calculation completes the coarse localization of the nerve root and outputs... It is then sent to the mask purification stage.

[0056] S332: Masking and Refinement Stage: Parallel multi-scale depthwise separable convolutions extract contextual features, which are then concatenated to generate pixel confidence maps and truncated with a threshold to filter out fat, instrument background noise, and retain weak neural pixels. Specifically: Coarse alignment features Record Three sets of multi-scale depthwise separable convolutional branches are constructed in parallel to extract the context, corresponding to the formula:

[0057]

[0058]

[0059] In the formula, for Depth convolution, with PW being 1×1 per-convolution to achieve channel fusion.

[0060] Three multi-scale features are concatenated and then converted into the original confidence map via 1×1 convolution and sigmoid activation.

[0061] In the formula, Activate Sigmoid.

[0062] Then, weak neural pixels are preserved by threshold truncation:

[0063] In the formula, S is the truncated confidence graph. This is the original confidence graph without truncation. Clip represents the interval truncation operation. The optimal threshold is truncated to the lower bound of the confidence mask.

[0064] Background noise reduction is achieved by element-wise multiplication of coarse alignment features with the truncated confidence map. The calculation formula is as follows:

[0065] in, For masking purification features, This is the element-wise product of Hadama.

[0066] S333: Fine-tuning stage: Based on mask-refined features, edge enhancement branches are constructed to generate update queries. Secondary cross-attention iteratively corrects neural distortion contours, and refined features are fed back into the global feature pool for use by the lower-level decoder. Specifically: The fine-tuning stage is based on mask-based feature extraction and edge enhancement query vector. Calculation formula:

[0067] In the formula, For batch normalization, Depth-separable convolution with a 3×3 kernel. For dimension reconstruction operations.

[0068] Use DropPath regular expressions to construct residual structures and update query vectors:

[0069] In the formula, For the updated query vector, Discard regular expressions for random paths.

[0070] The updated query is again combined with normalization. , Perform cross-attention iterative correction of boundaries and output refined features. Calculation formula:

[0071] In the formula, MLP is a multilayer perceptron that performs nonlinear feature mapping. This is for cross-attention calculation.

[0072] S340: Construct the IRB iterative region boundary loss and perform hierarchical joint supervised computation.

[0073] IRB employs multi-layer deep joint supervision, simultaneously constraining the integrity and contour smoothness of the segmented region. The formulas for calculating the total loss are explained in detail below.

[0074] The first formula is the formula for calculating regional loss:

[0075] In the formula, This represents the loss function for a single-channel region in hierarchical decoding. The Dess loss is used to measure the degree of overlap between the predicted segmentation region and the ground truth annotation. , is the focus loss, used to reduce the weight of simple background samples and focus on weak neural pixels that are difficult to distinguish; P is the single-level segmentation prediction map of the model; G is the ground truth map of the nerve roots annotated by the spinal surgeon; It is the loss balance coefficient of the focus loss, used to adjust the proportion of loss for hard samples.

[0076] The second set of formulas are for approximate solutions of the gradient between the prediction and the true value:

[0077] In the formula, This represents an approximate value of the gradient magnitude of the predicted image. This represents an approximate value of the gradient magnitude in the true image. This is the convolution operator; It is a Sobel horizontal gradient convolution kernel used to extract horizontal edge information of the image; This is a Sobel vertical gradient convolution kernel used to extract vertical edges of an image; absolute value addition replaces the square root operation, simplifying gradient calculation overhead.

[0078] The third equation is the boundary gradient loss formula:

[0079] In the formula, L2 loss is used for boundary gradients; Normalize the predicted features and output the class probability; It is the gradient of the normalized prediction graph. The gradient of the truth value; represents the Hadamard element-wise multiplication operation; M is the effective region mask for the neural network, used to filter out irrelevant background areas such as fat and instruments, and the gradient error is calculated only within the neural root region; represents the squared L2 norm, used to measure the difference between the predicted gradient and the true gradient.

[0080] The fourth formula is the formula for calculating the total global loss:

[0081] In the formula, This represents the total loss during network training. The final fusion of the main output loss weights; This represents the final neural root segmentation result after multi-layer feature fusion; This represents the summation of the auxiliary losses of the decoders at layers 2 through 4. The auxiliary loss weights for the i-th layer decoder; A fixed balancing hyperparameter is used to control the weight ratio of regional overlap loss; This is the coarse segmentation prediction map for the i-th layer decoder; A fixed balancing hyperparameter is used for the boundary gradient loss to control the weight ratio of the contour gradient loss. This is the fine segmentation prediction map for the i-th layer decoder.

[0082] Step S400: Continue training until the preset maximum number of iterations is reached or the early stopping condition is met. Save the network weights with the best validation set metrics, load the best model to perform inference on the intraoperative real-time endoscopic video frames, and output a visual image of nerve root segmentation.

[0083] S410: Network iterative training and optimal model preservation determination.

[0084] The method for determining network iterative training and model saving is as follows: each forward propagation round completes GIQR feature refinement and multi-layer IRB loss calculation, with the total loss... To optimize the target, the AdamW algorithm is used to update all network parameters of the encoder and GIQR module in reverse. Every 50 iterations, segmentation metrics such as mDice and mIoU are calculated on the validation set. If there is no improvement in mDice on the validation set for 200 consecutive rounds, an early stopping mechanism is triggered. Training is terminated directly after reaching the maximum of 25,000 iterations. The model weight file with the best segmentation effect on the validation set is automatically saved throughout the process.

[0085] S420: Intraoperative real-time reasoning and clinical support workflow execution.

[0086] The operating room endoscopic equipment acquires real-time surgical video streams, performs standardized preprocessing frame-by-frame consistent with the training phase, and then inputs the pre-trained lightweight hybrid segmentation network. Under a single RTX 4090 graphics card environment, the inference frame rate can reach 190.52 FPS. It can output clear and complete visual segmentation masks of nerve roots without delay, and the real-time segmented images are displayed synchronously on the surgical monitor screen, highlighting occluded and low-contrast nerve areas, assisting spinal surgeons in quickly locating nerve roots, especially reducing the surgical risk of inexperienced surgeons having difficulty identifying nerves and accidentally damaging them. At the same time, the trained network model can be transferred to gastrointestinal polyp endoscopic segmentation tasks without readjustment, and has general capabilities for multiple endoscopic scenarios.

[0087] In this embodiment, comprehensive performance verification was carried out based on the self-built clinical PELD endoscopy dataset and five sets of publicly available gastrointestinal polyp endoscopy datasets. Multiple sets of quantitative tables and figure visualization results were provided. The experimental hardware was a single RTX4090 24G graphics card, and the software environment adopted Python 3.8, PyTorch 1.10, and MMSegmentation framework.

[0088] On the self-built PELD neural root segmentation dataset, the BRI-MixFormer proposed in this embodiment is compared with mainstream segmentation algorithms such as PIDNet, MobileNetV3-Large, U-Net, MobileNetV2, Mask2Former, SegNeXt, SwinTransformer, DeepLabV3, FeedFormer, SegFormer, DeepLabV3+, and U-MixFormer. The quantitative evaluation metrics are shown in Table 1. In the table, the arrow "↑" indicates that a larger metric value indicates better model performance, and the arrow "↓" indicates that a smaller metric value indicates better model performance.

[0089] Table 1 Performance comparison with the SOTA model on the PELD dataset

[0090] As shown in Table 1, the BRI-MixFormer proposed in this embodiment outperforms all baseline models in all core segmentation metrics, including DSC, IoU, Precision, and Recall. At the same time, it achieves a lightweight balance between parameter count and inference frame rate, taking into account both segmentation accuracy and the need for real-time inference during surgery. like Figure 3 The figure shown is a qualitative segmentation effect diagram of this embodiment and various comparison algorithms under typical intraoperative visual field with instrument occlusion and low contrast. Existing methods generally have defects such as missed nerve detection, regional breakage, and contour jagged drift. This embodiment can completely restore the anatomical morphology of slender nerve roots and the boundary fits the labeled true value.

[0091] like Figure 4 The image shown is a heatmap visualization of the feature evolution of the GIQR module. From left to right, it displays the original input endoscopic image, the ground truth value of the nerve root annotation, the heatmap of the coarse alignment stage, the confidence mask heatmap, and the feature heatmap after fine optimization. The coarse alignment stage can only vaguely locate the general area of ​​the nerve. The confidence mask can accurately distinguish the nerve from the surrounding fat, surgical instruments, bleeding, and other interfering tissues. After iterative query correction, the feature heatmap completely fits the real nerve contour, intuitively demonstrating the complete optimization process of the GIQR module, which progressively reduces noise, enhances weak nerve features, and corrects distortion boundaries.

[0092] At the same time, multiple ablation control experiments were conducted.

[0093] First, ablation was performed on the GIQR mask branch, and the quantitative results are shown in Table 2.

[0094] Table 2. Experimental results of GIQR masked branch ablation

[0095] Secondly, a sensitivity analysis experiment was conducted on the lower limit truncation threshold τ of the mask, and the experimental data are shown in Table 3.

[0096] Table 3 Comparison results of different mask truncation thresholds

[0097] As shown in Table 3, a threshold of τ=0.4 can achieve the optimal balance between neural foreground preservation and background suppression, resulting in the highest global mDice and mIoU indices.

[0098] Sensitivity experiments were conducted on the IRB loss weights α and β, and the results are shown in Table 4.

[0099] Table 4. Experimental results comparing different loss weight hyperparameters α and β

[0100] As shown in Table 4, the segmentation comprehensive index is optimal when α=1.0 and β=0.8, which is the fixed hyperparameter combination in this embodiment.

[0101] The experimental results of synergistic ablation using the GIQR module and the IRB loss module are shown in Table 5.

[0102] Table 5. Ablation Experiment Results of Core Modules on the PELD Dataset

[0103] As shown in Table 5, enabling GIQR or IRB alone can only slightly improve the performance of a single item. Using both together achieves bidirectional synergy between feature optimization and boundary constraints, and the segmentation accuracy and contour error reach the optimal level simultaneously.

[0104] To verify the model's generalization ability across endoscopic scenarios, five publicly available gastrointestinal polyp datasets—ClinicDB, Kvasir, ETIS, ColonDB, and EndoScene—were selected for a comparative analysis. The quantitative indicators are shown in Table 6.

[0105] Table 6 Performance Comparison Results of Polyp Segmentation Dataset

[0106] As shown in Table 6, this embodiment achieves the best mDice index on the ClinicDB, Kvasir, and EndoScene datasets, and maintains a high level of segmentation on the ColonDB and ETIS unfamiliar lesion datasets. It can adapt to different endoscopic lesion segmentation tasks without backbone network fine-tuning, demonstrating outstanding cross-scenario generalization ability and verifying the versatility of the architecture of this embodiment.

[0107] Example 2 like Figure 2 As shown, this embodiment discloses a lightweight nerve root segmentation system for intraoperative PELD images, including: The data acquisition module is configured to: acquire ethically approved raw video frames of PELD clinical intraoperative endoscopy and perform standardized preprocessing to construct image training samples; The network construction initialization module is configured to: build and initialize a lightweight hybrid segmentation network, including a MiT-B0 lightweight hierarchical encoder, a multi-layer global iterative query refinement decoder, and an iterative region boundary loss calculation unit; The network training module is configured as follows: pre-configure training hyperparameters and iteratively train the lightweight hybrid segmentation network using image training samples; each layer of the global iterative query refinement decoder completes three stages of multi-scale feature iterative refinement: coarse alignment, mask purification, and fine correction; the iterative region boundary loss calculation unit performs hierarchical region and edge joint supervision on the output of the multi-layer global iterative query refinement decoder; after each round of training, all network parameters are updated in reverse after forward inference. The intraoperative real-time inference module is configured to: continuously train until the preset maximum number of iterations is reached or the early stopping condition is met, save the network weights with the best validation set metrics, load the optimal model to perform inference on the intraoperative real-time endoscopic video frames, and output a visual image of nerve root segmentation.

[0108] Example 3 The purpose of this embodiment is to provide a computer-readable storage medium.

[0109] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of a lightweight segmentation method for nerve roots in PELD intraoperative images as described in Embodiment 1 of this disclosure.

[0110] Example 4 The purpose of this embodiment is to provide an electronic device.

[0111] An electronic device includes a memory, a processor, and a program stored in the memory and executable on the processor. When the processor executes the program, it implements the steps in a lightweight segmentation method for nerve roots in PELD intraoperative images as described in Embodiment 1 of this disclosure.

[0112] The steps and methods involved in the apparatuses of Embodiments 2, 3, and 4 above correspond to those in Embodiment 1. For specific implementation details, please refer to the relevant description section of Embodiment 1. The term "computer-readable storage medium" should be understood as a single medium or multiple media including one or more instruction sets; it should also be understood as including any medium capable of storing, encoding, or carrying an instruction set for execution by a processor and enabling the processor to perform any of the methods in this invention.

[0113] Those skilled in the art will understand that the modules or steps of the present invention described above can be implemented using general-purpose computer devices. Optionally, they can be implemented using computer-executable program code, thereby allowing them to be stored in a storage device for execution by a computer device, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. The present invention is not limited to any particular combination of hardware and software.

[0114] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A lightweight segmentation method for nerve roots in intraoperative images of PELD surgery, characterized in that, include: Obtain ethically approved raw video frames of intraoperative endoscopy in PELD clinical procedures and perform standardized preprocessing to construct image training samples; A lightweight hybrid segmentation network was built and initialized, including a MiT-B0 lightweight hierarchical encoder, a multi-layer global iterative query refinement decoder, and an iterative region boundary loss calculation unit. Training hyperparameters are pre-configured, and the lightweight hybrid segmentation network is iteratively trained using the image training samples. Each layer of the global iterative query refinement decoder completes three stages of multi-scale feature iterative refinement: coarse alignment, mask purification, and fine correction. The iterative region boundary loss calculation unit performs hierarchical region and edge joint supervision on the output of the multi-layer global iterative query refinement decoder. After each round of training, all network parameters are updated in reverse after forward inference. Continue training until the preset maximum number of iterations is reached or the early stopping condition is met. Save the network weights with the best validation set metrics, load the optimal model to perform inference on real-time intraoperative endoscopic video frames, and output a visualized image of nerve root segmentation.

2. The lightweight nerve root segmentation method for PELD intraoperative images as described in claim 1, characterized in that, The standardized preprocessing includes: The original video frames are uniformly scaled to adjust the resolution; the images are normalized; online augmentation is performed using random scaling, random cropping, horizontal flipping, rotation, brightness shifting, and zero-filling of edges; and the dataset is divided into training, validation, and test sets according to the specified proportions.

3. The lightweight nerve root segmentation method for PELD intraoperative images as described in claim 1, characterized in that, The MiT-B0 lightweight hierarchical encoder downsamples the image training samples four times consecutively and outputs four-layer multi-scale encoder features.

4. The lightweight nerve root segmentation method for PELD intraoperative images as described in claim 1, characterized in that, Each layer of global iterative query refinement decoder independently builds a global feature pool, constructing a bidirectional, multi-scale feature iterative interaction mechanism where high-level semantics are transmitted from top to bottom and shallow refinement features are fed back from bottom to top.

5. The lightweight nerve root segmentation method for PELD intraoperative images as described in claim 1, characterized in that, The pre-configured training hyperparameters include: AdamW optimizer initial learning rate, weight decay coefficient, mask truncation threshold, IRB loss region balancing weights, IRB loss boundary balancing weights, validation interval iterations, early stopping tolerance iterations, and global maximum training iterations.

6. The lightweight nerve root segmentation method for PELD intraoperative images as described in claim 1, characterized in that, In the coarse alignment stage of the global iterative query refinement decoder, the features of the four-layer multi-scale encoder and the upper-layer refinement features are fused to construct a global feature pool. Attention queries, key and value features are generated through scale uniform transformation and linear projection. Coarse localization of the neural root is completed by relying on scaling dot product attention. The mask purification stage of the global iterative query refines the decoder, and parallel multi-scale deep separable convolution extracts contextual features. After concatenation, pixel confidence maps are generated and threshold truncation is performed to filter out fat, instrument background noise and retain weak neural pixels. The global iterative query refines the fine correction stage of the decoder. Based on the mask-purified features, edge enhancement branches are constructed to generate update queries. The secondary cross-attention iteratively corrects the neural distortion contours, and the refined features are fed back into the global feature pool for use by the lower-level decoder.

7. The lightweight nerve root segmentation method for PELD intraoperative images as described in claim 1, characterized in that, The iterative region boundary loss includes a region loss term and a boundary gradient loss term; the region loss term integrates Dice loss and Focal loss to enhance the recognition ability of weak neural networks; the boundary gradient loss term relies on the Sobel operator to extract the ground truth and predicted gradient, and calculates the gradient error only within the effective neural mask; the weighted summation of the coarse and fine prediction results of multi-layer decoding constitutes the overall training loss.

8. A lightweight nerve root segmentation system for intraoperative PELD images, characterized in that, include: The data acquisition module is configured to: acquire ethically approved raw video frames of PELD clinical intraoperative endoscopy and perform standardized preprocessing to construct image training samples; The network construction initialization module is configured to: build and initialize a lightweight hybrid segmentation network, including a MiT-B0 lightweight hierarchical encoder, a multi-layer global iterative query refinement decoder, and an iterative region boundary loss calculation unit; The network training module is configured to: pre-configure training hyperparameters and use the image training samples to iteratively train the lightweight hybrid segmentation network; wherein, each layer of the global iterative query refinement decoder completes three stages of multi-scale feature iterative refinement: coarse alignment, mask purification, and fine correction; the iterative region boundary loss calculation unit performs hierarchical region and edge joint supervision on the output of the multi-layer global iterative query refinement decoder; after each round of training, all network parameters are updated in reverse after forward inference; The intraoperative real-time inference module is configured to: continuously train until the preset maximum number of iterations is reached or the early stopping condition is met, save the network weights with the best validation set metrics, load the optimal model to perform inference on the intraoperative real-time endoscopic video frames, and output a visual image of nerve root segmentation.

9. A computer-readable storage medium having a program stored thereon, characterized in that, When executed by a processor, the program implements the steps in a lightweight segmentation method for nerve roots in PELD intraoperative images as described in any one of claims 1-7.

10. An electronic device comprising a memory, a processor, and a program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps in the lightweight segmentation method for nerve roots in PELD intraoperative images as described in any one of claims 1-7.

Images