A neurosurgical operation scene intelligent understanding method

Abstract

The application discloses a neurosurgery operation scene intelligent understanding method, which is realized through a trained neurosurgery operation scene intelligent understanding network; the network comprises a time coding layer and a space coding layer, and the output end of the space coding layer is connected with at least one fuzzy attention module; the time feature output by the time coding layer and the fuzzy space feature output by the fuzzy attention module sequentially pass through the cross-modal fusion operation of a decoding layer and the dimension reduction processing of a feedforward neural network, and output an operation stage recognition prediction result; the fuzzy attention module evaluates and quantifies the reliable degree of the space visual feature at each position in the feature sequence output by the space coding layer, dynamically adjusts the attention weight and gives the space visual feature to form a fuzzy space feature; the method effectively improves the recall rate and Jaccard coefficient while maintaining high accuracy and high precision, and proves that the method has the effect of significantly optimizing the fuzzy recognition and prediction accuracy.

Description

Technical Field

[0001] This invention relates to the field of medical image segmentation technology, and in particular to an intelligent understanding method for neurosurgical surgical scenes. Background Technology

[0002] Accurate surgical stage identification is crucial for improving real-time cognitive assistance in minimally invasive surgery. Surgical procedures consist of a series of continuous and logically rigorous stages, with significant differences in surgical instruments, maneuvers, and scenarios at each stage. To improve the consistency and efficiency of surgical stage analysis, intelligent understanding methods for neurosurgical surgical scenarios have been continuously explored, especially with the introduction of deep learning technology. These methods learn the visual features of surgical instruments, tissue morphology, and surgeon maneuvers from video frame sequences and understand the contextual relationships between frames through temporal modeling, enabling the construction of neural network models to identify the current surgical stage. Such systems aim to help clinicians achieve objective and structured analysis of surgical procedures. However, in practical applications, surgical scenarios present challenges such as instrument occlusion, changing perspectives, lighting differences, and diverse surgeon operating habits. Coupled with the high timeliness requirements for real-time identification, the robustness and generalization ability of the models in complex scenarios face challenges, making it difficult to stably and reliably guide surgical process analysis and provide real-time assistance in real clinical environments.

[0003] The application of deep learning in surgery is primarily built upon the interdisciplinary advancements in computer vision, deep learning, and medical image analysis. In recent years, with the maturation of convolutional neural networks (CNNs), temporal models (such as LSTM, TCN, and Transformer), and multimodal learning methods, computer vision has become capable of identifying complex stages and diverse instruments in high-resolution videos in real time. In surgical scene analysis, these technologies can be used to identify the types of surgical instruments and the surgical process, thereby achieving structured analysis of the surgical procedure and helping doctors better learn and analyze the surgical process. Simultaneously, the vast amount of medical data currently available provides a crucial foundation for artificial intelligence models. Based on this, the accuracy of models has significantly improved with the support of various methods. In this technological environment, efficient medical image analysis methods and systems can be built, providing doctors with real-time surgical data and information, ensuring surgical safety, and providing subsequent medical researchers with exemplary data materials.

[0004] Previously, the application of artificial intelligence in assisting medical procedures was not fully developed, and its clinical support role was still unknown. Traditional neural network models mostly focused on recognizing single images, but in a surgical procedure, every step, even every second of movement, is interconnected. Clearly, traditional deep learning models for single scenes are no longer suitable for such an environment where every step is interdependent. With the emergence of neural network structures such as LSTM and TCN, more researchers can train models on a set of images with a temporal logical relationship, allowing the trained models to more accurately identify surgical stages based on the temporal and spatial logical connections between surgical actions. Early research primarily employed model-based and feature-based machine learning and statistical methods, such as Support Vector Machines, Random Forests, and Hidden Markov Models. While these methods achieved some success, their performance heavily relied on expert experience and extensive annotation, lacking universality and adaptability in complex and ever-changing surgical scenarios. Subsequently, Long Short-Term Memory (LSTM) networks, due to their advantages in modeling sequential data, gradually became the mainstream technology for surgical stage recognition. Representative works include: Twinanda et al. extracted frame-by-frame visual features by integrating ResNet-50 and used a two-layer bidirectional LSTM to model temporal dependencies, thus constructing the first CNN-bidirectional LSTM collaborative recognition framework; Lee et al. proposed a two-stream CNN-LSTM model that utilizes both local segment temporal information and global temporal context, achieving better performance than the traditional single-stream architecture; Jin et al. introduced a multi-kernel temporal convolution module as an external memory unit to extract multi-scale long-term dependencies, but the simplified weighted fusion mechanism they adopted reduced computational complexity while also weakening the ability to model global context.

[0005] With the success of Transformer in natural language processing, its self-attention mechanism has provided a new approach for long-term temporal modeling. However, there is a significant contradiction between the computational complexity of the standard Transformer and long surgical videos, so most studies adopt segmented sampling to balance efficiency and feature integrity. In this direction, Yue et al. designed a cascaded multi-stage Transformer architecture, which improves the temporal coherence of stage recognition by achieving frame-level feature distillation and stage-level semantic fusion through an adaptive context aggregation module; Tao et al. proposed a multi-task driven structure-aware Transformer, which combines latent space learning constrained by variational autoencoders to achieve joint semantic modeling of surgical stages and instrument usage, thereby supporting hierarchical video structure understanding. In recent years, in order to more accurately capture fine-grained changes during stage transitions, hybrid models have begun to attempt to fuse local action features with global stage semantics. Jin et al. used Transformer to fuse multi-scale temporal representations and controlled the computational burden through uniform subsampling; Gao et al. used temporal convolutional networks to extract long-range dependencies and then used Transformer to achieve spatiotemporal feature fusion; Liu et al. developed a temporally enhanced spatial feature extractor and combined it with a stage transition graph and a multi-scale temporal aggregation module to enhance the interaction between local and global features.

[0006] However, even with temporal and spatial features, models trained using CNN networks by extracting specific images from a video frame by frame still suffer from significant noise interference. For example, camera motion blur can cause pixel blur and ghosting when doctors use instruments. This phenomenon is inevitable, primarily because current cameras are not yet adept at capturing micro-movements. This leads to substantial interference during ResNet network visual training. Therefore, given the inability to solve the image blur problem, accurately training on blurred elements becomes crucial for improving model accuracy. Consequently, it is necessary to design a novel neural network model for image processing to enhance the network's ability to perceive and process blurred and uncertain features in surgical scene images, thereby improving the accuracy and robustness of surgical stage recognition in complex surgical environments. Summary of the Invention

[0007] The purpose of this invention is to provide an intelligent understanding method for neurosurgical surgical scenarios that solves the above-mentioned technical problems.

[0008] Therefore, the technical solution of the present invention is as follows:

[0009] A method for intelligent understanding of neurosurgical surgical scenarios is provided, which is implemented through a trained intelligent understanding network for neurosurgical surgical scenarios. The intelligent understanding network for neurosurgical surgical scenarios includes a temporal coding layer and a spatial coding layer, and the output of the spatial coding layer is connected to at least one fuzzy attention module. The temporal features output by the temporal coding layer and the fuzzy spatial features output by the fuzzy attention module are sequentially processed by cross-modal fusion operation of the decoding layer and dimensionality reduction processing of the feedforward neural network to output the surgical stage recognition prediction result. Among them, the fuzzy attention module dynamically adjusts the attention weights and assigns them to the spatial visual features by evaluating and quantifying the reliability of the spatial visual features at each position in the feature sequence output by the spatial coding layer, so as to form fuzzy spatial features.

[0010] Furthermore, the fuzzy attention module processes the spatial visual features at each location of the spatial coding layer output as follows:

[0011] 1) Spatial visual features are fed into multiple independent linear projection layers to generate multiple query vectors in parallel; through a multi-head attention mechanism, each query vector is reshaped into a multi-head query vector feature.

[0012] 2) Spatial visual features simultaneously obtain keys through a shared key-value projection layer, and through a multi-head attention mechanism, the keys are reshaped into multi-head key features;

[0013] 3) For each query vector, calculate the similarity matrix between the query vector features and the key features, and normalize it using the Softmax function to obtain the basic attention weights; calculate the mean values ​​of the query vector features and the key features in each attention head, and concatenate them into a concatenated feature.

[0014] 4) The concatenated features of each query vector are fed into a fuzzy decision network consisting of a first fully connected layer, a second fully connected layer, and a Sigmoid activation function connected in sequence to generate a soft decision value between 0 and 1.

[0015] 5) Multiply the base attention weight of each query vector by its soft decision value to obtain the processed features;

[0016] 6) The processed features of multiple query vectors are concatenated and fused to obtain fused features;

[0017] 7) Perform residual connection and layer normalization on the fusion features, and then add them to the spatial visual features to form fuzzy spatial features.

[0018] Furthermore, in the fuzzy attention module, the number of linear projection layers is set to 2, and the multi-head attention mechanism adopts an octa-head attention mechanism.

[0019] Furthermore, in the fuzzy attention module, the fused features are obtained by sequentially performing linear transformation, layer normalization, ReLU activation function, and Dropout operation on the processed features of multiple query vectors.

[0020] Furthermore, the intelligent understanding network for neurosurgical surgical scenarios also includes a spatial constraint masking module to obtain soft decision values ​​based on spatial locality priors for correction. The processing steps are as follows:

[0021] 1) Based on time series features, generate sequence indices for the time features to represent the position of each time feature in the time feature sequence; for each time feature, calculate the normalized absolute distance between its sequence indices and those of other time features.

[0022] 2) Set a distance threshold and sequentially determine the relationship between the absolute distance and the distance threshold to generate a binary mask matrix as a spatial constraint mask; where, when the absolute distance ≤ the distance threshold, the mask value is 1, indicating that attention interaction is allowed; when the absolute distance > the distance threshold, the mask value is 0, indicating that the remote location is blocked.

[0023] 3) Expand the spatial constraint mask to the same size as the similarity matrix, and then combine it with the similarity matrix through the Masked_Fill function to obtain a new spatial constraint mask;

[0024] 4) Replace the similarity matrix calculation results corresponding to the positions with mask values ​​of 0 in the new spatial constraint mask with -1e. 9 .

[0025] Furthermore, the intelligent understanding network for neurosurgical surgical scenarios also includes a preprocessing module to preprocess the spatial and temporal information in the surgical phase dataset, including:

[0026] 1) Extract spatial information from the surgical stage image dataset. First, reduce the dimension of spatial visual features through a fully connected layer. Then, construct a spatial feature sequence with 10 spatial locations through a sliding window and input it into the spatial coding layer.

[0027] 2) Extract the time information from the surgical stage image dataset. First, transpose the first and second dimensions, and then construct a time feature sequence with 30 time positions through a sliding window and input it into the time coding layer.

[0028] Furthermore, the decoding layer consists of a first fusion module, a first Softmax function, a second fusion module, and a second Softmax function connected in sequence. The input of the first fusion module is connected to the output of the fuzzy attention module and the temporal coding layer, respectively, and the second fusion module is also connected to the output of the temporal coding layer. Both the first and second fusion modules use the temporal feature sequence as the key and value, and the spatial feature sequence or the fused feature sequence as the query, and complete the fusion through the encoder-decoder attention mechanism to output the fused feature sequence.

[0029] Furthermore, the feedforward neural network consists of a fully connected projection layer, a ReLU activation function, and a Dropout layer connected in sequence, and the output feature dimension is set to N, which is specifically set to be consistent with the number of surgical stage categories to be identified according to the network training purpose.

[0030] Compared with existing technologies, this intelligent understanding method for neurosurgical surgical scenes effectively mitigates recognition errors caused by the inability of images in the dataset to be completely clear by improving the network's ability to process motion blur and noise in blurred images during the training process. Testing shows that this method effectively improves recall and Jaccard coefficient while maintaining high accuracy and precision, demonstrating that it significantly optimizes blur recognition and prediction accuracy. This verifies that the method significantly improves the recognition process of dynamic and behavioral blur in surgical records, and shows great promise for clinical applications involving motion blur. Attached Figure Description

[0031] Figure 1 This is a flowchart of the intelligent understanding method for neurosurgical surgical scenarios in an embodiment of the present invention;

[0032] Figure 2 This is a schematic diagram of the structure of the intelligent understanding network for neurosurgical surgical scenarios in an embodiment of the present invention. Detailed Implementation

[0033] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, but the following embodiments are by no means intended to limit the present invention.

[0034] See Figure 1 The specific implementation steps of this intelligent understanding method for neurosurgical surgical scenarios are described below.

[0035] Step 1: Construct an intelligent understanding network for neurosurgical surgical scenarios.

[0036] To address the challenge of handling ambiguity in surgical scene recognition networks, an intelligent understanding network for neurosurgical surgical scenarios was first constructed. This network comprises a temporal coding layer, a spatial coding layer, a fuzzy attention module, a decoding layer, and a feedforward neural network. In terms of module structure design, the temporal coding layer, spatial coding layer, decoding layer, and feedforward neural network all utilize existing structures, which can be referenced in the deep learning model AI-Endo. [7] .

[0037] The temporal encoding layer and spatial encoding layer are configured in parallel to output temporal and spatial features, respectively. Specifically, the temporal encoding layer is an encoding layer configured with an eight-head attention mechanism to model the temporal dependencies of a 512×30×5 temporal feature sequence, and its output also has a feature dimension of 512×30×5. The spatial feature encoding layer is also an encoding layer configured with an eight-head attention mechanism to model the temporal dependencies of a 512×10×5 spatial feature sequence, and its output also has a feature dimension of 512×10×5. The temporal and spatial encoding layers are configured in parallel and are connected to the output of the preprocessing module, respectively, to output temporal and spatial features.

[0038] The dataset includes a preprocessing module preceding the temporal and spatial coding layers to preprocess the spatial and temporal information in the surgical phase dataset, respectively. This preprocessing module can utilize, but is not limited to, ResNet50.

[0039] The preprocessing steps for spatial information are as follows: Based on the original input tensor size of the spatial information, the spatial visual features are first reduced in dimensionality and projected through a fully connected layer to shrink the dimensionality of the spatial visual features. Then, a spatial feature sequence is constructed using a sliding window. In this embodiment, the original input tensor size of the spatial information is 1×512×2048. It is first processed through a fully connected layer to reduce the output size to 1×512×5, and then the spatial feature sequence is constructed to make the output size 512×10×5.

[0040] The preprocessing steps for time information are as follows: Based on the original input tensor size of the time information, the first two dimensions are first transformed by transpose, and then a time feature sequence is constructed using a sliding window. In this embodiment, the original input tensor size of the time information is 1×5×512. It is first transposed to change the tensor size to 1×512×5, and then the time feature sequence is constructed, resulting in an output size of 512×30×5.

[0041] It should be noted that in the above preprocessing process, setting the length of the spatial feature sequence to 10 spatial locations is an optimized configuration verified by experiments. Specifically, when the spatial sequence length is less than 10, the spatial context information is insufficient, leading to a decrease in the network's robustness to locally blurred features; when the spatial sequence length is greater than 10, the computational complexity increases significantly while the performance improvement is limited. Similarly, setting the length of the temporal feature sequence to 30 temporal locations is also an optimized configuration verified by experiments. Specifically, when the time sequence length is less than 30, the temporal context information is insufficient, affecting the accurate identification of stage transitions; when the time sequence length is greater than 30, although it can capture longer temporal dependencies, training stability and convergence speed will be negatively affected. Therefore, the configuration described in this embodiment achieves the best balance between performance and efficiency.

[0042] The fuzzy attention module is a newly designed module that dynamically adjusts attention weights by evaluating and quantifying the reliability of spatial visual features at each location in the feature sequence. Before feature fusion, it "differentiates" spatial features at different locations to emphasize the focus on clear, key features during network training and suppress interference from fuzzy, noisy features. Furthermore, when it is assumed that features at different locations have different reliability for surgical stage identification, the fuzzy query attention module selects features beneficial for stage discrimination and suppresses unreliable features. This improves the robustness and accuracy of stage identification in surgical images with interference such as motion blur and tissue occlusion.

[0043] Specifically, in this embodiment, two fuzzy attention modules are configured, that is, two fuzzy attention modules are sequentially connected from the output of the spatial coding layer. The first fuzzy attention module processes the spatial visual features output by the spatial coding layer, while the second fuzzy attention module processes the fuzzy spatial features output by the first fuzzy attention module, so as to further improve the robustness to fuzzy features.

[0044] The specific processing steps of the fuzzy attention module for the spatial visual features at each location in the output of the spatial coding layer are as follows:

[0045] 1) Spatial visual features are fed into two independent linear projection layers to generate two query vectors (Q) in parallel, each representing a potential feature interaction pattern; then, the two query vectors are reshaped into two query vector features in the form of eight heads through an eight-head attention mechanism.

[0046] 2) Spatial visual features are simultaneously converted into keys (K) and values ​​(V) through a shared key-value projection layer, and the keys are reshaped into key features in the form of eight heads through an octagonal attention mechanism;

[0047] 3) For each query vector generated in step 1), calculate the similarity matrix between the query vector features and the key features, and normalize it using the Softmax function to obtain the basic attention weights; then, calculate the mean values ​​of the query vector features and the key features in each attention head, and concatenate them to obtain the concatenated features; in this embodiment, the similarity matrix is ​​calculated by scaling the dot product.

[0048] 4) The concatenated features of each query vector are fed into a fuzzy decision network consisting of a first fully connected layer, a second fully connected layer, and a Sigmoid activation function connected in sequence to generate a soft decision value that represents the credibility of the spatial visual features. The soft decision value is a value between 0 and 1, where a soft decision value of 0 indicates that it is completely untrustworthy and a soft decision value of 1 indicates that it is completely trustworthy.

[0049] 5) Multiply the basic attention weight of each query vector by its soft decision value to form a processed feature that strengthens attention to high-confidence features and suppresses low-confidence features;

[0050] 6) The two processed features are concatenated and fused to obtain the fused features; in this embodiment, the concatenation and fusion operation specifically involves performing four steps on multiple processed features in sequence: linear transformation, layer normalization, ReLU activation function and Dropout operation;

[0051] 7) Perform residual connection and layer normalization on the fusion features, and then add them to the spatial visual features to form fuzzy spatial features, which replace the initial spatial visual features and are used for subsequent cross-modal fusion and other operations with temporal features.

[0052] Preferably, the intelligent understanding network for neurosurgical scenarios also includes a spatial constraint mask module, which constructs a spatial constraint mask based on sequence position distance to simulate the spatial locality prior of image data, and corrects the soft decision value obtained in step 4) based on the spatial locality prior.

[0053] Specifically, the spatial constraint mask module performs the following steps to correct soft decisions:

[0054] 1) Based on time series features, generate sequence indices for the time features to represent the position of each time feature in the time feature sequence; for each time feature, calculate the normalized absolute distance between its sequence indices and those of other time features.

[0055] 2) Set a distance threshold and sequentially determine the relationship between the absolute distance and the distance threshold to generate a binary mask matrix as a spatial constraint mask; where, when the absolute distance ≤ the distance threshold, the mask value is 1, indicating that attention interaction is allowed; when the absolute distance > the distance threshold, the mask value is 0, indicating that the remote location is blocked.

[0056] 3) Expand the spatial constraint mask to the same size as the similarity matrix, and then combine it with the similarity matrix through the Masked_Fill function to obtain a new spatial constraint mask;

[0057] 4) Replace the similarity matrix calculation results corresponding to the positions with mask values ​​of 0 in the new spatial constraint mask with -1e. 9 After Softmax normalization in step 3) of the fuzzy attention module, the basic attention weights at these locations approach zero, thereby achieving local spatial constraints.

[0058] This spatial constraint masking module ensures that the receptive field of each feature location is effectively limited to the local neighborhood when calculating attention, significantly reducing the computational complexity of the network. More importantly, when processing blurred image features, it forces the network to prioritize and integrate feature information from spatially adjacent locations, thereby providing an attention basis that conforms to the spatial topology of the image for the subsequent blurred decision network, enhancing the network's ability to make reliable inferences using local context in blurred regions.

[0059] Unlike previous methods that indiscriminately trained images based on temporal relationships across all contexts, this invention designs and adds a fuzzy attention module. This allows the network to first generate a soft decision value by statistically analyzing query and key features within multi-head attention. This soft decision value is then used to weight spatial information, enabling targeted training on spatial features and allowing the network to more clearly and comprehensively understand the spatial characteristics of a single image. Furthermore, during training, each image is assigned a fuzzy weight parameter, which is incorporated into the calculation of spatial information channels. This allows the network to analyze the importance of a single image within the entire dataset. Specifically, if an image is very blurry, the network assigns a smaller weight to dilute its impact on the overall training, effectively filtering out blurry data and making the algorithm more efficient and accurate.

[0060] The decoding layer consists of a first fusion module, a first Softmax function, a second fusion module, and a second Softmax function connected sequentially. The input of the first fusion module is connected to the output of both the fuzzy attention module and the temporal coding layer, and the second fusion module is also connected to the output of the temporal coding layer. Specifically, the cross-modal fusion operation of the first and second fusion modules is as follows: using the temporal feature sequence as the key (K) and value (V), and the spatial feature sequence or the output of the first Softmax function as the query (Q), fusion is completed through an encoder-decoder attention mechanism, and the fused feature sequence is output. In this embodiment, the size of the fused feature sequence is 512×10×5. This decoding layer improves the temporal coherence and spatial accuracy of stage recognition by performing cross-modal fusion of temporal features and fuzzy spatial features.

[0061] The feedforward neural network consists of a fully connected projection layer, a ReLU activation function, and a Dropout layer connected sequentially. The output feature dimension is set to N, which is specifically set to match the number of surgical stage categories to be identified, depending on the network training objective. In this embodiment, since the surgical stage dataset used in the network training process contains 4 surgical stage categories, N is also set to 4. This allows the fused feature sequence to pass through the feedforward neural network, mapping the feature dimension from 5 to 4, resulting in an output size of 512×10×4. Furthermore, the output result of the last time step is extracted from the feature sequence output by the feedforward neural network and used as the network prediction result, thus completing the entire surgical stage identification process.

[0062] Step 2: Create a dataset of surgical stage images and train the intelligent understanding network for neurosurgical surgical scenes constructed in Step 1.

[0063] In this embodiment, the surgical stage image dataset is derived from the Multi-center Neurosurgery Standardized Procedural Case Library (MNSPC Library), specifically selecting and using 29,510 images from the dataset; this dataset includes four surgical stage categories, and the corresponding output feature dimension in the aforementioned feedforward neural network is set to N=4.

[0064] Before training the intelligent understanding network for neurosurgical surgical scenarios, the images in the surgical stage image dataset are randomly selected and divided into training images, validation images, and test images according to the KFold method.

[0065] The training set described above is input into the neurosurgical scene intelligent understanding network. The network is trained using labeled images from the surgical stage image dataset as input and image segmentation information as output. Simultaneously, during training, ablation experiments are performed on the network after each training round using a validation set. Finally, the weights corresponding to the round with the best recognition accuracy during training are selected as the optimal weights and saved, completing the training. In this embodiment, the training parameters are shown in Table 1 below, and the cross-entropy loss function is used.

[0066] Table 1:

[0067] Parameter name Parameter value Batch size 1 Sequence length 30 Learning rate 1e-2 Weight decay 5e-3 Total number of iterations 4000 Learning rate decay steps [1500, 2500]

[0068] Step 3: Input the collected images of the surgical stages into the neurosurgical scene intelligent understanding network trained in Step 2, and the network will output accurate prediction results of the surgical stages.

[0069] In step S3, the surgical stage image should be consistent with the surgical stage type contained in the surgical stage image data used for training.

[0070] To further verify the effectiveness of the method of this invention in the identification task during neurosurgical procedures, its identification performance was quantitatively evaluated. Specifically, accuracy, precision, recall, and Jaccard coefficient were used, and a comprehensive calculation was performed based on True Positive (TP), True Negative (TN), False Positive (FP), and False Negative (FN) to mitigate the evaluation bias that may be caused by the uneven distribution of time duration in different stages, and to ensure a balanced and robust measurement of the identification performance at each stage.

[0071] Precision reflects the proportion of samples correctly identified by the network as belonging to a specific surgical stage. This metric focuses on the model's accuracy in classifying positive samples. A higher precision indicates fewer instances where the model misclassifies other stages as that stage, meaning a lower false positive (FP) rate. The formula for calculating precision is:

[0072] ,

[0073] Recall measures a model's ability to correctly identify samples that actually belong to a specific surgical stage. It focuses on whether the model can adequately capture the target stage. A higher recall indicates fewer true stage samples missed by the model, meaning a lower false negative (FN) rate. The formula for recall is:

[0074] ,

[0075] The Jaccard coefficient evaluates model performance by calculating the overlap ratio between predicted and actual stage segments. This metric is particularly focused on the accuracy of stage boundary identification. A higher Jaccard coefficient indicates that the model's predicted stage start and end times are closer to the actual situation. Its calculation formula is as follows:

[0076] ,

[0077] Accuracy represents the proportion of times a model makes a correct prediction across all samples. It is used to comprehensively evaluate the model's overall performance in identifying different surgical stages. A higher accuracy typically indicates that the model can output the correct stage classification result in most cases. The formula for calculating accuracy is:

[0078] ,

[0079] Meanwhile, to verify the superiority of the method of this application compared with the prior art, while evaluating the method of this invention using the above-mentioned indicators, other prior art methods were also evaluated using the same indicators, and a comparative analysis was formed. The prior art methods used for comparison are several currently recognized state-of-the-art surgical stage recognition algorithms, including TeCNO, TMRNet, Trans-SVNet, CMTNet, LAST, and LoViT.

[0080] The specific test results are shown in Table 2 below.

[0081] Table 2:

[0082] Test methods accuracy Accuracy Recall rate Jaccard coefficient <![CDATA[TeCNO [1] ]]> 88.93±7.7 78.4±6.7 77.6±5.6 63.7±9.7 <![CDATA[TMRNet [2] ]]> 89.26±9.1 80.31±8.9 78.5±10.1 67.82±7.6 <![CDATA[Trans-SVNet [3] ]]> 90.09±7.1 82.26±8.7 79.53±7.7 68.96±6.3 <![CDATA[CMTNet [4] ]]> 88.76±6.3 81.77±5.1 79.23±7.2 68.42±6.9 <![CDATA[LAST [5] ]]> 90.36±6.9 85.64±5.9 82.81±4.1 76.68±7.2 <![CDATA[LoViT [6] ]]> 91.35±5.7 92.67±8.2 88.64±5.7 82.26±6.8 <![CDATA[AI-endo [7] ]]> 90.23±6.4 91.38±7.4 78.84±8.9 78.20±7.7 This invention 90.23±6.1 90.20±4.8 90.23±5.7 82.51±4.9

[0083] As shown in Table 2, the improved neurosurgical scene intelligent understanding network obtained in this invention, after training, achieves accuracy and precision comparable to the highest-level surgical stage recognition algorithms currently available. Furthermore, its recall and Jaccard coefficient are the best among these state-of-the-art algorithms. Compared to AI-endo, this invention maintains comparable accuracy and precision while further improving recall and Jaccard coefficient. This performance improvement is primarily attributed to the model's ability to handle motion blur and noise in blurred images within the dataset, significantly optimizing blur recognition and prediction accuracy. This demonstrates that the method of this invention effectively improves the recognition process of dynamic and behavioral blur generated in surgical records. Moreover, after complete training, this method can achieve a processing speed of 30 FPS during the inference phase, meeting the requirements of real-time analysis.

[0084] References:

[0085] [1]T. Czempiel et al., "TeCNO: Surgical Phase Recognition with Multi-stage Temporal Convolutional Networks," in Medical Image Computing and Computer Assisted Intervention – MICCAI 2020, Cham, 2020, pp. 343-352: Springer International Publishing.

[0086] [2]Y. Jin, Y. Long, C. Chen, Z. Zhao, Q. Dou and P.-A. Heng, "Temporal memory relation network for workflow recognition from surgicalvideo", IEEE Trans. Med. Imag., vol. 40, no. 7, pp. 1911-1923, Jul. 2021.

[0087] [3]Y. Jin, Y. Long, X. Gao, D. Stoyanov, Q. Dou, and P.-A. Heng, "Trans-SVNet: hybrid embedding aggregation Transformer for surgical workflowanalysis," International Journal of Computer Assisted Radiology and Surgery,vol. 17, no. 12, pp. 2193-2202, 2022.

[0088] [4]W. Yue, H. Liao, Y. Xia, V. Lam, J. Luo, and Z. Wang, "CascadeMulti-Level Transformer Network for Surgical Workflow Analysis," IEEETransactions on Medical Imaging, vol. 42, no. 10, pp. 2817-2831, 2023.

[0089] [5]R. Tao, X. Zou, and G. Zheng, "LAST: LAtent Space-ConstrainedTransformers for Automatic Surgical Phase Recognition and Tool PresenceDetection," IEEE Transactions on Medical Imaging, vol. 42, no. 11, pp. 3256-3268, 2023.

[0090] [6]Y. Liu et al., "LoViT: Long Video Transformer for surgical phaserecognition," Medical Image Analysis, vol. 99, p. 103366, 2025

[0091] [7]J. Cao et al., "Intelligent surgical workflow recognition forendoscopic submucosal dissection with real-time animal study," NatureCommunications, vol. 14, no. 1, p. 6676, 2023。

Claims

1. A method for intelligent understanding of neurosurgical surgical scenarios, characterized in that, It is achieved through a trained intelligent understanding network for neurosurgical surgical scenarios. The intelligent understanding network for neurosurgical surgical scenarios includes a temporal coding layer and a spatial coding layer. The output of the spatial coding layer is connected to at least one fuzzy attention module. The temporal features output by the temporal coding layer and the fuzzy spatial features output by the fuzzy attention module are sequentially processed by cross-modal fusion operation of the decoding layer and dimensionality reduction processing of the feedforward neural network to output the surgical stage recognition prediction result. Among them, the fuzzy attention module dynamically adjusts the attention weight and assigns it to the spatial visual features by evaluating and quantifying the reliability of the spatial visual features at each position in the feature sequence output by the spatial coding layer, so as to form fuzzy spatial features.

2. The intelligent understanding method for neurosurgical surgical scenarios according to claim 1, characterized in that, The fuzzy attention module processes the spatial visual features at each location in the output of the spatial coding layer in the following steps: 1) Spatial visual features are fed into multiple independent linear projection layers to generate multiple query vectors in parallel; through a multi-head attention mechanism, each query vector is reshaped into a multi-head query vector feature. 2) Spatial visual features simultaneously obtain keys through a shared key-value projection layer, and through a multi-head attention mechanism, the keys are reshaped into multi-head key features; 3) For each query vector, calculate the similarity matrix between the query vector features and the key features, and normalize it using the Softmax function to obtain the basic attention weights; Calculate the mean values ​​of query vector features and key features in each attention head separately, and then concatenate them into a concatenated feature; 4) The concatenated features of each query vector are fed into a fuzzy decision network consisting of a first fully connected layer, a second fully connected layer, and a Sigmoid activation function connected in sequence to generate a soft decision value between 0 and 1. 5) Multiply the base attention weight of each query vector by its soft decision value to obtain the processed features; 6) The processed features of multiple query vectors are concatenated and fused to obtain fused features; 7) Perform residual connection and layer normalization on the fusion features, and then add them to the spatial visual features to form fuzzy spatial features.

3. The intelligent understanding method for neurosurgical surgical scenarios according to claim 2, characterized in that, In the fuzzy attention module, the number of linear projection layers is set to 2, and the multi-head attention mechanism adopts an octa-head attention mechanism.

4. The intelligent understanding method for neurosurgical surgical scenarios according to claim 2, characterized in that, In the fuzzy attention module, the fused features are obtained by sequentially performing linear transformation, layer normalization, ReLU activation function, and Dropout operation on the processed features of multiple query vectors.

5. The intelligent understanding method for neurosurgical surgical scenarios according to claim 2, characterized in that, The intelligent understanding network for neurosurgical surgical scenarios also includes a spatial constraint masking module to obtain soft decision values ​​based on spatial locality priors for correction. The processing steps are as follows: 1) Based on time series features, generate sequence indices for the time features to represent the position of each time feature in the time feature sequence; for each time feature, calculate the normalized absolute distance between its sequence indices and those of other time features. 2) Set a distance threshold and sequentially determine the relationship between the absolute distance and the distance threshold to generate a binary mask matrix as a spatial constraint mask; where, when the absolute distance ≤ the distance threshold, the mask value is 1, indicating that attention interaction is allowed; when the absolute distance > the distance threshold, the mask value is 0, indicating that the remote location is blocked. 3) Expand the spatial constraint mask to the same size as the similarity matrix, and then combine it with the similarity matrix through the Masked_Fill function to obtain a new spatial constraint mask; 4) Replace the similarity matrix calculation results corresponding to the positions with mask values ​​of 0 in the new spatial constraint mask with -1e. 9 .

6. The intelligent understanding method for neurosurgical surgical scenarios according to claim 1, characterized in that, The intelligent understanding network for neurosurgical surgical scenarios also includes a preprocessing module to preprocess the spatial and temporal information in the surgical phase dataset, including: 1) Extract spatial information from the surgical stage image dataset. First, reduce the dimension of spatial visual features through a fully connected layer. Then, construct a spatial feature sequence with 10 spatial locations through a sliding window and input it into the spatial coding layer. 2) Extract the time information from the surgical stage image dataset. First, transpose the first and second dimensions, and then construct a time feature sequence with 30 time positions through a sliding window and input it into the time coding layer.

7. The intelligent understanding method for neurosurgical surgical scenarios according to claim 1, characterized in that, The decoding layer consists of a first fusion module, a first softmax function, a second fusion module, and a second softmax function connected in sequence. The input of the first fusion module is connected to the output of the fuzzy attention module and the temporal coding layer, respectively, and the second fusion module is also connected to the output of the temporal coding layer. Both the first and second fusion modules use the temporal feature sequence as the key and value, and the spatial feature sequence or the fused feature sequence as the query. They complete the fusion through an encoder-decoder attention mechanism to output the fused feature sequence.

8. The intelligent understanding method for neurosurgical surgical scenarios according to claim 1, characterized in that, The feedforward neural network consists of a fully connected projection layer, a ReLU activation function, and a Dropout layer connected in sequence, and the output feature dimension is set to N. The specific value of N is set according to the network training purpose and is consistent with the number of surgical stage categories that need to be identified.

Images