Surgical phase recognition method, apparatus, electronic device, and readable storage medium

Abstract

The present disclosure relates to a surgical stage recognition method and device, electronic equipment and readable storage medium. The spatial features of the surgical image data are extracted based on a spatial feature extraction network to obtain a spatial feature vector, the temporal features of the surgical image data are extracted based on a temporal feature extraction network to obtain a temporal feature vector, the spatial feature vector and the temporal feature vector are fused to obtain a target fusion feature vector, the surgical stage recognition is performed based on a target recognition network and the target fusion feature vector to recognize the surgical stage corresponding to the surgical image data. Compared with the prior art, the embodiments of the present disclosure can effectively capture the details of the key parts and operation steps in the surgical process, effectively capture the time dependence of the actions and steps in the surgical process, improve the timing accuracy of the surgical stage recognition, realize accurate classification and recognition of the surgical stage through spatio-temporal feature fusion, and improve the intelligent level of the surgical process monitoring and guidance.

Description

[0001] This disclosure claims priority to Chinese Patent Application No. 202511004308.8, filed on July 21, 2025, entitled "Surgical Stage Identification Method, Apparatus, Electronic Device and Readable Storage Medium", the entire contents of which are incorporated herein by reference. Technical Field

[0002] This disclosure relates to the field of surgical assistance technology, and in particular to a surgical stage identification method, device, electronic device, and readable storage medium. Background Technology

[0003] Laparoscopic cholecystectomy (LC) is a routine surgical procedure for treating gallbladder diseases, especially gallstones and cholecystitis. Compared to traditional open surgery, laparoscopic surgery is performed through several small incisions, which not only reduces postoperative pain and shortens recovery time but also lowers the incidence of complications such as infection and bleeding. However, despite the many advantages of laparoscopic techniques, the complexity of the surgery and the skill level required of the surgeon remain high, especially in cases with complex anatomical structures and multiple comorbidities, where the surgical risks and postoperative complication rates are higher.

[0004] Traditional laparoscopic cholecystectomy typically involves several stages: dissecting the Calot's triangle, transection of the cystic artery, transection of the cystic duct, and freeing the gallbladder bed. Each stage requires precise surgical technique and close monitoring. Currently, during the procedure, surgeons primarily rely on their experience and intraoperative imaging to determine the surgical stage and then proceed with the operation.

[0005] However, traditional methods for determining surgical stages have certain limitations: 1) Reliance on experience: The surgeon's experience and skill level greatly influence the success rate of the surgery, and novice surgeons are prone to misjudgment and errors during the procedure; 2) Difficulty in intraoperative monitoring: During the surgery, the surgeon needs to view the surgical site through a monitor, and the difficulty of the operation and the complexity of intraoperative monitoring increase the workload; 3) High operational risks: Complex anatomical and clinical variations may lead to prolonged operation time, increasing the patient's anesthesia risks and postoperative recovery time, and easily causing postoperative complications. Therefore, there is an urgent need for a surgical stage identification method to accurately identify the surgical stage and assist the surgeon in making precise operations. Summary of the Invention

[0006] In order to solve the above-mentioned technical problems, or at least partially solve the above-mentioned technical problems, this disclosure provides a surgical stage identification method, device, electronic device and readable storage medium.

[0007] In a first aspect, embodiments of this disclosure provide a method for identifying surgical stages, the method comprising: Spatial feature vectors are obtained by extracting spatial features from surgical image data using a spatial feature extraction network. The temporal features of the surgical image data are extracted using a temporal feature extraction network to obtain a temporal feature vector; The spatial feature vector and the temporal feature vector are fused to obtain the target fused feature vector; Based on the target recognition network and the target fusion feature vector, the surgical stage is identified, and the surgical stage corresponding to the surgical image data is identified.

[0008] In some embodiments, the method further includes: Obtain a pre-trained spatial feature extraction model and a pre-trained temporal feature extraction model; The pre-trained spatial feature extraction model is optimized and trained to obtain a trained spatial feature extraction network; The pre-trained time feature extraction model is optimized and trained to obtain a well-trained time feature extraction network.

[0009] In some embodiments, optimizing the pre-trained spatial feature extraction model to obtain a trained spatial feature extraction network includes: Multiple sample image frames were extracted from the training sample image data, and each sample image frame was labeled with the corresponding surgical stage label. Each sample image frame is preprocessed to obtain a preprocessed sample image frame. The preprocessing operations include image scaling, random cropping, random rotation, color dithering, and pixel normalization. A fully connected layer is added to the pre-trained spatial feature extraction model, and the parameters of the target convolutional layer in the pre-trained spatial feature extraction model are frozen to obtain the spatial feature extraction network. Set the optimizer, loss function, and training strategy for the spatial feature extraction network; Based on each preprocessed sample image frame, the spatial feature extraction network is trained to obtain a trained spatial feature extraction network.

[0010] In some embodiments, optimizing the pre-trained time feature extraction model to obtain a trained time feature extraction network includes: A spatial feature extraction network extracts spatial feature sequences based on training sample image data; Modeling is performed based on the spatial feature sequence to obtain time series data, and the time series data is preprocessed to obtain preprocessed time series data. The data preprocessing includes feature standardization, time series segmentation, and data augmentation. By adding a fully connected layer and an output layer to the pre-trained temporal feature extraction model, a temporal feature extraction network is obtained. Set the optimizer, loss function, and training strategy for the time feature extraction network; Based on the preprocessed time series data, the time feature extraction network is trained to obtain a trained time feature extraction network.

[0011] In some embodiments, fusing spatial feature vectors and temporal feature vectors to obtain a target fused feature vector includes: The spatial feature vector and the corresponding temporal feature vector at each time step are concatenated along the feature dimension to form the first fused feature vector; The first fused feature vector is standardized to obtain the second fused feature vector; Add positional encoding to the second fused feature vector to obtain the target fused feature vector.

[0012] In some embodiments, before identifying the surgical stage corresponding to the surgical image data based on the target recognition network and the target fusion feature vector, the method further includes: Obtain a pre-trained multimodal fusion network, wherein the multimodal fusion network includes a multi-head self-attention mechanism and a feedforward neural network; By adding a fully connected layer and an output layer to the pre-trained multimodal fusion network, a fusion recognition network is obtained; An optimizer, loss function, and training strategy are set for the fusion recognition network, and the fusion recognition network is trained to obtain the target recognition network; The surgical stage identification based on the target recognition network and the target fusion feature vector, identifying the surgical stage corresponding to the surgical image data, includes: The target fusion feature vector is input into the target recognition network, and the target recognition network outputs the surgical stage corresponding to the surgical image data.

[0013] In some embodiments, after training the fusion recognition network to obtain the target recognition network, the method further includes: The model parameters of the target recognition network are fine-tuned based on the validation set to obtain the surgical stage recognition results under different model parameters. The surgical stage recognition results of the validation set under different model parameters are evaluated using preset evaluation indicators to obtain the evaluation scores of the target recognition network under different model parameters. The evaluation scores of the target recognition network under different model parameters are compared, and the model parameters corresponding to the highest evaluation score are selected as the model parameters of the target recognition network.

[0014] Secondly, embodiments of this disclosure provide a surgical stage identification device, the device comprising: The spatial feature extraction unit is used to extract spatial features from surgical image data based on the spatial feature extraction network to obtain spatial feature vectors; The time feature extraction unit is used to extract the time features of the surgical image data based on the time feature extraction network to obtain a time feature vector; The feature fusion unit is used to fuse spatial feature vectors and temporal feature vectors to obtain the target fused feature vector; The identification unit is used to identify the surgical stage based on the target identification network and the target fusion feature vector, and to identify the surgical stage corresponding to the surgical image data.

[0015] Thirdly, embodiments of this disclosure provide an electronic device, including: Memory; Processor; and Computer programs; The computer program is stored in the memory and configured to be executed by the processor to implement the method as described in the first aspect.

[0016] Fourthly, embodiments of this disclosure provide a computer-readable storage medium having a computer program stored thereon, the computer program being executed by a processor to implement the method as described in the first aspect.

[0017] Fifthly, embodiments of this disclosure also provide a computer program product comprising a computer program or instructions that, when executed by a processor, implement the method described in the first aspect.

[0018] The surgical stage identification method, apparatus, electronic device, and readable storage medium provided in this disclosure extract spatial features from surgical image data using a spatial feature extraction network to obtain a spatial feature vector, and extract temporal features from the surgical image data using a temporal feature extraction network to obtain a temporal feature vector. The spatial and temporal feature vectors are then fused to obtain a target fusion feature vector. Surgical stage identification is performed based on a target recognition network and the target fusion feature vector, thus identifying the surgical stage corresponding to the surgical image data. Compared to existing technologies, this disclosure effectively captures details of key parts and operational steps during surgery by extracting spatial features from surgical videos. By extracting temporal features, it effectively captures the temporal dependencies of actions and steps during surgery, improving the temporal accuracy of surgical stage identification. Furthermore, the spatiotemporal feature fusion enables precise classification and identification of surgical stages, enhancing the intelligence level of surgical process monitoring and guidance. Attached Figure Description

[0019] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0020] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 A flowchart of the surgical stage identification method provided in this embodiment of the disclosure; Figure 2 A flowchart of a surgical stage identification method provided in another embodiment of this disclosure; Figure 3 A flowchart of a surgical stage identification method provided in another embodiment of this disclosure; Figure 4 This is a schematic diagram of the overall architecture of the surgical stage identification method provided in the embodiments of this disclosure; Figure 5 A flowchart illustrating the training process of the spatial feature extraction network provided in this embodiment of the disclosure; Figure 6 A flowchart illustrating the training process of the temporal feature extraction network provided in this embodiment of the disclosure; Figure 7 A flowchart illustrating the training process of the target recognition network provided in this embodiment of the disclosure; Figure 8 This is a schematic diagram of the surgical stage identification device provided in an embodiment of the present disclosure; Figure 9 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0022] To better understand the above-mentioned objectives, features, and advantages of this disclosure, the solutions disclosed herein will be further described below. It should be noted that, unless otherwise specified, the embodiments and features described herein can be combined with each other.

[0023] Numerous specific details are set forth in the following description in order to provide a full understanding of this disclosure, but this disclosure may also be implemented in other ways different from those described herein; obviously, the embodiments in the specification are only some, and not all, of the embodiments of this disclosure.

[0024] To address this issue, this disclosure provides a method for identifying surgical stages, which will be described below with reference to specific embodiments.

[0025] Figure 1 This is a flowchart illustrating the surgical stage identification method provided in this embodiment. The method is executed by an electronic device, which can be a portable mobile device such as a smartphone, tablet, or laptop; or a fixed device such as a personal computer or server. The server can be a single server, a server cluster, a distributed cluster, or a centralized cluster. This method can be applied to scenarios requiring surgical stage identification, specifically for laparoscopic cholecystectomy or other surgical procedures. It can improve the accuracy of surgical stage identification and assist surgeons in performing precise operations. It is understood that the surgical stage identification method provided in this embodiment can also be applied to other scenarios.

[0026] The following is about Figure 1 The surgical stage identification method shown is described below, and the specific steps of this method are as follows: S101. Based on the spatial feature extraction network, the spatial features of the surgical image data are extracted to obtain the spatial feature vector.

[0027] In this step, such as Figure 4 As shown, the electronic device constructs a spatial feature extraction network. Further, it acquires surgical image data and extracts spatial features from the surgical image data based on the spatial feature extraction network to obtain a spatial feature vector. Optionally, the spatial feature extraction network can be a SpatiExNet network; no specific limitation is made.

[0028] S102. Extract the temporal features of the surgical image data based on the temporal feature extraction network to obtain a temporal feature vector.

[0029] In this step, such as Figure 4 As shown, the electronic device constructs a temporal feature extraction network. Further, based on this network, the electronic device extracts temporal features from the surgical image data to obtain a temporal feature vector. Optionally, the temporal feature extraction network can be a TempExNet network; no specific limitation is imposed.

[0030] S103. The spatial feature vector and the temporal feature vector are fused to obtain the target fused feature vector.

[0031] In this step, the electronic device fuses the spatial feature vector and the temporal feature vector to obtain the target fused feature vector, so as to fully understand each stage of the surgical process.

[0032] S104. Based on the target recognition network and the target fusion feature vector, the surgical stage is identified to determine the surgical stage corresponding to the surgical image data.

[0033] In this step, after obtaining the target fusion feature vector, the electronic device can perform surgical stage identification based on the target recognition network and the target fusion feature vector, identifying the surgical stage corresponding to the surgical image data. Specifically, as shown... Figure 4 As shown, the target recognition network can be a ResLSTransNet decision network, without specific limitations.

[0034] This embodiment extracts spatial features from surgical image data using a spatial feature extraction network to obtain a spatial feature vector, and extracts temporal features from the surgical image data using a temporal feature extraction network to obtain a temporal feature vector. The spatial and temporal feature vectors are then fused to obtain a target fusion feature vector. Based on a target recognition network and the target fusion feature vector, surgical stage identification is performed to identify the surgical stage corresponding to the surgical image data. Compared to existing technologies, this embodiment effectively captures details of key parts and operational steps during surgery by extracting spatial features from surgical videos. By extracting temporal features, it effectively captures the temporal dependencies of actions and steps during surgery, improving the temporal accuracy of surgical stage identification. Furthermore, spatiotemporal feature fusion enables precise classification and identification of surgical stages, enhancing the intelligence level of surgical process monitoring and guidance.

[0035] Figure 2 A flowchart of a surgical stage identification method provided in another embodiment of this disclosure is shown below. Figure 2 As shown, the method includes the following steps: S201. Obtain the pre-trained spatial feature extraction model and the pre-trained temporal feature extraction model.

[0036] In this step, the electronic device acquires a pre-trained spatial feature extraction model and a pre-trained temporal feature extraction model. In some embodiments, the pre-trained spatial feature extraction model is a ResNet-50 model, and the pre-trained temporal feature extraction model is a Long Short-Term Memory (LSTM) network model, but other models are also acceptable, without specific limitations.

[0037] S202. Optimize and train the pre-trained spatial feature extraction model to obtain a trained spatial feature extraction network.

[0038] A ResNet-50 model was used as the pre-trained spatial feature extraction model. The deep residual structure of this model effectively alleviates the gradient vanishing problem during deep network training, thus ensuring the extraction of deeper features. The initial weights of the pre-trained spatial feature extraction model were derived from pre-training on the large-scale ImageNet dataset, providing the model with rich low- and mid-level visual features, accelerating convergence and improving generalization ability for small-sample tasks. The pre-trained weights enable the network to quickly adapt to new tasks on limited surgical data while retaining the general visual features learned from ImageNet, which is beneficial for subsequent fine-grained surgical stage classification tasks.

[0039] In some embodiments, S202 may include, but is not limited to, S2021, S2022, S2023, S2024, and S2025: S2021. Extract multiple sample image frames based on the training sample image data, and label each sample image frame with the corresponding surgical stage label.

[0040] In this step, the electronic device acquires training sample image data and extracts consecutive image frames from the training sample image data. Each frame carries a specific surgical stage label. Each sample image frame contains spatial information and a corresponding timestamp, facilitating subsequent fusion analysis. For example... Figure 5 As shown, multiple sample image frames and their corresponding surgical stage labels are used as input data for training the model. The input data consists of static image frames extracted from surgical video clips. During extraction, it is crucial to ensure that the image quality meets training requirements, such as sharpness, contrast, and lighting balance. To ensure label accuracy, surgical stage labels are annotated by experts with extensive clinical experience, or initially classified using an automated labeling system and then reviewed and confirmed by experts.

[0041] S2022. Perform preprocessing operations on each sample image frame to obtain each preprocessed sample image frame. The preprocessing operations include image scaling, random cropping, random rotation, color dithering, and pixel normalization.

[0042] In this step, such as Figure 5As shown, the electronic device performs preprocessing operations on each sample image frame to obtain each preprocessed sample image frame. To enhance data diversity, reduce overfitting, and ensure that the input data better matches the expected distribution of the pre-trained spatial feature extraction model, multiple preprocessing operations are employed. These preprocessing operations include image scaling, random cropping, random rotation, color dithering, and pixel normalization, as detailed below: 1) Image scaling: All input images are uniformly scaled to 250×250 pixels using bilinear interpolation to maintain the smoothness and detail integrity of the image during the scaling process.

[0043] 2) Random cropping: For each 250×250 pixel image, a 224×224 region is randomly selected for cropping. This operation can simulate local features under different viewpoints and increase the randomness of training data, effectively preventing the model from memorizing fixed regions and causing overfitting.

[0044] 3) Random rotation: Each image undergoes a random rotation during preprocessing, with the rotation angle randomly selected between -5° and +5°. This step aims to enhance the model's robustness to slight rotational transformations, ensuring accurate feature extraction even when facing different shooting angles in real surgical scenarios.

[0045] 4) Color Dithering: Randomly perturbs the image's color attributes to simulate color variations caused by different lighting conditions and imaging equipment in the operating room. Specific parameter settings are as follows: A. Brightness: Randomly varies within the range of 0.9 to 1.1 times the original brightness; B. Contrast: Adjust randomly between 0.9 and 1.1 times; C. Saturation: The random variation range is 0.9 to 1.1 times; D. Hue: Random offset between -0.05 and +0.05.

[0046] Preprocessing with color dithering helps the model maintain stable feature extraction performance under different environments.

[0047] 5) Pixel Standardization: To ensure the input data distribution conforms to the expectations of the pre-trained spatial feature extraction model, each channel of the image needs to be standardized to have zero mean and unit variance. Specific parameters are: RGB mean: [0.41757566, 0.26098573, 0.25888634]; RGB standard deviation: [0.21938758, 0.1983, 0.19342837].

[0048] Standardized operations can accelerate network convergence, stabilize the inputs of each layer, and reduce training fluctuations.

[0049] S2023. Add a fully connected layer to the pre-trained spatial feature extraction model and freeze the parameters of the target convolutional layer in the pre-trained spatial feature extraction model to obtain the spatial feature extraction network.

[0050] In this step, the pre-trained spatial feature extraction model is first loaded, and the original classification layer is removed, such as... Figure 5 As shown, a fully connected layer adapted for the surgical stage recognition task is added to the pre-trained spatial feature extraction model. Dropout and Batch Normalization are introduced in the structural design to further enhance the network's ability to distinguish surgical stage features. During the initial training phase, the parameters of the low-level feature layers in the pre-trained spatial feature extraction model are frozen, focusing on training the newly added fully connected layer to prevent over-adjustment of the pre-trained weights. As training progresses, some higher layers are gradually unfrozen to achieve fine-tuning of the network and obtain better classification results.

[0051] S2024. Set the optimizer, loss function and training strategy for the spatial feature extraction network.

[0052] To better balance the learning needs of the pre-trained layers and the newly added fully connected layers, such as Figure 5 As shown, a multi-optimizer pattern is used: A. Pre-trained layers: Use a low learning rate (5e-5) to ensure that the weights of these layers are updated smoothly and to prevent the destruction of existing general features.

[0053] B. Fully connected layer: Use a higher learning rate (5e-4) to accelerate the convergence speed of newly added fully connected layer weights.

[0054] like Figure 5 As shown, the loss function chosen is cross-entropy loss, which is suitable for multi-class classification tasks. Simultaneously, the learning rate is automatically adjusted based on the loss changes on the validation set; when the validation loss stops decreasing, the learning rate is automatically reduced to further improve training stability.

[0055] like Figure 5 As shown, the training strategy adopted is: A. Batch settings: The size of each training batch is set to 100 images to ensure that while making full use of GPU parallel computing, excessive memory pressure is not introduced.

[0056] B. Validation mechanism: After each batch epoch, the model performance is evaluated on an independent validation set, mainly monitoring accuracy, recall and F1 score.

[0057] C. Early stopping mechanism: When the validation accuracy does not improve significantly after several consecutive epochs, the early stopping strategy is triggered to prevent the model from overfitting on the training set and to save training resources.

[0058] S2025. Based on each preprocessed sample image frame, the spatial feature extraction network is trained to obtain a trained spatial feature extraction network.

[0059] In this step, the electronic device trains the spatial feature extraction network based on each preprocessed sample image frame to obtain a trained spatial feature extraction network.

[0060] In some embodiments, accuracy, recall, and F1 score are used as the main evaluation metrics to quantitatively assess the model's performance on classification tasks at different surgical stages. Simultaneously, a confusion matrix can be constructed to further analyze misclassifications across different surgical stages, pinpoint model weaknesses, and guide subsequent improvements. Throughout the training process, model parameters are saved based on the optimal performance on the validation set. These saved optimal model parameters are used for training subsequent temporal feature extraction and object recognition networks, ensuring the overall system possesses optimal feature representation capabilities in subsequent tasks.

[0061] S203. Optimize and train the pre-trained time feature extraction model to obtain a trained time feature extraction network.

[0062] In this step, a Long Short-Term Memory (LSTM) network model is used as a pre-trained time feature extraction model, which is used to model time series features.

[0063] In some embodiments, S203 may include, but is not limited to, S2031, S2032, S2033, S2034, and S2035: S2031. Obtain spatial feature sequences extracted by the spatial feature extraction network based on training sample image data.

[0064] In this step, the training sample image data is input into the spatial feature extraction network to obtain the spatial feature sequence.

[0065] S2032. Modeling is performed based on the spatial feature sequence to obtain time series data, and the time series data is preprocessed to obtain preprocessed time series data. The data preprocessing includes feature standardization, time series segmentation, and data augmentation.

[0066] Furthermore, the electronic device will model based on the spatial feature sequence to obtain time-series data, with each time-series data corresponding to a specific surgical stage label. Each time-series data contains temporal information and corresponding spatial features, facilitating subsequent fusion analysis. And, as... Figure 6 As shown, the time series data is preprocessed to obtain preprocessed time series data. The data preprocessing includes feature standardization, time series segmentation, and data augmentation, as detailed below: 1) Feature standardization: The spatial feature sequence extracted from the spatial feature extraction network is standardized so that the distribution of each feature channel has zero mean and unit variance.

[0067] 2) Time series segmentation: such as Figure 6 As shown, continuous time series data is divided into time windows of fixed length for processing by the LSTM network.

[0068] 3) Data augmentation: Randomly shuffle or add noise to time series data to increase data diversity and robustness and prevent overfitting.

[0069] S2033. Add a fully connected layer and an output layer to the pre-trained time feature extraction model to obtain the time feature extraction network.

[0070] In this step, a pre-trained temporal feature extraction model is first loaded, the original fully connected layer is removed or adjusted, and a custom fully connected layer and output layer adapted to the surgical stage recognition task are added to the pre-trained temporal feature extraction model to obtain the temporal feature extraction network.

[0071] S2034. Set the optimizer, loss function and training strategy for the time feature extraction network.

[0072] In this step, such as Figure 6 As shown, the optimizer, loss function, and training strategy are set for the temporal feature extraction network, as detailed below: a. Use the Adam optimizer for training, and set the initial learning rate to 0.001.

[0073] b. The loss function adopted is the cross-entropy loss function, which is used for multi-class surgical stage classification tasks.

[0074] c. Learning rate adjustment strategy: Adjust the learning rate based on the validation set loss. When the validation loss no longer decreases, the learning rate is automatically reduced.

[0075] D. Training Strategy: The batch size is set to 64. Validation is performed after each epoch, and the accuracy and loss on the validation set are calculated to evaluate model performance. Early stopping is implemented when the validation accuracy no longer improves to prevent overfitting.

[0076] S2035. Based on the preprocessed time series data, the time feature extraction network is trained to obtain a trained time feature extraction network.

[0077] In this step, the electronic device trains the time feature extraction network based on the preprocessed time series data to obtain a trained time feature extraction network.

[0078] In some embodiments, accuracy, recall, and F1 score are used as the primary evaluation metrics to measure the performance of the temporal feature extraction network. The best-performing model during training is saved for subsequent target recognition network training and feature extraction.

[0079] S204. Based on the spatial feature extraction network, the spatial features of the surgical image data are extracted to obtain the spatial feature vector.

[0080] Specifically, the implementation process and principle of S204 and S101 are the same, and will not be repeated here.

[0081] S205. Extract the temporal features of the surgical image data based on the temporal feature extraction network to obtain a temporal feature vector.

[0082] Specifically, the implementation process and principle of S205 and S102 are the same, and will not be repeated here.

[0083] S206. Connect the spatial feature vector and the corresponding temporal feature vector of each time step along the feature dimension to form the first fused feature vector.

[0084] In this step, the spatial features and corresponding temporal features of each time step are concatenated along the feature dimension to form the first fused feature vector. The spatial feature vector has a dimension of D_s, the temporal feature vector has a dimension of D_t, and the first fused feature vector has a dimension of D_s + D_t.

[0085] S207. The first fused feature vector is standardized to obtain the second fused feature vector.

[0086] In this step, the first fused feature vector is standardized to obtain the second fused feature vector, so that the distribution of each dimension has zero mean and unit variance.

[0087] S208. Add positional encoding to the second fused feature vector to obtain the target fused feature vector.

[0088] Furthermore, positional encoding is added to the second fused feature vector to obtain the target fused feature vector, in order to preserve time series information.

[0089] S209. Based on the target recognition network and the target fusion feature vector, the surgical stage is identified to determine the surgical stage corresponding to the surgical image data.

[0090] Specifically, the implementation process and principle of S209 and S104 are the same, and will not be repeated here.

[0091] This embodiment of the disclosure obtains a pre-trained spatial feature extraction model and a pre-trained temporal feature extraction model. The pre-trained spatial feature extraction model is then optimized and trained to obtain a trained spatial feature extraction network. Similarly, the pre-trained temporal feature extraction model is optimized and trained to obtain a trained temporal feature extraction network. Further, spatial features of the surgical image data are extracted based on the spatial feature extraction network to obtain spatial feature vectors. Temporal features of the surgical image data are extracted based on the temporal feature extraction network to obtain temporal feature vectors. Next, the spatial feature vectors and corresponding temporal feature vectors for each time step are concatenated along the feature dimension to form a first fused feature vector. The first fused feature vector is then standardized to obtain a second fused feature vector. A positional encoding is added to the second fused feature vector to obtain a target fused feature vector. Finally, based on a target recognition network and the target fused feature vector, surgical stage identification is performed to identify the surgical stage corresponding to the surgical image data. Through this method, the embodiments of this disclosure can effectively capture the details of key parts and operation steps during the operation, effectively capture the time dependence of actions and steps during the operation, improve the temporal accuracy of surgical stage identification, achieve accurate classification and identification of surgical stages, and enhance the level of intelligence in surgical process monitoring and guidance.

[0092] Figure 3 A flowchart of a surgical stage identification method provided in another embodiment of this disclosure is shown below. Figure 3 As shown, the method includes the following steps: S301. Based on the spatial feature extraction network, the spatial features of the surgical image data are extracted to obtain the spatial feature vector.

[0093] Specifically, the implementation process and principle of S301 and S101 are the same, and will not be repeated here.

[0094] S302. Extract the temporal features of the surgical image data based on the temporal feature extraction network to obtain a temporal feature vector.

[0095] Specifically, the implementation process and principle of S302 and S102 are the same, and will not be repeated here.

[0096] S303. The spatial feature vector and the temporal feature vector are fused to obtain the target fused feature vector.

[0097] Specifically, the implementation process and principle of S303 and S103 are the same, and will not be repeated here.

[0098] S304. Obtain a pre-trained multimodal fusion network, wherein the multimodal fusion network includes a multi-head self-attention mechanism and a feedforward neural network.

[0099] Optional, such as Figure 7 As shown, the multimodal fusion network is a Transformer network, without limitation. The multimodal fusion network includes a multi-head self-attention mechanism and a feedforward neural network. Encoder: Contains N encoder layers, each including a multi-head self-attention mechanism and a feedforward neural network. Decoder: Contains M decoder layers, each including a multi-head self-attention mechanism, an encoder-decoder attention mechanism, and a feedforward neural network.

[0100] S305. Add a fully connected layer and an output layer to the pre-trained multimodal fusion network to obtain the fusion recognition network.

[0101] In this step, a pre-trained multimodal fusion network is first loaded, along with the weights of the pre-trained model, to accelerate training and improve performance. For example... Figure 7 As shown, a custom fully connected layer and output layer adapted to the surgical stage recognition task are added to the pre-trained multimodal fusion network to obtain the fusion recognition network.

[0102] S306. Set an optimizer, loss function and training strategy for the fusion recognition network, and train the fusion recognition network to obtain the target recognition network.

[0103] In this step, such as Figure 7 As shown, the optimizer, loss function, and training strategy are set for the fusion recognition network, as detailed below: 1) Use the Adam optimizer for training, and set the initial learning rate to 0.001.

[0104] 2) The loss function adopted is the cross-entropy loss function, which is used for multi-class surgical stage classification tasks.

[0105] 3) Learning rate adjustment strategy: Adjust the learning rate based on the validation set loss. When the validation loss no longer decreases, the learning rate is automatically reduced.

[0106] 4) Training strategy: The batch size is set to 64. Validation is performed after each epoch, and the accuracy and loss on the validation set are calculated to evaluate model performance. Early stopping is performed when the validation accuracy no longer improves to prevent overfitting.

[0107] S307. The model parameters of the target recognition network are tuned based on the validation set to obtain the surgical stage recognition results under different model parameters.

[0108] In this step, the electronic device will use the validation set to fine-tune the model parameters of the target recognition network, and obtain the surgical stage recognition results of the validation set under different model parameters.

[0109] S308. The surgical stage recognition results of the validation set under different model parameters are evaluated using preset evaluation indicators to obtain the evaluation scores of the target recognition network under different model parameters.

[0110] Furthermore, the surgical stage recognition results of the validation set under different model parameters are evaluated using preset evaluation metrics to obtain evaluation scores for the target recognition network under different model parameters. Preset evaluation metrics include, but are not limited to, accuracy, recall, and F1 score. In this embodiment, accuracy, recall, and F1 score are used as the main evaluation metrics to measure the performance of the target recognition network.

[0111] S309. Compare the evaluation scores of the target recognition network under different model parameters, and select the model parameters corresponding to the highest evaluation score as the model parameters of the target recognition network.

[0112] In this embodiment, the evaluation scores of the target recognition network under different model parameters are compared, and the model parameters corresponding to the highest evaluation score are selected as the model parameters of the target recognition network. The model that performs best during training is saved for subsequent surgical stage recognition tasks and applications.

[0113] S310. Input the target fusion feature vector into the target recognition network, and output the surgical stage corresponding to the surgical image data through the target recognition network.

[0114] In this step, the target fusion feature vector is used as the input to the target recognition network model, and the target recognition network outputs the surgical stage corresponding to the surgical image data.

[0115] This embodiment extracts spatial features from surgical image data using a spatial feature extraction network to obtain a spatial feature vector, and extracts temporal features from the surgical image data using a temporal feature extraction network to obtain a temporal feature vector. The spatial and temporal feature vectors are then fused to obtain a target fusion feature vector. Further, a pre-trained multimodal fusion network is obtained, comprising a multi-head self-attention mechanism and a feedforward neural network. Fully connected layers and an output layer are added to the pre-trained multimodal fusion network to obtain a fusion recognition network. An optimizer, loss function, and training strategy are set for the fusion recognition network, and the network is trained to obtain a target recognition network. Next, the model parameters of the target recognition network are fine-tuned based on a validation set to obtain surgical stage recognition results under different model parameters. The surgical stage recognition results under different model parameters are evaluated using a preset evaluation index to obtain evaluation scores for the target recognition network under different model parameters. The evaluation scores of the target recognition network under different model parameters are compared, and the model parameters corresponding to the highest evaluation score are selected as the model parameters of the target recognition network. The target fusion feature vector is then input into a target recognition network, which outputs the surgical stage corresponding to the surgical image data. This method effectively captures details of key areas and operational steps during surgery, effectively captures the temporal dependencies of actions and steps, improves the temporal accuracy of surgical stage identification, and enables precise classification and identification of surgical stages, thereby enhancing the intelligence level of surgical process monitoring and guidance.

[0116] Figure 8 This is a schematic diagram of the surgical stage identification device provided in this embodiment. The surgical stage identification device can be an electronic device as described in the above embodiment, or it can be a component or assembly within that electronic device. The surgical stage identification device provided in this embodiment can execute the processing flow provided in the surgical stage identification method embodiment, such as... Figure 8 As shown, the surgical stage identification device 40 includes: a spatial feature extraction unit 41, a temporal feature extraction unit 42, a feature fusion unit 43, and an identification unit 44; wherein, the spatial feature extraction unit 41 is used to extract spatial features from the surgical image data based on a spatial feature extraction network to obtain a spatial feature vector; the temporal feature extraction unit 42 is used to extract temporal features from the surgical image data based on a temporal feature extraction network to obtain a temporal feature vector; the feature fusion unit 43 is used to fuse the spatial feature vector and the temporal feature vector to obtain a target fused feature vector; and the identification unit 44 is used to identify the surgical stage based on a target identification network and the target fused feature vector, thereby identifying the surgical stage corresponding to the surgical image data.

[0117] Optionally, the surgical stage recognition device 40 further includes: an acquisition unit 45; the acquisition unit 45 is used to acquire a pre-trained spatial feature extraction model and a pre-trained temporal feature extraction model; optimize the pre-trained spatial feature extraction model to obtain a trained spatial feature extraction network; optimize the pre-trained temporal feature extraction model to obtain a trained temporal feature extraction network.

[0118] Optionally, when the obtaining unit 45 optimizes and trains the pre-trained spatial feature extraction model to obtain a trained spatial feature extraction network, it specifically performs the following: extracting multiple sample image frames based on training sample image data, with each sample image frame labeled with a corresponding surgical stage label; performing preprocessing operations on each sample image frame to obtain each preprocessed sample image frame, the preprocessing operations including image scaling, random cropping, random rotation, color dithering, and pixel normalization; adding a fully connected layer to the pre-trained spatial feature extraction model, freezing the parameters of the target convolutional layer in the pre-trained spatial feature extraction model to obtain a spatial feature extraction network; setting an optimizer, loss function, and training strategy for the spatial feature extraction network; and training the spatial feature extraction network based on each preprocessed sample image frame to obtain a trained spatial feature extraction network.

[0119] Optionally, when the obtaining unit 45 optimizes and trains the pre-trained temporal feature extraction model to obtain a trained temporal feature extraction network, it is specifically used for: obtaining spatial feature sequences extracted by the spatial feature extraction network based on training sample image data; modeling based on the spatial feature sequences to obtain time series data, and performing data preprocessing on the time series data to obtain preprocessed time series data, wherein the data preprocessing includes feature standardization, time series segmentation, and data augmentation; adding fully connected layers and output layers to the pre-trained temporal feature extraction model to obtain a temporal feature extraction network; setting an optimizer, loss function, and training strategy for the temporal feature extraction network; and training the temporal feature extraction network based on the preprocessed time series data to obtain a trained temporal feature extraction network.

[0120] Optionally, when the feature fusion unit 43 fuses the spatial feature vector and the temporal feature vector to obtain the target fused feature vector, it is specifically used to: connect the spatial feature vector and the corresponding temporal feature vector at each time step in the feature dimension to form a first fused feature vector; perform standardization processing on the first fused feature vector to obtain a second fused feature vector; and add position encoding to the second fused feature vector to obtain the target fused feature vector.

[0121] Optionally, before identifying the surgical stage corresponding to the surgical image data based on the target recognition network and the target fusion feature vector, the surgical stage recognition device 40 further includes: a training unit 46; the training unit 46 is used to acquire a pre-trained multimodal fusion network, the multimodal fusion network including a multi-head self-attention mechanism and a feedforward neural network; add a fully connected layer and an output layer to the pre-trained multimodal fusion network to obtain a fusion recognition network; set an optimizer, loss function and training strategy for the fusion recognition network, and train the fusion recognition network to obtain a target recognition network; The identification unit 44 identifies the surgical stage based on the target identification network and the target fusion feature vector. When identifying the surgical stage corresponding to the surgical image data, it is specifically used to: input the target fusion feature vector into the target identification network, and output the surgical stage corresponding to the surgical image data through the target identification network.

[0122] Optionally, after training the fusion recognition network to obtain the target recognition network, the surgical stage recognition device 40 further includes: a determination unit 47; the determination unit 47 is used to optimize the model parameters of the target recognition network based on the validation set to obtain the surgical stage recognition results of the validation set under different model parameters; to evaluate the surgical stage recognition results of the validation set under different model parameters using a preset evaluation index to obtain the evaluation score of the target recognition network under different model parameters; to compare the evaluation scores of the target recognition network under different model parameters and select the model parameters corresponding to the highest evaluation score as the model parameters of the target recognition network.

[0123] Figure 8 The surgical stage identification device shown in the embodiment can be used to execute the technical solution of the above method embodiment. Its implementation principle and technical effect are similar, and will not be repeated here.

[0124] Figure 9 This is a schematic diagram of the structure of an electronic device according to an embodiment of this disclosure. See below for details. Figure 9 It shows a schematic diagram of a structure suitable for implementing the electronic device 600 in the embodiments of this disclosure. Figure 9 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0125] like Figure 9As shown, the electronic device 600 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage device 608 into a random access memory (RAM) 603 to implement the surgical stage identification method as described in the embodiments of this disclosure. Various programs and data required for the operation of the electronic device 600 are also stored in the RAM 603. The processing device 601, ROM 602, and RAM 603 are interconnected via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0126] Typically, the following devices can be connected to I / O interface 605: input devices 606 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 607 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 608 including, for example, magnetic tapes, hard disks, etc.; and communication devices 609. Communication device 609 allows electronic device 600 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 9 An electronic device 600 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0127] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts, thereby implementing the surgical stage identification method as described above. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 609, or installed from a storage device 608, or installed from a ROM 602. When the computer program is executed by the processing device 601, it performs the functions defined in the methods of embodiments of this disclosure.

[0128] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0129] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol) and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.

[0130] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0131] The aforementioned computer-readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to: Spatial feature vectors are obtained by extracting spatial features from surgical image data using a spatial feature extraction network. The temporal features of the surgical image data are extracted using a temporal feature extraction network to obtain a temporal feature vector; The spatial feature vector and the temporal feature vector are fused to obtain the target fused feature vector; Based on the target recognition network and the target fusion feature vector, the surgical stage is identified, and the surgical stage corresponding to the surgical image data is identified.

[0132] Optionally, when one or more of the above-described procedures are executed by the electronic device, the electronic device may also perform other steps described in the above embodiments.

[0133] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including but not limited to object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0134] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0135] The units described in the embodiments of this disclosure can be implemented in software or hardware. The names of the units are not, in some cases, intended to limit the specific unit.

[0136] The functions described above in this document can be performed at least in part by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), and so on.

[0137] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0138] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

[0139] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0140] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.

Claims

1. A method for identifying surgical stages, characterized in that, The method includes: Spatial feature vectors are obtained by extracting spatial features from surgical image data using a spatial feature extraction network. The temporal features of the surgical image data are extracted using a temporal feature extraction network to obtain a temporal feature vector; The spatial feature vector and the temporal feature vector are fused to obtain the target fused feature vector; Based on the target recognition network and the target fusion feature vector, the surgical stage is identified, and the surgical stage corresponding to the surgical image data is identified.

2. The method according to claim 1, characterized in that, The method further includes: Obtain a pre-trained spatial feature extraction model and a pre-trained temporal feature extraction model; The pre-trained spatial feature extraction model is optimized and trained to obtain a trained spatial feature extraction network; The pre-trained time feature extraction model is optimized and trained to obtain a well-trained time feature extraction network.

3. The method according to claim 2, characterized in that, The step of optimizing and training the pre-trained spatial feature extraction model to obtain a trained spatial feature extraction network includes: Multiple sample image frames were extracted from the training sample image data, and each sample image frame was labeled with the corresponding surgical stage label. Each sample image frame is preprocessed to obtain a preprocessed sample image frame. The preprocessing operations include image scaling, random cropping, random rotation, color dithering, and pixel normalization. A fully connected layer is added to the pre-trained spatial feature extraction model, and the parameters of the target convolutional layer in the pre-trained spatial feature extraction model are frozen to obtain the spatial feature extraction network. Set the optimizer, loss function, and training strategy for the spatial feature extraction network; Based on each preprocessed sample image frame, the spatial feature extraction network is trained to obtain a trained spatial feature extraction network.

4. The method according to claim 2, characterized in that, The step of optimizing the pre-trained time feature extraction model to obtain a trained time feature extraction network includes: A spatial feature extraction network extracts spatial feature sequences based on training sample image data; Modeling is performed based on the spatial feature sequence to obtain time series data, and the time series data is preprocessed to obtain preprocessed time series data. The data preprocessing includes feature standardization, time series segmentation, and data augmentation. By adding a fully connected layer and an output layer to the pre-trained temporal feature extraction model, a temporal feature extraction network is obtained. Set the optimizer, loss function, and training strategy for the time feature extraction network; Based on the preprocessed time series data, the time feature extraction network is trained to obtain a trained time feature extraction network.

5. The method according to claim 1, characterized in that, The process of fusing spatial feature vectors and temporal feature vectors to obtain a target fused feature vector includes: The spatial feature vector and the corresponding temporal feature vector at each time step are concatenated along the feature dimension to form the first fused feature vector; The first fused feature vector is standardized to obtain the second fused feature vector; Add positional encoding to the second fused feature vector to obtain the target fused feature vector.

6. The method according to claim 1, characterized in that, Before identifying the surgical stage corresponding to the surgical image data based on the target recognition network and the target fusion feature vector, the method further includes: Obtain a pre-trained multimodal fusion network, wherein the multimodal fusion network includes a multi-head self-attention mechanism and a feedforward neural network; By adding a fully connected layer and an output layer to the pre-trained multimodal fusion network, a fusion recognition network is obtained; An optimizer, loss function, and training strategy are set for the fusion recognition network, and the fusion recognition network is trained to obtain the target recognition network; The surgical stage identification based on the target recognition network and the target fusion feature vector, identifying the surgical stage corresponding to the surgical image data, includes: The target fusion feature vector is input into the target recognition network, and the target recognition network outputs the surgical stage corresponding to the surgical image data.

7. The method according to claim 6, characterized in that, After training the fusion recognition network to obtain the target recognition network, the method further includes: The model parameters of the target recognition network are fine-tuned based on the validation set to obtain the surgical stage recognition results under different model parameters. The surgical stage recognition results of the validation set under different model parameters are evaluated using preset evaluation indicators to obtain the evaluation scores of the target recognition network under different model parameters. The evaluation scores of the target recognition network under different model parameters are compared, and the model parameters corresponding to the highest evaluation score are selected as the model parameters of the target recognition network.

8. A surgical stage identification device, characterized in that, include: The spatial feature extraction unit is used to extract spatial features from surgical image data based on the spatial feature extraction network to obtain spatial feature vectors; The time feature extraction unit is used to extract the time features of the surgical image data based on the time feature extraction network to obtain a time feature vector; The feature fusion unit is used to fuse spatial feature vectors and temporal feature vectors to obtain the target fused feature vector; The identification unit is used to identify the surgical stage based on the target identification network and the target fusion feature vector, and to identify the surgical stage corresponding to the surgical image data.

9. An electronic device, characterized in that, include: Memory; processor; as well as Computer programs; The computer program is stored in the memory and configured to be executed by the processor to implement the method as described in any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-7.

Images