Spine endoscope tissue instance segmentation method, device, equipment and medium

Abstract

The invention relates to a spine endoscope tissue instance segmentation method, device and equipment and a medium, and belongs to the technical field of medical image processing and artificial intelligence. The spine endoscope tissue instance segmentation method comprises the steps that spine tissue endoscope images in an operation are collected in real time; the spine tissue endoscopic image is identified and segmented based on a trained improved YOLOv8 model, a segmentation result is obtained, the improved YOLOv8 model comprises a backbone network layer, a bottleneck network layer and a detection head layer, and the detection head layer comprises a shared convolution module and a plurality of mask modules; and the category and position of the spine tissue are determined based on the segmentation result, so that the segmentation precision of the spine endoscope image is improved.

Description

Technical Field

[0001] This invention relates to the fields of medical image processing and artificial intelligence technology, and in particular to a method, apparatus, device and medium for segmenting tissue instances in spinal endoscopy. Background Technology

[0002] Spinal endoscopic surgery is a minimally invasive surgical technique that diagnoses and treats diseases or lesions inside the spine. It uses specialized spinal endoscopes, which are inserted into the spine through small incisions or percutaneous punctures to observe and treat spinal-related diseases or lesions.

[0003] During surgery, the internal structure of the spine is complex, containing many important nerve and blood vessel structures. Currently, doctors mainly rely on experience and intuition to identify these structures, which increases the difficulty and risk of the surgery. Due to limited field of vision, doctors may not be able to see the details of the operating area comprehensively and clearly. When precise positioning and navigation are required within the patient's body to reach a specific target area, it is difficult to do so based solely on experience.

[0004] YOLO (You Only Look Once), a classic real-time object detection algorithm, has demonstrated high detection speed and certain accuracy in the field of medical imaging. It features a lighter network structure, higher detection accuracy, and real-time performance. However, YOLOv8 has limitations in detecting small targets with blurred boundaries, especially in complex backgrounds, such as the neural and vascular structures of the spinal tissue in endoscopic images. In these cases, the detection and segmentation accuracy of small targets and structures with blurred boundaries in the spinal tissue tends to decrease. Summary of the Invention

[0005] In view of this, it is necessary to provide a method, apparatus, device and medium for segmenting spinal endoscopic tissue instances to solve the technical problem of decreased detection and segmentation accuracy of small targets and blurred-boundary structures in spinal tissue in endoscopic images.

[0006] To address the aforementioned problems, in a first aspect, the present invention provides a method for segmenting tissue instances in spinal endoscopy, comprising: Real-time acquisition of endoscopic images of spinal tissues during surgery; The improved YOLOv8 model is used to identify and segment the spinal tissue endoscopic image to obtain the segmentation result. The improved YOLOv8 model includes a backbone network layer, a bottleneck network layer, and a detection head layer. The detection head layer includes a shared convolutional module and multiple masking modules. The shared convolutional module is used to extract the prediction box and spinal tissue category information of the spinal tissue endoscopic image. The multiple masking modules are used to extract the masking features of the spinal tissue endoscopic image. The category and location of spinal tissues are determined based on the segmentation results.

[0007] In one possible implementation, prior to the real-time acquisition of intraoperative endoscopic images of spinal tissue, the following steps are included: The acquired spinal tissue endoscopic images are preprocessed to obtain a preprocessed spinal tissue endoscopic image set. Tissue category labeling is performed on the preprocessed spinal tissue endoscopic image set to obtain a spinal tissue endoscopic image set containing location labels; The set of spinal tissue endoscopic images containing location labels is divided into a training set, a validation set, and a test set according to a preset ratio. Based on the training set, validation set, and test set, the improved YOLOv8 model to be trained is iteratively trained to obtain a well-trained improved YOLOv8 model.

[0008] In one possible implementation, the segmentation result obtained by recognizing and segmenting the spinal tissue endoscopic image based on the trained improved YOLOv8 model includes: Based on the backbone network layer, feature extraction is performed on the endoscopic image of the spinal tissue to obtain an initial feature map; Based on the bottleneck network layer, feature fusion is performed on the initial feature map to obtain the feature map to be tested; The detection head layer is used to perform instance segmentation on the feature map to be tested, and the segmentation result is obtained.

[0009] In one possible implementation, the backbone network layer includes a CBS module, a C2F module, and an SPPF module. The CBS module includes two-dimensional convolution, batch normalization, and an activation function. The C2F module includes convolutional layers, segmentation layers, attention mechanism layers, and connection layers. The step of extracting features from the spinal tissue endoscopic image based on the backbone network layer to obtain an initial feature map includes: The CBS module sequentially performs convolution processing, batch normalization processing, and nonlinear change processing of activation function on the endoscopic images of the spinal tissue to obtain multi-scale feature maps. The C2f module performs preliminary feature extraction on the multi-scale feature map through convolutional layers. The feature map is then segmented by a segmentation layer to obtain a first feature map and a second feature map. The second feature map is dynamically weighted by an attention mechanism layer, and then connected to the first feature map in the channel dimension by a connection layer to obtain a fused feature map. The initial feature map is obtained by performing multi-scale max pooling on the fused feature map using the SPPF module.

[0010] In one possible implementation, the bottleneck network layer has an ASF-YOLO architecture, which includes a C2f module with an embedded attention mechanism and a TFE module. The step of fusing features from the initial feature map based on the bottleneck network layer to obtain the feature map to be tested includes: ASF-YOLO is used to extract multi-level features from the initial feature map, and the extracted multi-level features are fused by a fusion factor to obtain the feature map to be tested. In particular, TFE and C2F modules are used to perform deep feature extraction on the initial feature map to obtain multi-level features.

[0011] In one possible implementation, the shared convolutional module includes a first convolutional group normalization, a second convolutional group normalization, a classification convolutional layer, a regression convolutional layer, and a scale adjustment module. The plurality of masking modules include a first masking module, a second masking module, and a third masking module. The first masking module, the second masking module, and the third masking module have the same structure. The first masking module includes... Convolutional group normalization, The convolutional group normalization and masked convolutional layers, the first mask module, the second mask module, and the third mask module. The convolutional group normalization is connected to the first convolutional group normalization of the shared convolutional module; the step of identifying and segmenting the feature map to be tested based on the detection head layer to obtain the segmentation result includes: The feature map to be tested is subjected to convolution, group normalization, aggregation, and masking processing by multiple masking modules to obtain a first feature mask, a second feature mask, and a third feature mask. The first convolutional group of the shared convolutional module is normalized to share the first mask module, the second mask module, and the third mask module. The convolutional group normalization generates a first feature map, a second feature map, and a third feature map. The first feature map, the second feature map, and the third feature map are aggregated by the second convolutional group normalization. The aggregated feature map is classified by a classification convolutional layer to obtain spinal tissue category information. The aggregated feature map is regressed by a regression convolutional layer. Finally, the feature map after regression is adaptively scaled by a scaling module to obtain the prediction box. The segmentation result is obtained based on the first feature mask, the second feature mask, the third feature mask, the spinal tissue category information, and the prediction box.

[0012] In one possible implementation, the categories of spinal tissue include the ligamentum flavum, nerve roots, bone tissue, and fat.

[0013] In a second aspect, the present invention also provides a spinal endoscopic tissue segmentation device, comprising: The image acquisition module is used to acquire endoscopic images of spinal tissues in real time during surgery; The instance segmentation module is used to identify and segment the spinal tissue endoscopic image based on a trained improved YOLOv8 model to obtain segmentation results. The improved YOLOv8 model includes a backbone network layer, a bottleneck network layer, and a detection head layer. The detection head layer includes a shared convolutional module and multiple masking modules. The shared convolutional module is used to extract the prediction box and spinal tissue category information of the spinal tissue endoscopic image. The multiple masking modules are used to extract the masking features of the spinal tissue endoscopic image. A spinal tissue determination module is used to determine the category and location of spinal tissue based on the segmentation results.

[0014] Thirdly, the present invention also provides a real-time surgical assistance device, comprising: a processor and a memory; The memory stores a computer-readable program that can be executed by the processor; When the processor executes the computer-readable program, it implements the steps in the spinal endoscopic tissue instance segmentation method as described above.

[0015] Fourthly, the present invention also provides a computer-readable storage medium for storing a computer-readable program or instructions, which, when executed by a processor, can implement the steps of the spinal endoscopic tissue instance segmentation method described in any one of the above-mentioned method items.

[0016] The beneficial effects of this invention are as follows: It uses an improved YOLOv8 model to identify and segment endoscopic images of spinal tissues, obtaining segmentation results. The improved YOLOv8 model includes a backbone network layer, a bottleneck network layer, and a detection head layer. The detection head layer includes a shared convolutional module and multiple masking modules. Based on the segmentation results, it determines the category and location of spinal tissues. The C2f module, which incorporates an attention mechanism within the backbone network layer, enhances feature representation while maintaining a lightweight model, thereby improving instance segmentation. The ASF-YOLO module in the bottleneck network layer enables spatial adaptive fusion of multi-scale features, improving the recognition accuracy of small targets and boundary structures. The shared convolutional module in the detection head layer significantly reduces the number of parameters, minimizing accuracy loss. Using the improved YOLOv8 model to identify and segment endoscopic images of spinal tissues effectively improves the detection and segmentation accuracy of small targets and blurred boundary structures in spinal endoscopic images, enhances the intelligent image analysis capabilities during surgery, assists doctors in accurately identifying tissue structures, and improves surgical safety and efficiency. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 A flowchart illustrating an embodiment of the spinal endoscopic tissue instance segmentation method provided by the present invention; Figure 2 A schematic diagram of the backbone network layer in the spinal endoscopic tissue instance segmentation method provided by the present invention; Figure 3 A schematic diagram of the C2f module of the spinal endoscopic tissue instance segmentation method provided by the present invention; Figure 4 A schematic diagram of the ASF-YOLO module of the spinal endoscopic tissue instance segmentation method provided by the present invention. Figure 5 This is a schematic diagram of the detection head of the spinal endoscopic tissue instance segmentation method provided by the present invention. Figure 6a This is a schematic diagram of the segmentation result of the spinal endoscopic tissue instance segmentation method provided by the present invention. Figure 6b This is a schematic diagram of the segmentation result of the spinal endoscopic tissue instance segmentation method provided by the present invention. Figure 7 A schematic diagram of a structure of an embodiment of the spinal endoscopic tissue segmentation device provided by the present invention; Figure 8 This is a schematic diagram of an embodiment of the real-time surgical assistance device provided by the present invention. Detailed Implementation

[0019] Preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings, which form part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, but are not intended to limit the scope of the present invention.

[0020] In this document, the term "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0021] This invention discloses a method, apparatus, device, and medium for segmenting tissue samples in spinal endoscopy, which can be used in a computer. The method, apparatus, or computer-readable storage medium involved in this invention can be integrated with the aforementioned apparatus or can be relatively independent.

[0022] One specific embodiment of the present invention discloses a method for segmenting tissue instances in spinal endoscopy, which can be executed by a computer, specifically by one or more processors of the computer. For example... Figure 1 As shown, the method for segmenting tissue instances in spinal endoscopy includes: S101. Real-time acquisition of endoscopic images of spinal tissues during surgery; It should be noted that before acquiring endoscopic images of the spinal tissue during surgery, the improved YOLOv8 model to be trained was iteratively trained using the acquired endoscopic images of the spinal tissue to obtain the improved YOLOv8 model.

[0023] S102. Based on the trained improved YOLOv8 model, the spinal tissue endoscopic image is identified and segmented to obtain the segmentation result. The improved YOLOv8 model includes a backbone network layer, a bottleneck network layer and a detection head layer. The detection head layer includes a shared convolutional module and multiple mask modules. The shared convolutional module is used to extract the prediction box and spinal tissue category information of the spinal tissue endoscopic image. The multiple mask modules are used to extract the mask features of the spinal tissue endoscopic image. It should be noted that feature extraction is performed through the backbone network layer, feature fusion is performed through the bottleneck network layer, and instance segmentation is performed through the detection head layer, which effectively improves the detection and segmentation accuracy of small targets and structures with blurred boundaries in spinal endoscopy images.

[0024] S103. Determine the category and location of spinal tissues based on the segmentation results.

[0025] It should be noted that the categories of spinal tissues include ligamentum flavum, nerve roots, bone tissue, and fat. Instance segmentation is performed using an improved YOLOv8 model to determine the category and location of spinal tissues, assisting doctors in accurately identifying tissue structures and improving surgical safety and efficiency.

[0026] In some embodiments, in step S101, real-time acquisition of endoscopic images of spinal tissues during surgery, i.e., acquiring images from the intervertebral foramen lens during real-time surgery, prepares for the identification and segmentation of spinal tissues during surgery. Before acquiring the endoscopic images of spinal tissues during surgery, the acquired endoscopic images of spinal tissues are used to train the improved YOLOv8 model to be trained, so as to obtain a fully trained improved YOLOv8 model. Specifically, the acquired endoscopic images of spinal tissues are preprocessed to obtain a preprocessed set of endoscopic images of spinal tissues; the preprocessed set of endoscopic images of spinal tissues is labeled with tissue categories to obtain a set of endoscopic images of spinal tissues containing location labels; and data videos are collected from the hospital, and screenshots are taken from the videos to obtain images, the image size is... To ensure high image clarity and resolution, surgical videos from multiple patients were collected to guarantee dataset diversity. The preprocessed image training set was labeled with tissue categories to obtain an image training set containing location labels. The tissue categories in the images were labeled, and software was used to annotate the spinal tissue regions with labels. Finally, a new dataset containing images and labels was obtained. The spinal endoscopic image set with location labels was divided into training, validation, and test sets according to a preset ratio. The improved YOLOv8 model was iteratively trained based on the training, validation, and test sets to obtain a trained improved YOLOv8 model. The performance of the trained improved YOLOv8 model was evaluated to obtain the mAP80-95 value. The mAP80-95 (mean Average Precision from IoU=0.80 to...) is... mAP80-95 is a commonly used evaluation metric in instance segmentation tasks. It is an important indicator for measuring the overall detection capability of a model. The higher the mAP80-95, the stronger the overall detection capability. The parameters of the trained improved YOLOv8 model are adjusted based on the mAP80-95 value. Ground truth boxes are determined based on the validation set. The trained improved YOLOv8 model is then used to identify and segment spinal endoscopic tissue images on the validation set, obtaining predicted boxes and a mask. The predicted boxes are the target bounding boxes predicted by the model, representing the model's prediction of the target's position in the image. The ground truth boxes are the bounding boxes manually annotated by the annotators, representing the target's actual position in the image. Therefore, ground truth boxes can be determined on the validation set, and the trained improved YOLOv8 model is used to classify spinal endoscopic tissue images on the validation set to obtain predicted boxes. Simultaneously, a mask is generated. The mask is an irregularly shaped image that fits the contour, accurate to the pixel level, and can identify its true shape and size.

[0027] In some embodiments, in step S102, the spinal endoscopic image is identified and segmented based on a trained improved YOLOv8 model to obtain a segmentation result. The improved YOLOv8 model includes a backbone layer, a bottleneck layer, and a detection head layer. Features are extracted from the spinal endoscopic image based on the backbone layer to obtain an initial feature map; features are fused from the initial feature map based on the bottleneck layer to obtain a test feature map; and instance segmentation is performed on the test feature map based on the detection head layer to obtain a segmentation result. For a schematic diagram of the backbone layer, please refer to [link to schematic diagram]. Figure 2 ,like Figure 2 As shown, the backbone network layer includes a CBS module, a C2f (Cross Stage Partial Bottleneck with 2 convolutions) module with embedded attention mechanism, and an SPPF (Spatial Pyramid Pooling–Fast) module. The CBS module includes two-dimensional convolution (Conv), batch normalization (BN), and an activation function (SiLU). For a schematic diagram of the C2f module, please refer to [link to schematic diagram]. Figure 3 ,like Figure 3 As shown, the C2f module includes Conv (two-dimensional convolutional) layers, Split layers, EMAblock (attention mechanism) layers, and Concat layers; the bottleneck network layer uses the ASF-YOLO architecture. For a schematic diagram of the ASF-YOLO architecture, please refer to [link to schematic diagram]. Figure 4 ,like Figure 4 As shown, ASF-YOLO (Adaptive Spatial Feature Fusion) includes a C2f module with an embedded attention mechanism and a TFE module.

[0028] Feature extraction of spinal endoscopic images is performed based on the backbone network layer to obtain an initial feature map. Specifically, the CBS module sequentially performs convolution, batch normalization, and nonlinear transformation of the activation function on the spinal endoscopic images to obtain a multi-scale feature map. The CBS module consists of three basic operations: two-dimensional convolution (Conv), batch normalization (BN), and the SiLU activation function. The two-dimensional convolution uses... Convolution is used to better adapt to different input image sizes and kernel sizes. Batch normalization is then performed on the convolutional feature maps to effectively improve the model's generalization ability. Finally, the SiLU activation function is used to process the feature maps to give them better non-linear features. The C2f module performs initial feature extraction on multi-scale feature maps using the Conv layer. The Split layer then segments the extracted feature maps to obtain a first and a second feature map. An EMAblock layer dynamically assigns weights to the second feature map, and a Concat layer connects the dynamically weighted second feature map to the first feature map along the channel dimension to obtain a fused feature map. The input data to the C2f module first passes through a first convolutional layer. The output is divided into two parts: one part is sent directly to the output, and the other part is processed through multiple attention mechanisms (EMAblock). Finally, the results of these two parts are concatenated along the channel dimension and passed through a second convolutional layer (Concat) to obtain the final output. The C2f module primarily uses local convolutional operations for feature extraction, excelling at capturing local spatial information. The C2f module employs a cross-stage feature fusion approach, effectively fusing and interacting high- and low-scale feature maps. This allows the model to better handle scale variations and different object sizes, and it can simultaneously receive feature maps of different scales and perform different levels of feature extraction and fusion operations. The module introduces an attention mechanism to enhance the model's responsiveness to key structural regions, making the model more focused on segmented regions and identifying edge tissues. Moreover, this attention mechanism is a lightweight attention computation structure, which reduces the amount of computation. While ensuring model accuracy, it greatly reduces FLOPs and the number of parameters, making it suitable for deployment on edge devices. The SPPF module performs multi-scale max pooling on the fused feature map to obtain the initial feature map. The SPPF module uses pyramid-shaped pooling operations to capture features of different ranges, thereby expanding the model's receptive field. The pooling results of different sizes are cascaded and fused to improve the model's feature representation ability and classification performance. The information extracted by multi-scale pooling helps the model capture the global semantics and local details of the target at different scales, thus improving the expressiveness of the feature map.

[0029] In the backbone network, useful features are selected through the CBS module to reduce feature redundancy. The C2F module is then used to process the data, extract useful information from deep and shallow feature maps, and fuse them together to enhance the accuracy of multi-scale detection. This ensures lightweight operation while obtaining richer gradient flow information.

[0030] The initial feature map is fused based on the bottleneck network layer to obtain the test feature map. Specifically, ASF-YOLO is used to extract multi-level features from the initial feature map, and the extracted multi-level features are fused using a fusion factor to obtain the test feature map. Specifically, the TFE and C2F modules are used to perform deep feature extraction on the initial feature map to obtain multi-level features. The bottleneck network (Neck) is replaced with ASF-YOLO, which extracts multi-level features from the sample data and fuses them using a fusion factor to achieve spatial adaptive fusion of multi-scale features, improving the recognition accuracy of small targets and boundary structures, thereby improving the model's detection accuracy. The ASF-YOLO Triple Feature Encoder (TFE) is used in this process. The Encoder module captures local details of small objects, enabling the integration of local and global feature information to produce more accurate segmentation maps. The Triple Feature Encoding (TEF) module adjusts the number of feature channels before feature encoding to ensure consistency with the main scale characteristics. After processing the large feature map (Large) by the convolution module, its channel count is adjusted, and then a hybrid structure of max pooling and average pooling is used for downsampling. This helps reduce the spatial dimension of the features and achieves translation invariance, thereby enhancing the network's robustness to spatial changes and translations in the input image. For the small feature map (Small), the convolution module is used to adjust the channel count, and then nearest neighbor interpolation is used. This involves selecting the value of the nearest original pixel at the target location as the result for upsampling, which helps preserve local features and prevents the loss of small object feature information, as nearest neighbor interpolation can fill the feature map with neighboring pixels and consider sub-pixel neighborhoods. Finally, the three feature maps of the same size (large, medium, and small) are convolved once and then concatenated along the channel dimension.

[0031] The detection head performs instance segmentation on the feature map under test to obtain the segmentation result. For a schematic diagram of the detection head structure, please refer to [link to schematic diagram]. Figure 5 ,like Figure 5 As shown, the detection head layer includes a shared convolutional module and multiple masking modules. The shared convolutional module is used to extract the prediction bounding box and spinal tissue category information of the spinal tissue endoscopic image. The multiple masking modules are used to extract the masking features of the spinal tissue endoscopic image. The shared convolutional module includes a first convolutional group normalization (Conv_GN). ), second convolutional group normalization (Conv_GN) The system includes a classification convolutional layer (Conv_Cls), a regression convolutional layer (Conv_Reg), a scale adjustment module (Scale), and multiple masking modules, including a first masking module, a second masking module, and a third masking module. The first, second, and third masking modules have the same structure and all include... Convolutional group normalization (Conv_GN) ), Convolutional group normalization (Conv_GN) ) and masked convolutional layers (Conv_Mask), the first mask module, the second mask module, and the third mask module. The convolutional group normalization is connected to the first convolutional group normalization of the shared convolutional module; the first mask module's The convolutional group normalization corresponds to P3, and the second mask module's The convolutional group normalization corresponds to P4, and the third mask module's The convolutional group normalization corresponds to P5, the first mask module, the second mask module, and the third mask module. The convolutional group normalization is respectively connected to the first mask module, the second mask module, and the third mask module. The convolutional group normalization is performed, and the second convolutional group normalization, which shares a convolutional module, is concatenated with the first convolutional group normalization. The detection head layer performs instance segmentation, specifically by performing convolution and group normalization, aggregation, and masking on the feature map to be tested through multiple masking modules to obtain a first feature mask, a second feature mask, and a third feature mask. Specifically, the first masking module, the second masking module, and the third masking module... The convolutional group normalization is applied to the feature maps to be tested, respectively, to obtain the first feature map, the second feature map, and the third feature map; then, through the first mask module, the second mask module, and the third mask module... The convolutional group normalization is used to aggregate the first, second, and third feature maps, respectively. Then, masking processing is performed through the mask convolutional layers of the first, second, and third mask modules to obtain the first, second, and third feature masks. The first convolutional group normalization of the shared convolutional module shares the first, second, and third feature maps, and the second convolutional group normalization aggregates them. A classification convolutional layer performs classification operations on the aggregated feature maps to obtain spinal tissue category information. A regression convolutional layer further refines the aggregated feature maps. After regression analysis on the combined feature maps, the scale adjustment module adaptively scales the feature maps to obtain predicted bounding boxes. Segmentation results are obtained based on the first feature mask, second feature mask, third feature mask, spinal tissue category information, and predicted bounding boxes. A shared convolutional module replaces the conventional convolutional module in the original detection head, adding additional detection heads at key information segmentation points. Dense feature sharing is performed between each block. The segmentation head receives input features from layers P3 to P5, corresponding to the first, second, and third feature maps. Specifically, the batch normalization (BN) structure in the convolution is replaced with group normalization (GN). GN is a technique used in deep learning to normalize features in convolutional neural networks. It is more stable than BN on small-batch datasets. GN alleviates the poor performance of BN in small-batch training by dividing channels into multiple groups and performing normalization independently within each group. To enhance information flow between channels, a... Convolution is used for cross-channel information fusion, followed by two convolutions with shared weights. Convolutional kernels further aggregate features, effectively suppressing redundant information and improving local context modeling capabilities. Furthermore, the aggregated features are fed into classification and regression heads for object detection and prediction. A scale adjustment layer is introduced to adaptively scale multi-scale features, preserving discriminative information at different levels. This not only enhances the richness of feature representation but also significantly reduces the number of parameters and computational overhead, achieving a lightweight and efficient model. The lightweight detection head uses shared convolutions, which greatly reduces the number of parameters, making the model more portable. While using shared convolutions, to address the issue of inconsistent target scales detected by each detection head, a scale module is used to scale features, allowing the detection head to achieve fewer parameters and less computation while minimizing accuracy loss.

[0032] In some embodiments, in step S103, the category and location of spinal tissue are determined based on the segmentation results. The categories of spinal tissue include ligamentum flavum, nerve roots, bone tissue, and fat; see the schematic diagram of the segmentation results. Figure 6a , Figure 6b ,like Figure 6a , 6b As shown, real-time instance segmentation is performed using an improved YOLOv8 model, which monitors images in the intervertebral foramen lens in real time and identifies the types of spinal tissues seen. Automatic color and text labeling is also performed to distinguish between them. By identifying different tissues or lesion areas in the spinal endoscopy images, the results are visually fed back to the doctor to help the doctor more accurately identify key tissue structures and lesion areas during the operation, thereby improving the safety and efficiency of the operation.

[0033] The original YOLOv8 model and the improved YOLOv8 model are compared. Please refer to Table 1 for a comparison of the various metrics between the original and improved YOLOv8 models. Table 1

[0034] In Table 1, Floating Point Operations (FLOPs), the number of floating-point operations per second, is commonly used in deep learning to measure the computational complexity of a model. The lower the FLOPs, the less computation the model requires for each inference. Frames Per Second (FPS) is a metric for measuring the inference speed of a model in deep learning. FPS indicates how many images the model can process per second; the higher the FPS, the faster the response speed. mAP50-95 is a commonly used evaluation metric in instance segmentation tasks. It is an important indicator for measuring the overall detection capability of a model. The higher the mAP50-95, the stronger the overall detection capability. As shown in Table 1, compared to the original YOLOv8 model, the improved YOLOv8 model has lower FLOPs and higher FPS, more accurate segmentation results, a lighter model, and a faster response, making it very suitable for deployment in hospitals to assist doctors in real-time surgical diagnosis.

[0035] In summary, the spinal endoscopic tissue instance segmentation method provided by this invention acquires endoscopic images of spinal tissue in real time during surgery; it identifies and segments the spinal tissue endoscopic images based on a trained improved YOLOv8 model to obtain segmentation results. The improved YOLOv8 model includes a backbone network layer, a bottleneck network layer, and a detection head layer. The detection head layer includes a shared convolutional module and multiple mask modules. Based on the segmentation results, the method determines the category and location of spinal tissue, thereby improving the segmentation accuracy of spinal endoscopic images.

[0036] To better implement the spinal endoscopic tissue instance segmentation method in this embodiment of the invention, based on the spinal endoscopic tissue instance segmentation method, correspondingly, as follows: Figure 7 As shown, this embodiment of the invention also provides a spinal endoscopic tissue instance segmentation device, the spinal endoscopic tissue instance segmentation device 700 comprising: Image acquisition module 701 is used to acquire endoscopic images of spinal tissues in real time during surgery; The instance segmentation module 702 is used to identify and segment spinal tissue endoscopic images based on a trained improved YOLOv8 model to obtain segmentation results. The improved YOLOv8 model includes a backbone network layer, a bottleneck network layer, and a detection head layer. The detection head layer includes a shared convolutional module and multiple masking modules. The shared convolutional module is used to extract the prediction box and spinal tissue category information of the spinal tissue endoscopic image, and the multiple masking modules are used to extract the masking features of the spinal tissue endoscopic image. The spinal tissue determination module 703 is used to determine the category and location of spinal tissue based on the segmentation results.

[0037] like Figure 8 As shown, the present invention also provides a real-time surgical assistance device 800, which can be a mobile terminal, desktop computer, laptop, handheld computer, server, or other computing device. The real-time surgical assistance device 800 includes a processor 801, a memory 802, and a display 803. Figure 8 Only some components of the surgical real-time assist device 800 are shown; however, it should be understood that implementation of all shown components is not required, and more or fewer components may be implemented instead.

[0038] In some embodiments, the memory 802 may be an internal storage unit of the real-time surgical assistive device 800, such as a hard disk or memory of the real-time surgical assistive device 800. In other embodiments, the memory 802 may be an external storage device of the real-time surgical assistive device 800, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the real-time surgical assistive device 800. Further, the memory 802 may include both internal and external storage units of the real-time surgical assistive device 800. The memory 802 is used to store application software and various types of data installed on the real-time surgical assistive device 800, such as the program code for installing the real-time surgical assistive device 800. The memory 802 may also be used to temporarily store data that has been output or will be output. In one embodiment, the memory 802 stores a spinal endoscopy tissue instance segmentation program, which can be executed by the processor 801 to implement the spinal endoscopy tissue instance segmentation method of various embodiments of the present invention.

[0039] In some embodiments, processor 801 may be a central processing unit (CPU), microprocessor or other data processing chip, used to run program code stored in memory 802 or process data, such as a spinal endoscopy tissue instance segmentation method.

[0040] In some embodiments, display 803 may be an LED display, a liquid crystal display, a touch-screen liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen. Display 803 is used to display identification information for the spinal endoscopic tissue segmentation procedure and to display a visual user interface. Components 801-803 of the real-time surgical assistance device 800 communicate with each other via a system bus.

[0041] In some embodiments, when the processor 801 executes the spinal endoscopic tissue instance segmentation program in the memory 802, it implements each step of the spinal endoscopic tissue instance segmentation method as described in the above embodiments. Since the spinal endoscopic tissue instance segmentation method has been described in detail above, it will not be repeated here.

[0042] Accordingly, the present invention also provides a computer-readable storage medium for storing a computer-readable program or instruction, which, when executed by a processor, can implement the steps or functions of the spinal endoscopic tissue instance segmentation method provided in the above-described method embodiments.

[0043] Those skilled in the art will understand that all or part of the processes of the methods described in the above embodiments can be implemented by a computer program instructing related hardware, and the program can be stored in a computer-readable storage medium. The computer-readable storage medium may be a disk, optical disk, read-only memory, or random access memory, etc.

[0044] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for segmenting tissue instances in spinal endoscopy, characterized in that, include: Real-time acquisition of endoscopic images of spinal tissues during surgery; The improved YOLOv8 model is used to identify and segment the spinal tissue endoscopic image to obtain the segmentation result. The improved YOLOv8 model includes a backbone network layer, a bottleneck network layer, and a detection head layer. The detection head layer includes a shared convolutional module and multiple masking modules. The shared convolutional module is used to extract the prediction box and spinal tissue category information of the spinal tissue endoscopic image. The multiple masking modules are used to extract the masking features of the spinal tissue endoscopic image. The category and location of spinal tissues are determined based on the segmentation results.

2. The method for segmenting tissue instances in spinal endoscopy according to claim 1, characterized in that, Before the real-time acquisition of endoscopic images of spinal tissue during surgery, the following steps are included: The acquired spinal tissue endoscopic images are preprocessed to obtain a preprocessed spinal tissue endoscopic image set. Tissue category labeling is performed on the preprocessed spinal tissue endoscopic image set to obtain a spinal tissue endoscopic image set containing location labels; The set of spinal tissue endoscopic images containing location labels is divided into a training set, a validation set, and a test set according to a preset ratio. Based on the training set, validation set, and test set, the improved YOLOv8 model to be trained is iteratively trained to obtain a well-trained improved YOLOv8 model.

3. The method for segmenting tissue instances in spinal endoscopy according to claim 1, characterized in that, The improved YOLOv8 model, based on training, is used to identify and segment the endoscopic images of spinal tissue, obtaining segmentation results including: Based on the backbone network layer, feature extraction is performed on the endoscopic image of the spinal tissue to obtain an initial feature map; Based on the bottleneck network layer, feature fusion is performed on the initial feature map to obtain the feature map to be tested; The detection head layer is used to perform instance segmentation on the feature map to be tested, and the segmentation result is obtained.

4. The method for segmenting tissue instances in spinal endoscopy according to claim 3, characterized in that, The backbone network layer includes a CBS module, a C2F module, and an SPPF module. The CBS module includes two-dimensional convolution, batch normalization, and activation functions. The C2F module includes convolutional layers, segmentation layers, attention mechanism layers, and connection layers. The step of extracting features from the endoscopic image of the spinal tissue based on the backbone network layer to obtain an initial feature map includes: The CBS module sequentially performs convolution processing, batch normalization processing, and nonlinear change processing of activation function on the endoscopic images of the spinal tissue to obtain multi-scale feature maps. The C2f module performs preliminary feature extraction on the multi-scale feature map through convolutional layers. The feature map is then segmented by a segmentation layer to obtain a first feature map and a second feature map. The second feature map is dynamically weighted by an attention mechanism layer, and then connected to the first feature map in the channel dimension by a connection layer to obtain a fused feature map. The initial feature map is obtained by performing multi-scale max pooling on the fused feature map using the SPPF module.

5. The method for segmenting tissue instances in spinal endoscopy according to claim 4, characterized in that, The bottleneck network layer has an ASF-YOLO architecture, which includes a C2f module and a TFE module. The step of fusing features from the initial feature map based on the bottleneck network layer to obtain the feature map to be tested includes: ASF-YOLO is used to extract multi-level features from the initial feature map, and the extracted multi-level features are fused by a fusion factor to obtain the feature map to be tested. In particular, TFE and C2F modules are used to perform deep feature extraction on the initial feature map to obtain multi-level features.

6. The method for segmenting tissue instances in spinal endoscopy according to claim 5, characterized in that, The shared convolutional module includes a first convolutional group normalization, a second convolutional group normalization, a classification convolutional layer, a regression convolutional layer, and a scale adjustment module. The multiple masking modules include a first masking module, a second masking module, and a third masking module. The first masking module, the second masking module, and the third masking module have the same structure. The first masking module includes... Convolutional group normalization, The convolutional group normalization and masked convolutional layers, the first mask module, the second mask module, and the third mask module. The convolutional group normalization is connected to the first convolutional group normalization of the shared convolutional module; the step of identifying and segmenting the feature map to be tested based on the detection head layer to obtain the segmentation result includes: The feature map to be tested is subjected to convolution, group normalization, aggregation, and masking processing by multiple masking modules to obtain a first feature mask, a second feature mask, and a third feature mask. The first convolutional group of the shared convolutional module is normalized to share the first mask module, the second mask module, and the third mask module. The convolutional group normalization generates a first feature map, a second feature map, and a third feature map. The first feature map, the second feature map, and the third feature map are aggregated by the second convolutional group normalization. The aggregated feature map is classified by a classification convolutional layer to obtain spinal tissue category information. The aggregated feature map is regressed by a regression convolutional layer. Finally, the feature map after regression is adaptively scaled by a scaling module to obtain the prediction box. The segmentation result is obtained based on the first feature mask, the second feature mask, the third feature mask, the spinal tissue category information, and the prediction box.

7. The method for segmenting tissue instances in spinal endoscopy according to claim 3, characterized in that, The categories of spinal tissues include the ligamentum flavum, nerve roots, bone tissue, and fat.

8. A spinal endoscopic tissue segmentation device, characterized in that, include: The image acquisition module is used to acquire endoscopic images of spinal tissues in real time during surgery; The instance segmentation module is used to identify and segment the spinal tissue endoscopic image based on a trained improved YOLOv8 model to obtain segmentation results. The improved YOLOv8 model includes a backbone network layer, a bottleneck network layer, and a detection head layer. The detection head layer includes a shared convolutional module and multiple masking modules. The shared convolutional module is used to extract the prediction box and spinal tissue category information of the spinal tissue endoscopic image. The multiple masking modules are used to extract the masking features of the spinal tissue endoscopic image. A spinal tissue determination module is used to determine the category and location of spinal tissue based on the segmentation results.

9. A real-time surgical assistance device, characterized in that, Including memory and processor; The memory stores a computer-readable program that can be executed by the processor; When the processor executes the computer-readable program, it implements the steps in the spinal endoscopic tissue instance segmentation method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, Used to store computer-readable programs or instructions, which, when executed by a processor, can perform the steps in the spinal endoscopic tissue instance segmentation method according to any one of claims 1-7.

Images