A dynamic segmentation method of pulmonary vascular based on surgical video

Abstract

The present application provides a method for dynamic segmentation of pulmonary blood vessels based on surgical videos, which relates to the field of medical image technology and solves the technical problem of low accuracy in blood vessel recognition. The method includes: obtaining the original image of the original surgical video by taking screenshots at alternate frames, and extracting the video frame sequence of the original surgical video; determining the complementary information between adjacent frame images in the original surgical video according to the video frame sequence and the original image, and performing super-resolution reconstruction based on the complementary information to obtain a surgical image with a higher resolution than the preset high resolution; constructing an initial blood vessel segmentation model based on TransUNet; training the initial blood vessel segmentation model using surgical images to obtain a trained blood vessel segmentation model; inputting the surgical video to be segmented into the trained blood vessel segmentation model, and dynamically segmenting the blood vessels in the surgical video to be segmented using the trained blood vessel segmentation model to obtain a real-time contour map of the blood vessels in the surgical video to be segmented.

Description

Technical Field

[0001] The present application relates to the field of medical image technology, and in particular to a method for dynamic segmentation of pulmonary blood vessels based on surgical videos. Background Art

[0002] At present, modern surgery is more pursuing minimally invasive, precise and diversified diagnosis and treatment. Surgeons are becoming more specialized and precise in the diagnosis and treatment of diseases, which has led to the emergence of various subspecialties, with the purpose of making surgical treatment more precise, such as gastrointestinal surgery, thoracic surgery, hepatobiliary surgery, anorectal surgery and urology. At the same time, surgical methods have also developed from classic laparotomy and thoracotomy to the current laparoscopy, thoracoscopy and robot-assisted precision minimally invasive surgery. However, no matter how the surgical methods and related technologies develop, a full understanding of the local anatomical structure of the lesion and smooth exposure of free blood vessels are important prerequisites for the success of the operation. Especially in today's minimally invasive precision surgery, once vascular damage occurs, it will significantly increase the risk of surgery, leading to surgical failure or reducing the quality of postoperative prognosis. A full understanding of the local vascular course and full exposure of the blood vessels will be able to avoid vascular damage to a greater extent, reduce surgical accidents, and reduce the possibility of conversion to open surgery. In many surgical fields, there are many examples of poor intraoperative vascular exposure, vascular damage, intraoperative bleeding, and surgical failure, forced changes in surgical plans, and even reduced prognosis and quality of life.

[0003] At present, the method of accurately judging the local blood vessel course of the lesion relies more on the rich experience of surgeons to gradually expose the local blood vessels of the free lesion. For minimally invasive surgery that requires more precision, especially when there are many local blood vessel variations, the accuracy of this existing blood vessel identification method is low. Summary of the invention

[0004] The purpose of the present invention is to provide a pulmonary vascular dynamic segmentation method based on surgical videos to solve the technical problem of low vessel recognition accuracy in existing vessel recognition methods.

[0005] In a first aspect, an embodiment of the present application provides a method for dynamic segmentation of pulmonary blood vessels based on surgical videos, the method comprising:

[0006] Obtaining an original surgical video, obtaining an original image of the original surgical video by taking screenshots of alternate frames, and extracting a video frame sequence of the original surgical video;

[0007] Determining complementary information between adjacent frame images in the original surgical video according to the video frame sequence and the original image, and performing super-resolution reconstruction based on the complementary information to obtain a surgical image with a resolution higher than a preset high resolution;

[0008] Constructing an initial blood vessel segmentation model based on TransUNet (‌Transformer-based U-Net); the initial blood vessel segmentation model includes an encoder and a decoder, the segmentation model is used to extract blood vessel features through a convolutional neural network, generate a blood vessel feature map, convert the blood vessel feature map into a blood vessel sequence map, and capture global context information based on the blood vessel sequence map through the encoder, and restore the spatial resolution of the encoder output through the decoder;

[0009] The initial blood vessel segmentation model is trained using the surgical image to upsample the blood vessel feature map from a preset low resolution to the resolution of the surgical image, and the target blood vessel feature map in the encoder that is higher than the preset high resolution is directly combined with the global context feature by using the encoder and the decoder through a jump connection to obtain a trained blood vessel segmentation model; the result of combining the target blood vessel feature map with the global context feature is used to dynamically identify and locate the position of the blood vessel in the surgical image in real time;

[0010] The surgical video to be segmented is input into the trained blood vessel segmentation model, and the blood vessels in the surgical video to be segmented are dynamically segmented by the trained blood vessel segmentation model to obtain a real-time contour map of the blood vessels in the surgical video to be segmented.

[0011] In a possible implementation, before dynamically segmenting the blood vessels in the surgical video to be segmented by using the trained blood vessel segmentation model to obtain a real-time contour map of the blood vessels in the surgical video to be segmented, the method further includes:

[0012] Determine the one-to-one correspondence between adjacent frame images in the surgical video to be segmented by an image registration algorithm; wherein there is displacement or rotation between adjacent frame images in the surgical video to be segmented;

[0013] The surgical video to be segmented is reconstructed with super resolution by utilizing the one-to-one correspondence between adjacent frame images in the surgical video to be segmented, so as to obtain a target surgical video with a resolution higher than the preset high resolution.

[0014] In a possible implementation, dynamically segmenting the blood vessels in the surgical video to be segmented by using the trained blood vessel segmentation model to obtain a real-time contour map of the blood vessels in the surgical video to be segmented includes:

[0015] Based on a one-to-one correspondence between adjacent frame images in the surgical video to be segmented, adjacent frame images in the target surgical video containing the same blood vessel object are determined as related frame images;

[0016] The blood vessels in the target surgical video are dynamically segmented by using the trained blood vessel segmentation model, and when segmenting the blood vessels in the relevant frame images, the blood vessels in each relevant frame image are dynamically segmented based on the blood vessel segmentation results of multiple relevant frame images of the previous frame and the next frame of each relevant frame image, so as to maintain the correlation of the dynamic blood vessel segmentation results and obtain a real-time contour map of the blood vessels in the surgical video to be segmented.

[0017] In one possible implementation, the image registration algorithm is performed by the following steps:

[0018] For deformations greater than a preset deformation degree in the image of the surgical video to be segmented, the energy function of the Beltrami coefficient term is minimized, and the Beltrami coefficient corresponding to the bijective and smooth mapping is obtained by minimizing the energy function; wherein the Beltrami coefficient term is used to measure the degree of distortion of the quasi-conformal mapping and control the bijectivity and smoothness of the mapping; and / or,

[0019] A priori markers are obtained through a marking operation, and for deformations greater than a preset deformation degree in the image of the surgical video to be segmented, the registration between the images is calculated based on the marking constraints corresponding to the priori markers to obtain a registration result that maintains orientation homeomorphism.

[0020] In a possible implementation, the surgical images are divided into a training set and a test set, and the training set is used to train the initial blood vessel segmentation model; after the trained segmentation model is obtained, the following steps are also looped until the optimal segmentation model is obtained after adjustment:

[0021] Using the trained segmentation model as the segmentation model to be tested, and using the test set to test the performance of the segmentation model to be tested to obtain a test result;

[0022] The trained segmentation model is adjusted according to the test result to obtain an adjusted segmentation model, and the adjusted segmentation model is used as a new segmentation model to be tested.

[0023] In a possible implementation, after the super-resolution reconstruction is performed based on the complementary information to obtain a surgical image with a resolution higher than a preset high resolution, the method further includes:

[0024] grayscale the surgical image to eliminate color information of the surgical image and obtain a grayscale surgical image;

[0025] The distribution of pixel values ​​in the grayscale surgical image is adjusted by histogram equalization to obtain a histogram with a smoothness greater than a preset smoothness, and the noise in the histogram is reduced by a Gaussian filter to make the blood vessel area in the histogram stand out.

[0026] In one possible implementation, the encoder is a CNN-Transformer hybrid encoder, and the CNN-Transformer hybrid encoder includes multi-head attention and multi-layer perceptron;

[0027] The decoder includes multiple upsampling steps, each of which includes an upsampling operation, a convolution operation, and a linear rectification function (ReLU) activation layer.

[0028] In a possible implementation, after the super-resolution reconstruction is performed based on the complementary information to obtain a surgical image with a resolution higher than a preset high resolution, the method further includes:

[0029] The surgical image is data enhanced by means of rotation, horizontal mirroring, noise addition, brightness transformation and random translation to obtain a data enhancement result.

[0030] In a second aspect, an embodiment of the present application provides a pulmonary vascular dynamic segmentation device based on a surgical video, the device comprising:

[0031] An acquisition module is used to acquire the original surgical video, obtain the original image of the original surgical video by taking screenshots of alternate frames, and extract the video frame sequence of the original surgical video;

[0032] A determination module, configured to determine complementary information between adjacent frame images in the original surgical video according to the video frame sequence and the original image, and to perform super-resolution reconstruction based on the complementary information to obtain a surgical image with a resolution higher than a preset high resolution;

[0033] A construction module is used to construct an initial blood vessel segmentation model based on TransUNet; the initial blood vessel segmentation model includes an encoder and a decoder, the segmentation model is used to extract blood vessel features through a convolutional neural network, generate a blood vessel feature map, convert the blood vessel feature map into a blood vessel sequence map, and capture global context information based on the blood vessel sequence map through the encoder, and restore the spatial resolution of the output of the encoder through the decoder;

[0034] A training module, used to train the initial blood vessel segmentation model using the surgical image to upsample the blood vessel feature map from a preset low resolution to the resolution of the surgical image, and to combine the target blood vessel feature map in the encoder that is higher than the preset high resolution with the global context feature directly through a jump connection using the encoder and the decoder to obtain a trained blood vessel segmentation model; the result of combining the target blood vessel feature map with the global context feature is used to dynamically identify and locate the position of the blood vessel in the surgical image in real time;

[0035] The segmentation module is used to input the surgical video to be segmented into the trained blood vessel segmentation model, dynamically segment the blood vessels in the surgical video to be segmented through the trained blood vessel segmentation model, and obtain a real-time contour map of the blood vessels in the surgical video to be segmented.

[0036] In a third aspect, an embodiment of the present application provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program that can be executed on the processor, and when the processor executes the computer program, the method described in the first aspect is implemented.

[0037] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions. When the computer-executable instructions are called and executed by a processor, the computer-executable instructions prompt the processor to execute the method described in the first aspect above.

[0038] The embodiments of the present application bring the following beneficial effects:

[0039] The embodiment of the present application provides a method for dynamic segmentation of pulmonary blood vessels based on surgical videos, which can obtain the original surgical video, obtain the original image of the original surgical video by taking screenshots at alternate frames, and extract the video frame sequence of the original surgical video; determine the complementary information between adjacent frame images in the original surgical video according to the video frame sequence and the original image, and perform super-resolution reconstruction based on the complementary information to obtain a surgical image with a higher resolution than a preset high resolution; construct an initial blood vessel segmentation model based on TransUNet; the initial blood vessel segmentation model includes an encoder and a decoder, and the segmentation model is used to extract blood vessel features through a convolutional neural network to generate a blood vessel feature map, convert the blood vessel feature map into a blood vessel sequence map, and capture global context information based on the blood vessel sequence map through the encoder, and through the decoder The decoder restores the spatial resolution of the output of the encoder; the surgical image is used to train the initial blood vessel segmentation model to upsample the blood vessel feature map from a preset low resolution to the resolution of the surgical image, and the encoder and the decoder are used to directly combine the target blood vessel feature map in the encoder that is higher than the preset high resolution with the global context features through a jump connection to obtain a trained blood vessel segmentation model; the result of combining the target blood vessel feature map with the global context features is used to dynamically identify and locate the position of the blood vessels in the surgical image in real time; the surgical video to be segmented is input into the trained blood vessel segmentation model, and the blood vessels in the surgical video to be segmented are dynamically segmented by the trained blood vessel segmentation model to obtain a real-time contour map of the blood vessels in the surgical video to be segmented. In this scheme, since the object of dynamic blood vessel segmentation is a video, and adjacent frames in the video have extremely high similarity, a higher-resolution surgical video can be obtained by super-resolution reconstruction with the help of complementary information between adjacent frames. Moreover, the constructed initial blood vessel segmentation model is trained by this higher-resolution surgical image, and the blood vessel feature map can be upsampled from a preset low resolution to the resolution of the surgical image. The encoder and decoder are used to directly combine the high-resolution target blood vessel feature map in the encoder with the global context feature through a jump connection, thereby realizing dynamic recognition and real-time positioning of the blood vessel position in the surgical image, so that the final trained blood vessel segmentation model has higher accuracy, improves the accuracy of blood vessel recognition, and solves the technical problem of low blood vessel recognition accuracy in existing blood vessel recognition methods.

[0040] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the specific implementation methods of the present application or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0042] Figure 1 A schematic diagram of the prior art showing the left superior pulmonary vein being damaged due to poor local vascular exposure;

[0043] Figure 2 It is a schematic diagram of the free exposure of pulmonary blood vessels step by step based on experience in the prior art;

[0044] Figure 3 A schematic diagram of a flow chart of a method for dynamic segmentation of pulmonary blood vessels based on surgical videos provided in an embodiment of the present application;

[0045] Figure 4 An example of a segmentation model based on TransUNet in the dynamic segmentation method of pulmonary blood vessels based on surgical videos provided in an embodiment of the present application;

[0046] FIG5( a ) is an example diagram of registration in a registration result with folding in the prior art;

[0047] FIG5( b ) is another example diagram of registration in the registration result with folding in the prior art;

[0048] FIG5 (c) is an example of a registration result with obvious folding phenomenon in the prior art after registration;

[0049] FIG5(d) is an example of a grid corresponding to a mapping in a registration result with folding in the prior art;

[0050] Figure 6 An example of the registration result of the image registration algorithm based on quasi-conformal geometry provided in the embodiment of the present application on the ACDC heart data set;

[0051] Figure 7 An example of an image registration result based on given priori marking information provided in an embodiment of the present application;

[0052] Figure 8 A schematic diagram of the structure of a pulmonary vascular dynamic segmentation device based on surgical video provided in an embodiment of the present application;

[0053] Fig. 9 A schematic structural diagram of an electronic device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0054] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings. Apparently, the described embodiments are some, but not all, of the embodiments of this application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of this application without creative efforts shall fall within the scope of protection of this application.

[0055] As used in the embodiments of this application, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but optionally further includes other steps or units not listed, or optionally further includes other steps or units inherent to these processes, methods, products, or devices.

[0056] Currently, there are also some research and application technologies for assisting in the judgment of local lesions and blood vessels in current clinical practice, such as preoperative three-dimensional reconstruction planning technology, indocyanine green fluorescence injection imaging technology, etc. Among them, the indocyanine green fluorescence injection imaging technology has many defects and is currently less used in minimally invasive chest surgery. Currently, the preoperative three-dimensional reconstruction planning technology using application software is more widely used in clinical practice, mainly including software such as "vitaworks, inlook 3D, and Zero氪 3D". Its principle is to reconstruct data with the help of CT imaging results before surgery to construct local relationships such as lesions and local blood vessels. Currently, this technology has achieved good results in imaging local anatomical structures. Especially for vitaworks, it can better assist doctors in visually understanding the size, shape, and relationship with surrounding blood vessels and tissue structures in pulmonary segments, sub-segments, and hepatic segments, playing a role in avoiding tissue damage and reducing surgical risks. This technology can accurately analyze the position information of lesion nodules, and the reconstructed three-dimensional image data information can intuitively show the internal structure of the tissue and the three-dimensional relationship of the tumor, so as to locate the spatial position and anatomical distance of the lesion nodules and intuitively understand the variation of the internal structure of the patient's tissue. This technology plays an important reference role in assisting surgeons in understanding the local anatomical structure, position, and adjacent relationship with blood vessels, etc. of lesions in the tissue. Its starting point is mainly based on accurately judging the local structure and relationship of lesions in the tissue before surgery, playing an important guiding role in designing the surgical plan during surgery and precisely excising the lesion.

[0057] In many surgical fields, there are many cases where poor vascular exposure during surgery causes vascular damage, massive bleeding during surgery, and leads to surgical failure, forced changes in surgical plans, and even reduced prognosis and quality of life. In thoracic surgery, there are many literature reports on unexpected situations during thoracoscopic anatomical lung resection. The most direct manifestation of this is the analysis of the reasons for thoracoscopic conversion to open chest surgery. Common causes include vascular injury and bleeding, pulmonary fissure dysplasia, tumor factors, vascular anatomical variations, etc. Among them, the proportion of bleeding due to vascular injury reaches 10-15%. Figure 1 The local vascular exposure shown was poor, resulting in damage to the left superior pulmonary vein and massive intraoperative bleeding.

[0058] Currently, there are no more technical methods for free dissection of local pulmonary vessels and complete and smooth exposure, and there is still a lack of effective means for intraoperative dynamic vascular imaging and other technologies. It is more dependent on the surgeon's experience, familiarity with the local vascular structure and surgical operation skills to reduce vascular damage, intraoperative vascular rupture, massive bleeding and other intraoperative risks, such as Figure 2 Shown is an empirical step-by-step approach to free exposure of the pulmonary vasculature.

[0059] Regardless of which of the above schemes, including preoperative three-dimensional reconstruction planning technology, indocyanine green fluorescence injection imaging technology, etc., there are obvious defects in the process of intraoperative application. First, neither of them can expose blood vessels in real time during surgery, and cannot dynamically assist surgeons in performing vascular free operations; second, the defects of indocyanine green fluorescence injection imaging technology mainly include: 1) It requires repeated invasive injections before and during surgery, which can easily cause patient discomfort and affect surgical efficiency; 2) Indocyanine green fluorescence imaging is more targeted at target areas, such as lung nodules, lymph nodes, etc., and the imaging of local large blood vessels is poor. Even due to the high tissue permeability of indocyanine green, it will cause greater interference to the surgical field and affect the operator's operation; 3) Indocyanine green fluorescence imaging technology is limited by tissue depth due to scattering and other reasons, and it still cannot image deeper tissues well.

[0060] As for the preoperative 3D reconstruction planning technology, due to the limitations of its principles and functions, this technology also has two major defects: 1) It can only play a role in preoperative planning, helping surgeons understand the structural relationship of tissue lesions before surgery, and assisting surgeons in reference and comparison during surgery, but it cannot play a role in real-time identification and display of blood vessel courses during surgery; 2) Moreover, due to the large number of local vascular variations in tissue lesions, surgeons are required to constantly compare the 3D reconstruction results of the reference lesions during surgery to ensure accurate judgment of the local vascular relationship. It is impossible to image the blood vessels in real time, which reduces the efficiency of the operation and may even cause the surgeon to lose focus, resulting in greater surgical risks and reduced surgical quality.

[0061] Based on this, an embodiment of the present application provides a method for dynamic segmentation of pulmonary blood vessels based on surgical videos, which can solve the technical problem of low blood vessel recognition accuracy in existing blood vessel recognition methods.

[0062] The embodiments of the present invention are further described below in conjunction with the accompanying drawings.

[0063] Figure 3 The flowchart of a dynamic segmentation method of pulmonary blood vessels based on surgical video provided in the embodiment of the present application is as follows. Figure 3 As shown, the method includes:

[0064] Step S110, obtaining the original surgical video, obtaining the original image of the original surgical video by taking screenshots of alternate frames, and extracting the video frame sequence of the original surgical video.

[0065] As a possible implementation, in the process of constructing the data set, firstly, the original image is obtained by taking screenshots of alternate frames of the original surgical video, and the video frame sequence of the original surgical video is extracted.

[0066] Step S120, determining complementary information between adjacent frame images in the original surgical video according to the video frame sequence and the original image, and performing super-resolution reconstruction based on the complementary information to obtain a surgical image with a resolution higher than a preset high resolution.

[0067] It should be noted that, unlike the segmentation task for a single image, the object to be segmented in the embodiment of the present application is a video, and adjacent frames in the video have extremely high similarity. Therefore, super-resolution reconstruction can be performed with the help of complementary information between adjacent frames to obtain a high-resolution surgical video.

[0068] Step S130, constructing an initial blood vessel segmentation model based on TransUNet.

[0069] Among them, the initial vascular segmentation model includes an encoder and a decoder. The segmentation model is used to extract vascular features through a convolutional neural network, generate a vascular feature map, convert the vascular feature map into a vascular sequence map, and capture global context information based on the vascular sequence map through the encoder. The encoder output is restored to spatial resolution through the decoder.

[0070] It should be noted that Transformer-based U-Net (TransUNet) is a deep learning model that combines the Transformer and U-Net network structures, aiming to improve the accuracy and efficiency of image segmentation. This model combines the global feature learning ability of Transformer and the excellent segmentation performance of U-Net, and is particularly suitable for medical image segmentation, cell nucleus image segmentation, etc.

[0071] In an optional embodiment, the encoder is a CNN-Transformer hybrid encoder, which includes multi-head attention and a multi-layer perceptron; the decoder includes multiple upsampling steps, each of which includes an upsampling operation, a convolution operation, and a ReLU activation layer. The TransUNet solution provided in the embodiment of the present application can be applied to the dynamic identification and segmentation of pulmonary vascular intraoperatively. Among them, the full name of CNN-Transformer is a hybrid model of convolutional neural network and Transformer. CNN-Transformer is a model architecture that combines convolutional neural network (CNN) and Transformer. This combination is not simply to superimpose the two together, but to integrate the Self-attention mechanism into the backbone of CNN, thereby utilizing the local perception feature extraction capability of CNN and the global representation learning capability of Transformer. This hybrid model aims to solve problems such as the inability of traditional CNN to process sequence data, while improving the performance and efficiency of the model. The application of CNN-Transformer is not limited to image processing, but also shows potential in fields such as natural language processing. By combining the local feature extraction capability of CNN and the global dependency modeling capability of Transformer, this model architecture can better handle complex data structures and patterns.

[0072] For the construction of the segmentation model based on TransUNet, for example, Figure 4 As shown in the figure, the network model mainly consists of two parts: encoder and decoder. The encoder is a CNN-Transformer hybrid encoder. First, a convolutional neural network (CNN) is used as a feature extractor to generate a feature map. The feature map is then converted into a sequence as the input of the Transformer. The Transformer consists of multi-head attention and multi-layer perceptrons to capture global context information. The decoder consists of multiple upsampling steps to restore the spatial resolution of the encoder output. Each upsampling step includes an upsampling operation, a convolution operation, and a ReLU activation layer. This process gradually upsamples the feature map from a low resolution to the resolution of the original image.

[0073] Step S140, using the surgical image to train the initial vascular segmentation model to upsample the vascular feature map from a preset low resolution to the resolution of the surgical image, and using the encoder and decoder to directly combine the target vascular feature map in the encoder that is higher than the preset high resolution with the global context feature through a jump connection to obtain a trained vascular segmentation model.

[0074] For the training process of the segmentation model, for example, Figure 4As shown in FIG. 1 , the preprocessed surgical image can be input into the constructed segmentation model for training. It should be noted that the result of combining the target vessel feature map with the global context feature is used to dynamically identify and locate the position of the blood vessel in the surgical image in real time. For example, the encoder and decoder directly combine the high-resolution feature map in the encoder with the global context feature through a jump connection to achieve accurate positioning.

[0075] Step S150, inputting the surgical video to be segmented into the trained blood vessel segmentation model, dynamically segmenting the blood vessels in the surgical video to be segmented by the trained blood vessel segmentation model, and obtaining a real-time contour map of the blood vessels in the surgical video to be segmented.

[0076] For the testing and deployment of the model, for example, the surgical video to be segmented is input into the trained segmentation model, so as to intelligently segment and obtain the contour map of the blood vessels. Finally, the model is deployed in the surgical environment to help doctors complete the operation smoothly.

[0077] In the embodiment of the present application, since the object for dynamic blood vessel segmentation is a video, and adjacent frames in the video have extremely high similarity, a higher-resolution surgical video can be obtained by super-resolution reconstruction using complementary information between adjacent frames. The constructed initial blood vessel segmentation model is then trained using this higher-resolution surgical image, and the blood vessel feature map can be upsampled from a preset low resolution to the resolution of the surgical image. The encoder and the decoder are directly used to combine the high-resolution target blood vessel feature map in the encoder with the global context feature through a jump connection, thereby realizing dynamic recognition and real-time positioning of the blood vessel position in the surgical image, so that the final trained blood vessel segmentation model has higher accuracy, thereby improving the accuracy of blood vessel recognition.

[0078] By studying the intraoperative pulmonary vascular feature recognition and segmentation, and combining the research to propose a better intraoperative pulmonary vascular real-time segmentation scheme, on this basis, the dynamic perception and segmentation of pulmonary vessels in the natural lighting surgical field is achieved, which lays an important foundation for further realizing intraoperative dynamic imaging of pulmonary vessels, thereby assisting in the exposure of free blood vessels during surgery.

[0079] The above steps are described in detail below.

[0080] In some embodiments, before dynamically segmenting the blood vessels in the surgical video to be segmented by the trained blood vessel segmentation model in the above step S150 to obtain a real-time contour map of the blood vessels in the surgical video to be segmented, the method may further include the following steps:

[0081] Determine the one-to-one correspondence between adjacent frame images in the surgical video to be segmented by an image registration algorithm; wherein there is displacement or rotation between adjacent frame images in the surgical video to be segmented;

[0082] The one-to-one correspondence between adjacent frame images in the surgical video to be segmented is used to perform super-resolution reconstruction on the surgical video to obtain a target surgical video with a higher resolution than the preset one.

[0083] In an embodiment of the present application, taking into account that there may be a certain displacement or rotation between adjacent frame images in the surgical video to be segmented, a one-to-one correspondence between adjacent frame images in the surgical video to be segmented is first found before super-resolution is performed. In this way, the target surgical video obtained by super-resolution reconstruction is more accurate.

[0084] In some embodiments, the process of dynamically segmenting the blood vessels in the surgical video to be segmented by the trained blood vessel segmentation model in the above step S150 to obtain a real-time contour map of the blood vessels in the surgical video to be segmented may specifically include the following steps:

[0085] Based on a one-to-one correspondence between adjacent frame images in the surgical video to be segmented, adjacent frame images containing the same vascular object in the target surgical video are determined as related frame images;

[0086] The blood vessels in the target surgical video are dynamically segmented using the trained vascular segmentation model. When segmenting the blood vessels in the relevant frame images, the blood vessels in each relevant frame image are dynamically segmented based on the vascular segmentation results of multiple relevant frame images of the previous frame and the next frame of each relevant frame image, so as to maintain the correlation of the dynamic vascular segmentation results and obtain a real-time contour map of the blood vessels in the surgical video to be segmented.

[0087] In practical applications, adjacent frames of a video may contain the same blood vessel objects. When segmenting these related frames containing the same blood vessel objects, in order to maintain the relevance of the segmentation results, the segmentation of a certain frame refers to the segmentation results of the previous and next frames.

[0088] In an embodiment of the present application, segmentation based on multiple frames of previous and subsequent frames can effectively reduce the probability of being affected by adverse factors during the shooting process when decomposing based on a single frame, and improve the accuracy of segmentation. It can be seen that finding and utilizing the one-to-one correspondence between adjacent frames before segmentation can help improve the consistency of the segmentation results.

[0089] In some embodiments, the above image registration algorithm is performed by at least one of the following steps:

[0090] For deformations greater than a preset deformation degree in the image of the surgical video to be segmented, the energy function of the Beltrami coefficient term is minimized, and the Beltrami coefficient corresponding to the bijective and smooth mapping is obtained by minimizing the energy function; wherein the Beltrami coefficient term is used to measure the degree of distortion of the quasi-conformal mapping and control the bijectivity and smoothness of the mapping;

[0091] Priori markers are obtained through marking operations. For deformations greater than a preset degree in the image of the surgical video to be segmented, the registration between the images is calculated based on the marking constraints corresponding to the priori markers to obtain a registration result that maintains orientation homeomorphism.

[0092] It should be noted that the Beltrami coefficient is an important concept in quasi-conformal mapping, which is used to describe the transition from conformal mapping to quasi-conformal mapping. In conformal transformation, the infinitesimal circle of a surface is mapped to the infinitesimal circle of another surface, keeping the ratio of angles and lengths unchanged. In quasi-conformal mapping, the infinitesimal ellipse of a surface is mapped to the infinitesimal circle of another surface, and the angle remains unchanged, but the length ratio can change slightly. This change is quantified by the Beltrami coefficient, which reflects the degree of distortion of the mapping. The geometric meaning of the Beltrami coefficient can be interpreted as: the eccentricity and direction of the infinitesimal ellipse, which is given by the mapping of a small disk area on a surface to an elliptical disk area on another surface. The eccentricity and direction of the elliptical domain determine the specific form of the mapping, namely the Beltrami coefficient. This coefficient is not equivalent to the Jacobi matrix of the mapping, and it provides less information than the Jacobi matrix, but it is sufficient to replicate the original mapping. This can be understood in terms of the generalized Riemann mapping theorem, which states that given a measurable complex-valued function, there exists a homeomorphism from the unit disk to itself such that the Beltrami equations hold, and different mappings differ from each other by a Mobius transformation on the unit disk.

[0093] The embodiment of the present application provides a method for image registration based on quasi-conformal geometry, that is, a method for image registration that uses quasi-conformal mapping to obtain an orientation-preserving mapping, which can handle large deformations. In the embodiment of the present application, the process of finding the correspondence between images can be achieved through image registration. In medical image processing, registration can be applied to finding the correspondence between medical images, performing shape analysis on anatomical structures, and the like.

[0094] Most existing registration algorithms can accurately and efficiently calculate registration when the deformation of the object in the image is small. However, when there is a large deformation, the registration problem becomes challenging. The registration results obtained using traditional methods often have folding, and one-to-one mapping cannot be guaranteed. For example, as shown in Figure 5, in the registration result with folding, Figure 5 (a) and Figure 5 (b) are two registered images, and Figure 5 (c) is the image after registration, that is, after finding the mapping of the two images, the mapping is applied to Figure 5 (b) to obtain the result; obvious folding can be seen below Figure 5 (c); Figure 5 (d) is the grid corresponding to the mapping.

[0095] As an example, an embodiment of the present application proposes a method for image registration using quasi-conformal mapping to obtain an orientation-preserving mapping, which can handle larger deformations in the image. The key process of the method is to minimize the energy function containing the Beltrami coefficient term, where the Beltrami coefficient term measures the degree of distortion of the quasi-conformal mapping and can effectively control the bijectivity and smoothness of the mapping. Even if there is a very large deformation in the image, by minimizing the energy functional, the Beltrami coefficient corresponding to the bijective and smooth mapping can always be obtained.

[0096] For example, Figure 6 As shown in the figure, the registration results of the image registration algorithm based on quasi-conformal geometry on the ACDC heart dataset: the first and second columns are the two registered images, and the third column is the registered image, that is, after finding the mapping of the first and second columns, the mapping is applied to the second column. It can be seen that the third column is very similar to the first column; the fourth column is the grid corresponding to these mappings.

[0097] As another example, the image registration algorithm can also introduce prior markers given by professionals (such as doctors). Even if there are large deformations or a large number of marker constraints are given, the algorithm can effectively calculate the registration between images, and the obtained registration is guaranteed to maintain orientation homeomorphism. By applying the algorithm to scenarios such as medical image registration and image jitter removal, better experimental results can be obtained, verifying the feasibility of the algorithm.

[0098] For example, Figure 7 As shown in the figure, in the image registration result based on the given prior marker information, the experiment requires finding the mapping between (A) and (B) based on the given 6 marker points, so as to map (A) to (B). (C) is the result of the experiment. It can be seen that even if the distortion is very large, this algorithm can still ensure that the resulting mapping does not have folding.

[0099] In some embodiments, the surgical images are divided into a training set and a test set, and the training set is used to train the initial blood vessel segmentation model; after the above step S140, the following steps are also included in a loop until the optimal segmentation model is obtained after adjustment:

[0100] The trained segmentation model is used as the segmentation model to be tested, and the performance of the segmentation model to be tested is tested using the test set to obtain the test result;

[0101] The trained segmentation model is adjusted according to the test result to obtain an adjusted segmentation model, and the adjusted segmentation model is used as a new segmentation model to be tested.

[0102] For the training process of the segmentation model, the data is exemplarily divided into a training set and a test set, the preprocessed training set is input into the constructed segmentation model for training, and the performance of the segmentation model is evaluated, and the model is adjusted according to the evaluation results. By repeatedly performing this step, the optimal model file can be obtained more effectively to improve the accuracy of the segmentation model.

[0103] In some embodiments, after the above step S120, the method may further include the following steps:

[0104] The surgical image is grayed to eliminate the color information of the surgical image and obtain a grayed surgical image. The distribution of pixel values ​​in the grayed surgical image is adjusted by histogram equalization to obtain a histogram with a smoothness greater than a preset smoothness. The noise in the histogram is reduced by a Gaussian filter to make the blood vessel area in the histogram stand out.

[0105] In the embodiment of the present application, in order to eliminate or suppress unnecessary noise in the image, highlight the vascular area, and improve the accuracy of vascular segmentation, the surgical image is first preprocessed. The image preprocessing methods used include grayscale, histogram equalization, and Gaussian filtering. In order to reduce the amount of calculation in the later stage, the image is first grayscaled to eliminate color information. Secondly, the distribution of pixel values ​​in the image is adjusted by histogram equalization, so as to obtain a smooth histogram in the output image. Finally, the noise in the image is reduced by a Gaussian filter.

[0106] In some embodiments, after the above step S120, the method may further include the following steps: performing data enhancement on the surgical image by means of rotation, horizontal mirroring, adding noise, brightness transformation, and random translation to obtain a data enhancement result. By performing data enhancement on the image by means of random rotation, horizontal mirroring, adding noise, brightness transformation, random translation, etc., the surgical image can obtain an effective data enhancement effect.

[0107] Figure 8A schematic diagram of a dynamic segmentation device for pulmonary vessels based on surgical videos is provided. Figure 8 As shown, the pulmonary vascular dynamic segmentation device 800 based on surgical video includes:

[0108] The acquisition module 801 is used to acquire the original surgical video, obtain the original image of the original surgical video by taking screenshots of alternate frames, and extract the video frame sequence of the original surgical video;

[0109] A determination module 802 is used to determine complementary information between adjacent frame images in the original surgical video according to the video frame sequence and the original image, and perform super-resolution reconstruction based on the complementary information to obtain a surgical image with a resolution higher than a preset high resolution;

[0110] A construction module 803 is used to construct an initial blood vessel segmentation model based on TransUNet; the initial blood vessel segmentation model includes an encoder and a decoder, the segmentation model is used to extract blood vessel features through a convolutional neural network, generate a blood vessel feature map, convert the blood vessel feature map into a blood vessel sequence map, and capture global context information based on the blood vessel sequence map through the encoder, and restore the spatial resolution of the output of the encoder through the decoder;

[0111] The training module 804 is used to train the initial blood vessel segmentation model using the surgical image to upsample the blood vessel feature map from a preset low resolution to the resolution of the surgical image, and directly combine the target blood vessel feature map in the encoder that is higher than the preset high resolution with the global context feature through a jump connection using the encoder and the decoder to obtain a trained blood vessel segmentation model; the result of combining the target blood vessel feature map with the global context feature is used to dynamically identify and locate the position of the blood vessel in the surgical image in real time;

[0112] The segmentation module 805 is used to input the surgical video to be segmented into the trained blood vessel segmentation model, and dynamically segment the blood vessels in the surgical video to be segmented using the trained blood vessel segmentation model to obtain a real-time contour map of the blood vessels in the surgical video to be segmented.

[0113] The dynamic segmentation device for pulmonary blood vessels based on surgical videos provided in the embodiments of the present application has the same technical features as the dynamic segmentation method for pulmonary blood vessels based on surgical videos provided in the above embodiments, so it can also solve the same technical problems and achieve the same technical effects.

[0114] An electronic device provided in an embodiment of the present application is Fig. 9As shown, the electronic device 900 includes a processor 902 and a memory 901, wherein the memory stores a computer program that can be run on the processor, and the processor implements the steps of the method provided in the above embodiment when executing the computer program.

[0115] See also Fig. 9 The electronic device further includes: a bus 903 and a communication interface 904, a processor 902, a communication interface 904 and a memory 901 are connected via the bus 903; the processor 902 is used to execute an executable module stored in the memory 901, such as a computer program.

[0116] The memory 901 may include a high-speed random access memory (RAM), and may also include a non-volatile memory, such as at least one disk storage. The communication connection between the system network element and at least one other network element is realized through at least one communication interface 904 (which may be wired or wireless), and the Internet, wide area network, local area network, metropolitan area network, etc. may be used.

[0117] The bus 903 may be an ISA bus, a PCI bus, or an EISA bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig. 9 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0118] Among them, the memory 901 is used to store programs, and the processor 902 executes the program after receiving the execution instruction. The method executed by the device defined by the process disclosed in any embodiment of the present application can be applied to the processor 902 or implemented by the processor 902.

[0119] The processor 902 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 902. The above processor 902 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiments of the present application can be directly embodied as a hardware decoding processor to execute, or the hardware and software modules in the decoding processor are combined to execute. The software module may be located in a storage medium mature in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 901, and the processor 902 reads the information in the memory 901 and completes the steps of the above method in combination with its hardware.

[0120] Corresponding to the above-mentioned dynamic segmentation method of pulmonary blood vessels based on surgical videos, an embodiment of the present application also provides a computer-readable storage medium, which stores computer-executable instructions. When the computer-executable instructions are called and executed by a processor, the computer-executable instructions prompt the processor to execute the steps of the above-mentioned dynamic segmentation method of pulmonary blood vessels based on surgical videos.

[0121] The pulmonary vascular dynamic segmentation device based on surgical video provided in the embodiment of the present application can be specific hardware on the device or software or firmware installed on the device. The implementation principle and technical effects of the device provided in the embodiment of the present application are the same as those of the aforementioned method embodiment. For the sake of brief description, for matters not mentioned in the device embodiment, reference can be made to the corresponding contents in the aforementioned method embodiment. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, devices and units described above can all refer to the corresponding processes in the aforementioned method embodiment, and will not be repeated here.

[0122] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0123] For another example, the flowchart and block diagram in the accompanying drawings show the possible architecture, function and operation of the device, method and computer program product according to multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of the code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or the flowchart, and the combination of the boxes in the block diagram and / or the flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0124] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0125] In addition, each functional unit in the embodiments provided in the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0126] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the dynamic segmentation method of pulmonary blood vessels based on surgical videos described in various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, and other media that can store program codes.

[0127] It should be noted that similar numbers and letters represent similar items in the following figures. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. In addition, the terms "first", "second", "third", etc. are only used to distinguish the description and are not to be understood as indicating or implying relative importance.

[0128] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present application, which are used to illustrate the technical solution of the present application, rather than to limit it. The protection scope of the present application is not limited thereto. Although the present application is described in detail with reference to the aforementioned embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solution recorded in the aforementioned embodiments within the technical scope disclosed in the present application, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solution deviate from the scope of the technical solution of the embodiment of the present application. They should all be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A dynamic segmentation method of pulmonary blood vessels based on surgical videos, characterized in that: The method comprises: Obtaining an original surgical video, obtaining an original image of the original surgical video by taking screenshots of alternate frames, and extracting a video frame sequence of the original surgical video; Determining complementary information between adjacent frame images in the original surgical video according to the video frame sequence and the original image, and performing super-resolution reconstruction based on the complementary information to obtain a surgical image with a resolution higher than a preset high resolution; Constructing an initial blood vessel segmentation model based on TransUNet; the initial blood vessel segmentation model comprises an encoder and a decoder, the segmentation model is used to extract blood vessel features through a convolutional neural network, generate a blood vessel feature map, convert the blood vessel feature map into a blood vessel sequence map, and capture global context information based on the blood vessel sequence map through the encoder, and restore the spatial resolution of the encoder output through the decoder; The initial blood vessel segmentation model is trained using the surgical image to upsample the blood vessel feature map from a preset low resolution to the resolution of the surgical image, and the target blood vessel feature map in the encoder that is higher than the preset high resolution is directly combined with the global context feature by using the encoder and the decoder through a jump connection to obtain a trained blood vessel segmentation model; the result of combining the target blood vessel feature map with the global context feature is used to dynamically identify and locate the position of the blood vessel in the surgical image in real time; The surgical video to be segmented is input into the trained blood vessel segmentation model, and the blood vessels in the surgical video to be segmented are dynamically segmented by the trained blood vessel segmentation model to obtain a real-time contour map of the blood vessels in the surgical video to be segmented.

2. The method according to claim 1, characterized in that Before dynamically segmenting the blood vessels in the surgical video to be segmented by using the trained blood vessel segmentation model to obtain a real-time contour map of the blood vessels in the surgical video to be segmented, the method further includes: Determine the one-to-one correspondence between adjacent frame images in the surgical video to be segmented by an image registration algorithm; wherein there is displacement or rotation between adjacent frame images in the surgical video to be segmented; The surgical video to be segmented is reconstructed with super resolution by utilizing the one-to-one correspondence between adjacent frame images in the surgical video to be segmented, so as to obtain a target surgical video with a resolution higher than the preset high resolution.

3. The method according to claim 2, characterized in that The dynamically segmenting the blood vessels in the surgical video to be segmented by using the trained blood vessel segmentation model to obtain a real-time contour map of the blood vessels in the surgical video to be segmented includes: Based on a one-to-one correspondence between adjacent frame images in the surgical video to be segmented, adjacent frame images in the target surgical video containing the same blood vessel object are determined as related frame images; The blood vessels in the target surgical video are dynamically segmented by using the trained blood vessel segmentation model, and when segmenting the blood vessels in the relevant frame images, the blood vessels in each relevant frame image are dynamically segmented based on the blood vessel segmentation results of multiple relevant frame images of the previous frame and the next frame of each relevant frame image, so as to maintain the correlation of the dynamic blood vessel segmentation results and obtain a real-time contour map of the blood vessels in the surgical video to be segmented.

4. The method according to claim 2, characterized in that: The image registration algorithm is performed by the following steps: For deformations greater than a preset deformation degree in the image of the surgical video to be segmented, the energy function of the Beltrami coefficient term is minimized, and the Beltrami coefficient corresponding to the bijective and smooth mapping is obtained by minimizing the energy function; wherein the Beltrami coefficient term is used to measure the degree of distortion of the quasi-conformal mapping and control the bijectivity and smoothness of the mapping; and / or, A priori markers are obtained through a marking operation, and for deformations greater than a preset deformation degree in the image of the surgical video to be segmented, the registration between the images is calculated based on the marking constraints corresponding to the priori markers to obtain a registration result that maintains orientation homeomorphism.

5. The method according to claim 1, characterized in that The surgical images are divided into a training set and a test set, wherein the training set is used to train the initial blood vessel segmentation model; after obtaining the trained segmentation model, the following steps are also performed in a loop until an optimal segmentation model is obtained after adjustment: Using the trained segmentation model as the segmentation model to be tested, and using the test set to test the performance of the segmentation model to be tested to obtain a test result; The trained segmentation model is adjusted according to the test result to obtain an adjusted segmentation model, and the adjusted segmentation model is used as a new segmentation model to be tested.

6. The method according to claim 1, characterized in that After the super-resolution reconstruction is performed based on the complementary information to obtain a surgical image with a resolution higher than a preset high resolution, the method further includes: gray-scale the surgical image to eliminate color information of the surgical image and obtain a gray-scale surgical image; The distribution of pixel values ​​in the grayscale surgical image is adjusted by histogram equalization to obtain a histogram with a smoothness greater than a preset smoothness, and the noise in the histogram is reduced by a Gaussian filter to make the blood vessel area in the histogram stand out.

7. The method according to claim 1, characterized in that The encoder is a CNN-Transformer hybrid encoder, which includes multi-head attention and multi-layer perceptron; The decoder comprises a plurality of upsampling steps, each of which comprises an upsampling operation, a convolution operation and a ReLU activation layer.

8. The method according to claim 1, characterized in that: After the super-resolution reconstruction is performed based on the complementary information to obtain a surgical image with a resolution higher than a preset high resolution, the method further includes: The surgical image is data enhanced by means of rotation, horizontal mirroring, noise addition, brightness transformation and random translation to obtain a data enhancement result.

9. A pulmonary vascular dynamic segmentation device based on surgical video, characterized in that: The device comprises: An acquisition module is used to acquire the original surgical video, obtain the original image of the original surgical video by taking screenshots of alternate frames, and extract the video frame sequence of the original surgical video; A determination module, configured to determine complementary information between adjacent frame images in the original surgical video according to the video frame sequence and the original image, and to perform super-resolution reconstruction based on the complementary information to obtain a surgical image with a resolution higher than a preset high resolution; A construction module is used to construct an initial blood vessel segmentation model based on TransUNet; the initial blood vessel segmentation model includes an encoder and a decoder, the segmentation model is used to extract blood vessel features through a convolutional neural network, generate a blood vessel feature map, convert the blood vessel feature map into a blood vessel sequence map, and capture global context information based on the blood vessel sequence map through the encoder, and restore the spatial resolution of the output of the encoder through the decoder; A training module, used to train the initial blood vessel segmentation model using the surgical image to upsample the blood vessel feature map from a preset low resolution to the resolution of the surgical image, and to combine the target blood vessel feature map in the encoder that is higher than the preset high resolution with the global context feature directly through a jump connection using the encoder and the decoder to obtain a trained blood vessel segmentation model; the result of combining the target blood vessel feature map with the global context feature is used to dynamically identify and locate the position of the blood vessel in the surgical image in real time; The segmentation module is used to input the surgical video to be segmented into the trained blood vessel segmentation model, dynamically segment the blood vessels in the surgical video to be segmented through the trained blood vessel segmentation model, and obtain a real-time contour map of the blood vessels in the surgical video to be segmented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions. When the computer-executable instructions are called and executed by a processor, the computer-executable instructions prompt the processor to execute the method according to any one of claims 1 to 8.

Images